In this paper, we propose a modulation-adaptive acoustic gesture recognition system with smartphones (termed, MAA), which can achieve a high recognition accuracy under various modulation schemes and quickly adapt to a new modulation at low cost. Specifically, MAA creates an acoustic channel model to capture temporal and spatial features, and leverages a domain adversarial network to eliminate the difference among modulation schemes when performing the same gesture. The proposed framework includes a data collection module, a signal preprocessing module, a channel construction module, and a domain adaptation module. For data collection, we determine the appropriate signal length and bandwidth for transmitted acoustic signals. In the signal preprocessing module, channel estimation and background noise removal are incorporated. Then, we develop a tensor reconstruction network and a feature mapping network in the channel construction module to directly map features to specific gestures. For domain adaptation, we train the above two networks in the source domain, and use an adversarial network to adapt to the target domain. Experimental results show that the proposed MAA achieves a good performance on gesture recognition with different modulation schemes, with better adaptation to new modulation schemes than several state-of-the-art baselines.
