This paper introduces RFVibe, a system that enables contactless material and object identification through the fusion of millimeter wave wireless signals with acoustic signals. In particular, RFVibe plays an audio sound next to the object that generates micro-vibrations in the object. These micro-vibrations can be captured by shining a millimeter wave radar signal on the object and analyzing the phase of the reflected wireless signal. RFVibe can then extract several features including resonance frequencies and vibration modes, damping time of vibrations, and wireless reflection coefficients. These features are then used to enable more accurate identification, with a step towards generalizing towards different setups and locations. We implement RFVibe using an off-the-shelf millimeter-wave radar and an acoustic speaker. We evaluate it on 23 objects of 7 material types (Metal, Wood, Ceramic, Glass, Plastic, Cardboard, and Foam), obtaining 81.3% accuracy for material classification, a 30% improvement over prior work. RFVibe is able to classify with reasonable accuracy in scenarios that it has not encountered before, including different locations, angles, boundary conditions, and objects.
Paper
Contactless Material Identification with Millimeter Wave Vibrometry
Hailan Shanbhag, Sohrab Madani, Akhil Isanaka, Deepak Nair, Saurabh Gupta, Haitham Hassanieh
The 21st ACM International Conference on Mobile Systems, Applications, and Services (MobiSys), 2023
[PDF]
@inproceedings{10.1145/3581791.3596850,
author = {Shanbhag, Hailan and Madani, Sohrab and Isanaka, Akhil and Nair, Deepak and Gupta, Saurabh and Hassanieh, Haitham},
title = {Contactless Material Identification with Millimeter Wave Vibrometry},
year = {2023},
isbn = {9798400701108},
publisher = {Association for Computing Machinery},
address = {New York, NY, USA},
url = {https://doi.org/10.1145/3581791.3596850},
doi = {10.1145/3581791.3596850},
booktitle = {Proceedings of the 21st Annual International Conference on Mobile Systems, Applications and Services},
pages = {475–488},
numpages = {14},
keywords = {object classification, wireless vibrometry, material classification, millimeter-wave sensing},
location = {Helsinki, Finland},
series = {MobiSys ’23}
}