Object recognition creates new opportunities for smartwatch apps
11 November 2015
Technology developed by Carnegie Mellon University and Disney Research could enable smartwatches to automatically recognise what objects users are touching.
The technique, called EM-Sense, takes advantage of the body’s natural electrical conductivity to detect whether a person is touching an electrical or electromechanical device and, based on the distinctive electromagnetic noise emitted by such devices, automatically identify the object.
A smartwatch equipped with EM-Sense would have a much more detailed understanding of what the user is doing than is possible with common mobile sensors, such as accelerometers or pulse monitors.
“We are now able to gain a greater contextual understanding of user activities by recognising what objects they are interacting with,” says Alanson Sample, a research scientist at Disney Research.
EM-Sense is able to differentiate between scores of objects based on the ambient electromagnetic noise they emit, so objects need not be modified or have RFID tags for the technique to work.
“By extending the Internet to physical objects — what’s being called the Internet of Things (IoT) — we are creating new ways for people to interact with the world around them,” says Jessica Hodgins, vice president of Disney Research. “EM-Sense can make the IoT experience even richer by enabling people to get information or additional functionality simply by touching everyday objects.”
The human body serves as an antenna for EM-Sense. From any body part an object touches, its distinctive electromagnetic emissions propagate through the body to an electrode worn at the wrist. Kitchen appliances, power tools, electronic scales and door handles with electrically triggered locks are among the items that can be detected and identified.
According to Gierad Laput, a doctoral student in Carnegie Mellon’s Human-Computer Interaction Institute, the proof-of-concept sensor was built with off-the-shelf components – principally a small, software-defined radio. A smartwatch would provide the perfect platform, he adds, because the watch is in contact with the skin and also houses a highly capable computer for deciphering signals and running apps.
The researchers are able to filter out background noise, so the technique is able to recognise powered devices at any location. The technique can also work with large, non-powered conductive objects, such as doors, ladders and other structural elements.
YouTube video clip courtesy of Carnegie Mellon University and Disney Research