Paper to appear in Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) (Ubicomp’18).
Abstract: In this paper, we propose Auracle, a wearable earpiece that can automatically recognize eating behavior. More specifically, in free-living conditions, we can recognize when and for how long a person is eating. Using an off-the-shelf contact microphone placed behind the ear, Auracle captures the sound of a person chewing as it passes through the bone and tissue of the head. This audio data is then processed by a custom analog/digital circuit board. To ensure reliable (yet comfortable) contact between microphone and skin, all hardware components are incorporated into a 3D-printed behind-the-head framework. We collected field data with 14 participants for 32 hours in free-living conditions and additional eating data with 10 participants for 2 hours in a laboratory setting. We achieved accuracy exceeding 92.8% and F1 score exceeding 77.5% for eating detection. Moreover, Auracle successfully detected 20-24 eating episodes (depending on the metrics) out of 26 in free-living conditions. We demonstrate that our custom device could sense, process, and classify audio data in real time. Additionally, we estimate Auracle can last 28.1 hours with a 110 mAh battery while communicating its observations of eating behavior to a smartphone over Bluetooth.
Shengjie Bi, Tao Wang, Nicole Tobias, Josephine Nordrum, Shang Wang, George Halvorsen, Sougata Sen, Ronald Peterson, Kofi Odame, Kelly Caine, Ryan Halter, Jacob Sorber, and David Kotz. Auracle: Detecting Eating Episodes with an Ear-Mounted Sensor. Proceedings of the ACM on Interactive, Mobile, Wearable and Ubiquitous Technologies (IMWUT) (Ubicomp), 2(3), September 2018. DOI 10.1145/3264902.
The Auracle team presented an innovative new technique for interacting with wearable devices, during the UIST conference last week.
Abstract: We present Pyro, a micro thumb-tip gesture recognition technique based on thermal infrared signals radiating from the fingers. Pyro uses a compact, low-power passive sensor, making it suitable for wearable and mobile applications. To demonstrate the feasibility of Pyro, we developed a self-contained prototype consisting of the infrared pyroelectric sensor, a custom sensing circuit, and software for signal processing and machine learning. A ten-participant user study yielded a 93.9% cross-validation accuracy and 84.9% leave-one-session-out accuracy on six thumb-tip gestures. Subsequent lab studies demonstrated Pyro’s robustness to varying light conditions, hand temperatures, and background motion. We conclude by discussing the insights we gained from this work and future research questions.
J. Gong, Y. Zhang, X. Zhou, and X.-D. Yang, “Pyro: Thumb-Tip gesture recognition using pyroelectric infrared sensing,” in Proceedings of the Annual ACM Symposium on User Interface Software and Technology (UIST). ACM Press, Oct. 2017, pp. 553-563. Available: http://dx.doi.org/10.1145/3126594.3126615
A large subset of the Auracle team gathered in Hanover for a very productive full-day retreat, mapping out the coming year’s research agenda and making plans for new prototypes and papers. Great to have everyone (almost everyone) in one place!
David Kotz was recently quoted in a brief story about the Auracle project on the Engineering Innovation Podcast and Radio Series from the National Academy of Engineering, hosted by WTOP News. Listen on their website.
Abstract: Researchers strive to understand eating behavior as a means to develop diets and interventions that can help people achieve and maintain a healthy weight, recover from eating disorders, or manage their diet and nutrition for personal wellness. A major challenge for eating-behavior research is to understand when, where, what, and how people eat. In this paper, we evaluate sensors and algorithms designed to detect eating activities, more specifically, when people eat. We compare two popular methods for eating recognition (based on acoustic and electromyography (EMG) sensors) individually and combined. We built a data-acquisition system using two off-the-shelf sensors and conducted a study with 20 participants. Our preliminary results show that the system we implemented can detect eating with an accuracy exceeding 90.9% while the crunchiness level of food varies. We are developing a wearable system that can capture, process, and classify sensor data to detect eating in real-time.
Obesity is one of the most pressing health challenges faced by our country, and has been the target of much attention in the academic and commercial mobile health (mHealth) community. Despite the community’s significant effort in developing technology to measure physical activity (in an effort to estimate caloric output), little progress has been made in measuring eating and drinking behavior (caloric intake) – and yet the science of obesity indicates that diet is a major factor in behavioral change to encourage weight loss and healthy weight management.
In the Auracle project we plan to develop a digital earpiece – small and comfortable enough to wear in or behind the ear – that can sense and detect actions such as eating, drinking, smoking, and speaking, and measure physiological stress. The project’s long-term vision is that computational jewelry like this earpiece will enable behavioral-health researchers to better understand health-related behaviors and, subsequently, to support the validation and deployment of effective behavioral-health interventions that promote healthy diet and behavior.
The project’s approach is to build a prototype wireless earpiece, small enough to wear behind the ear, with low-power (microwatt-scale) electronics and software sufficient to allow for the battery to last a full waking day; to develop efficient algorithms for detecting and distinguishing health-related behaviors (eating, drinking, smoking, speaking, and stress); and to develop effective means for the wearer to interact with the earpiece and its applications.