Multimodal fusion of IMUs and EPS body worn sensors for scratch recognition

Jocys, Zygimantas, Pour Yazdan Panah Kermani, Arash and Roggen, Daniel (2020) Multimodal fusion of IMUs and EPS body worn sensors for scratch recognition. EAI PervasiveHealth 2020 - 14th EAI International Conference on Pervasive Computing Technologies for Healthcare, Atlanta, United States, October 6-8, 2020. Published in: Badram, Jakob, Chung, Chia-Fang, Jacobs, Maia, Morrisey, Kellie, Orji, Rita, Song, Junehwa and Yarosh, Lana, (eds.) Proceedings of Pervasive Health 2020. ACM (Accepted)

[img] PDF - Accepted Version
Restricted to SRO admin only

Download (8MB)


Itching may be caused by different skin diseases. In order to develop and evaluate how much itching affects a person’s daily life, it is useful to develop automated means to recognize the action of scratching. We present an investigation of sensors and algorithms to realize a wearable scratch detection device.We collected a dataset, where each user wore 4 IMU (Inertial Measurement Units) sensors and one EPS (electric potential sensor). Data was collected from 9 users, where each user followed a 40 minute protocol, which involved scratching different parts of head, shoulder and leg, as well as other activities such as walking, drinking water, brushing teeth and sitting next to the computer. The dataset contains 813 scratching instances and 5h 15 min of recorded data. We investigate trade-offs between number of devices worn, and hence comfort, and recognition performance.We trained k-NN and Random Forest using between 1 to 5 of the best features per channel to detect scratches. We conclude that scratch can be detected with 80.7% by using Random Forest on hand coordinates, which require 4 devices. However, f1 score of 70% can be achieved with k-NN with IMU and EPS data, which only requires 1 device. Moreover, fusion of IMU data with EPS data improved the accuracy and reduced the standard deviation between the folds. This expands the state of the art by opening up new trade-offs between accuracy and comfort for future research in skin conditions.

Item Type: Conference Proceedings
Schools and Departments: School of Engineering and Informatics > Engineering and Design
School of Engineering and Informatics > Informatics
Research Centres and Groups: Sensor Technology Research Centre
Subjects: Q Science > Q Science (General) > Q0300 Cybernetics
T Technology > TK Electrical engineering. Electronics Nuclear engineering
Depositing User: Daniel Roggen
Date Deposited: 13 Mar 2020 08:45
Last Modified: 02 Sep 2020 09:07

View download statistics for this item

📧 Request an update
Project NameSussex Project NumberFunderFunder Ref
Automated recognition of hair touch and scalp scratching through novel sensing modalities, sensor fusion and machine learning approachesG2738UNILEVERMA-2017-02004N