University of Sussex
Browse
untitled.pdf (5.04 MB)

Transportation mode recognition fusing wearable motion, sound and vision sensors

Download (5.04 MB)
Version 2 2023-06-12, 09:26
Version 1 2023-06-09, 21:03
journal contribution
posted on 2023-06-12, 09:26 authored by Sebastien Richoz, Lin Wang, Phil BirchPhil Birch, Daniel RoggenDaniel Roggen
We present the first work that investigates the potential of improving the performance of transportation mode recognition through fusing multimodal data from wearable sensors: motion, sound and vision. We first train three independent deep neural network (DNN) classifiers, which work with the three types of sensors, respectively. We then propose two schemes that fuse the classification results from the three mono-modal classifiers. The first scheme makes an ensemble decision with fixed rules including Sum, Product, Majority Voting, and Borda Count. The second scheme is an adaptive fuser built as another classifier (including Naive Bayes, Decision Tree, Random Forest and Neural Network) that learns enhanced predictions by combining the outputs from the three mono-modal classifiers. We verify the advantage of the proposed method with the state-of-the-art Sussex-Huawei Locomotion and Transportation (SHL) dataset recognizing the eight transportation activities: Still, Walk, Run, Bike, Bus, Car, Train and Subway. We achieve F1 scores of 79.4%, 82.1% and 72.8% with the mono-modal motion, sound and vision classifiers, respectively. The F1 score is remarkably improved to 94.5% and 95.5% by the two data fusion schemes, respectively. The recognition performance can be further improved with a post-processing scheme that exploits the temporal continuity of transportation. When assessing generalization of the model to unseen data, we show that while performance is reduced - as expected - for each individual classifier, the benefits of fusion are retained with performance improved by 15 percentage points. Besides the actual performance increase, this work, most importantly, opens up the possibility for dynamically fusing modalities to achieve distinct power-performance trade-off at run time.

Funding

Activity Sensing Technologies for Mobile Users; G2015; Huawei

History

Publication status

  • Published

File Version

  • Published version

Journal

IEEE Sensors Journal

ISSN

1530-437X

Publisher

IEEE

Issue

16

Volume

20

Page range

9314-9328

Department affiliated with

  • Engineering and Design Publications

Research groups affiliated with

  • Sensor Technology Research Centre Publications

Full text available

  • No

Peer reviewed?

  • Yes

Legacy Posted Date

2020-04-07

First Open Access (FOA) Date

2020-04-16

First Compliant Deposit (FCD) Date

2020-04-06

Usage metrics

    University of Sussex (Publications)

    Categories

    No categories selected

    Licence

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC