University of Sussex
Browse

File(s) not publicly available

S-SMART: a unified Bayesian framework for Simultaneous Semantic Mapping, Activity Recognition and Tracking

Version 2 2023-06-12, 06:36
Version 1 2023-06-08, 22:28
journal contribution
posted on 2023-06-12, 06:36 authored by Michael Hardegger, Daniel RoggenDaniel Roggen, Alberto Calatroni, Gerhard Tröster
The machine recognition of user trajectories and activities is fundamental to devise context-aware applications for support and monitoring in daily life. So far, tracking and activity recognition were mostly considered as orthogonal problems, which limits the richness of possible context inference. In this work, we introduce the novel unified computational and representational framework S-SMART that simultaneously models the environment state (semantic mapping), localizes the user within this map (tracking), and recognizes interactions with the environment (activity recognition). Thus, S-SMART identifies which activities the user executes where (e.g., turning a handle next to a window), and reflects the outcome of these actions by updating the world model (e.g., the window is now open). This in turn conditions the future possibility of executing actions at specific places (e.g., closing the window is likely to be the next action at this location). S-SMART works in a self-contained manner, and iteratively builds the semantic map from wearable sensors only. This enables the seamless deployment to new environments. We characterize S-SMART in an experimental dataset with people performing hand actions as part of their usual routines at home and in office buildings. The framework combines dead reckoning from a footworn motion sensor with template-matching-based action recognition, identifying objects in the environment (windows, doors, water taps, phones, etc.) and tracking their state (open/closed, etc.). In real-life recordings with up to 23 action classes, S-SMART consistently outperforms independent systems for positioning and activity recognition, and constructs accurate semantic maps. This environment representation enables novel applications that build upon information about the arrangement and state of the user’s surroundings. For example, it may be possible to remind elderly people of a window that they left open before leaving the house, or of a plant they did not water yet, using solely wearable sensors.

Funding

CuPiD; EU FP7; 288516

History

Publication status

  • Published

File Version

  • Published version

Journal

ACM Transactions on Intelligent Systems and Technology

ISSN

2157-6904

Publisher

ACM

Issue

3

Volume

7

Page range

34

Department affiliated with

  • Engineering and Design Publications

Full text available

  • No

Peer reviewed?

  • Yes

Legacy Posted Date

2015-09-14

First Compliant Deposit (FCD) Date

2015-09-11

Usage metrics

    University of Sussex (Publications)

    Categories

    No categories selected

    Exports

    RefWorks
    BibTeX
    Ref. manager
    Endnote
    DataCite
    NLM
    DC