HuMAn: Complex Activity Recognition with Multi-Modal Multi-Positional Body Sensing
Abstract
Current state-of-the-art systems in literature using wearables are not capable of distinguishing large number of fine-grained and/or complex human activities, which may appear similar but with vital differences in context, such as lying on floor versus lying on bed versus lying on sofa. This paper fills the gap by proposing a novel system, called HuMAn, that recognizes and classifies complex at-home activities of humans with wearable sensing. Specifically, HuMAn makes such classification feasible by leveraging selective multi-modal sensor suites from wearable devices, and enhances the richness of sensed information for activity classification by further leveraging placement of wearables across multiple positions on human body. The HuMAn system consists of the following components: (a) a practical feature set extraction method from specific multi-modal sensor suites; (b) a novel two-level structured classification algorithm that improves accuracy by leveraging sensors in multiple body positions; and (c) improved refinement in classification of complex activities with minimal external infrastructure support (e.g., only a few Bluetooth beacons used for location context). The proposed system is evaluated with 10 users in real home environments. Experimental results demonstrate that the HuMAn system can detect 21 complex at-home activities with high degree of accuracy. For same-user evaluation strategy, the average activity classification accuracy is as high as 95 percent over all of the 21 activities. For the case of 10-fold cross-validation evaluation strategy, the average classification accuracy is 92 percent, and for the case of leave-one-out cross-validation strategy, the average classification accuracy is 75 percent.
Recommended Citation
P. Bharti et al., "HuMAn: Complex Activity Recognition with Multi-Modal Multi-Positional Body Sensing," IEEE Transactions on Mobile Computing, vol. 18, no. 4, pp. 857 - 870, Institute of Electrical and Electronics Engineers (IEEE), Apr 2019.
The definitive version is available at https://doi.org/10.1109/TMC.2018.2841905
Department(s)
Computer Science
Research Center/Lab(s)
Intelligent Systems Center
Second Research Center/Lab
Center for High Performance Computing Research
Keywords and Phrases
Pattern recognition; Statistical methods; Structured programming; Wearable technology; 10-fold cross-validation; Activity classifications; Classification accuracy; Classification algorithm; Complex activity; Conditional random fields; Leave-one-out cross validations; Multimodal sensor; Classification (of information); Activity recognition; Biomedical monitoring; Complex activity recognition; Feature extraction; Legged locomotion; Smart health; Smartphone multi-modal sensors; Wrist
International Standard Serial Number (ISSN)
1536-1233; 1558-0660
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2019 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
01 Apr 2019
Comments
This work was supported in part by the US National Science Foundation (NSF) under Award Nos. IIS-1404673, IIP1648907, IIP-1540119, IIS-1254117 and II-New-1205695.