“In a worker-centered intelligent manufacturing system, sensing and understanding of the worker’s behavior are the primary tasks, which are essential for automatic performance evaluation & optimization, intelligent training & assistance, and human-robot collaboration. In this study, a worker-centered training & assistant system is proposed for intelligent manufacturing, which is featured with self-awareness and active-guidance. To understand the hand behavior, a method is proposed for complex hand gesture recognition using Convolutional Neural Networks (CNN) with multiview augmentation and inference fusion, from depth images captured by Microsoft Kinect. To sense and understand the worker in a more comprehensive way, a multi-modal approach is proposed for worker activity recognition using Inertial Measurement Unit (IMU) signals obtained from a Myo armband and videos from a visual camera. To automatically learn the importance of different sensors, a novel attention-based approach is proposed to human activity recognition using multiple IMU sensors worn at different body locations. To deploy the developed algorithms to the factory floor, a real-time assembly operation recognition system is proposed with fog computing and transfer learning. The proposed worker-centered training & assistant system has been validated and demonstrated the feasibility and great potential for applying to the manufacturing industry for frontline workers. Our developed approaches have been evaluated: 1) the multi-view approach outperforms the state-of-the-arts on two public benchmark datasets, 2) the multi-modal approach achieves an accuracy of 97% on a worker activity dataset including 6 activities and achieves the best performance on a public dataset, 3) the attention-based method outperforms the state-of-the-art methods on five publicly available datasets, and 4) the developed transfer learning model achieves a real-time recognition accuracy of 95% on a dataset including 10 worker operations”--Abstract, page iv.
Leu, M. C. (Ming-Chuan)
Liou, Frank W.
Mechanical and Aerospace Engineering
Ph. D. in Mechanical Engineering
Intelligent Systems Center
Missouri University of Science and Technology
Journal article titles appearing in thesis/dissertation
- A self-aware and active-guiding training & assistant system for worker-centered intelligent manufacturing
- American sign language alphabet recognition using convolutional neural networks with multiview augmentation and inference fusion
- Multi-modal recognition of worker activity for human-centered intelligent manufacturing
- Attention-based sensor fusion for human activity recognition using IMU signals
- Real-time assembly operation recognition with fog computing and transfer learning for human-centered intelligent manufacturing
xvii, 146 pages
© 2020 Wenjin Tao, All rights reserved.
Dissertation - Open Access
Electronic OCLC #
Tao, Wenjin, "Human behavior understanding for worker-centered intelligent manufacturing" (2020). Doctoral Dissertations. 2922.