Doctoral Dissertations

Wenjin Tao

Abstract

“In a worker-centered intelligent manufacturing system, sensing and understanding of the worker’s behavior are the primary tasks, which are essential for automatic performance evaluation & optimization, intelligent training & assistance, and human-robot collaboration. In this study, a worker-centered training & assistant system is proposed for intelligent manufacturing, which is featured with self-awareness and active-guidance. To understand the hand behavior, a method is proposed for complex hand gesture recognition using Convolutional Neural Networks (CNN) with multiview augmentation and inference fusion, from depth images captured by Microsoft Kinect. To sense and understand the worker in a more comprehensive way, a multi-modal approach is proposed for worker activity recognition using Inertial Measurement Unit (IMU) signals obtained from a Myo armband and videos from a visual camera. To automatically learn the importance of different sensors, a novel attention-based approach is proposed to human activity recognition using multiple IMU sensors worn at different body locations. To deploy the developed algorithms to the factory floor, a real-time assembly operation recognition system is proposed with fog computing and transfer learning. The proposed worker-centered training & assistant system has been validated and demonstrated the feasibility and great potential for applying to the manufacturing industry for frontline workers. Our developed approaches have been evaluated: 1) the multi-view approach outperforms the state-of-the-arts on two public benchmark datasets, 2) the multi-modal approach achieves an accuracy of 97% on a worker activity dataset including 6 activities and achieves the best performance on a public dataset, 3) the attention-based method outperforms the state-of-the-art methods on five publicly available datasets, and 4) the developed transfer learning model achieves a real-time recognition accuracy of 95% on a dataset including 10 worker operations”--Abstract, page iv.

Advisor(s)

Leu, M. C. (Ming-Chuan)

Committee Member(s)

Yin, Zhaozheng
Qin, Ruwen
He, Zhihai
Liou, Frank W.
Chandrashekhara, K.

Department(s)

Mechanical and Aerospace Engineering

Degree Name

Ph. D. in Mechanical Engineering

Comments

This research work is supported by the National Science Foundation grants CMMI- 1646162 and NRI-1830479, and also by the Intelligent Systems Center at Missouri University of Science and Technology.

Research Center/Lab(s)

Intelligent Systems Center

Publisher

Missouri University of Science and Technology

Publication Date

Summer 2020

Journal article titles appearing in thesis/dissertation

  • A self-aware and active-guiding training & assistant system for worker-centered intelligent manufacturing
  • American sign language alphabet recognition using convolutional neural networks with multiview augmentation and inference fusion
  • Multi-modal recognition of worker activity for human-centered intelligent manufacturing
  • Attention-based sensor fusion for human activity recognition using IMU signals
  • Real-time assembly operation recognition with fog computing and transfer learning for human-centered intelligent manufacturing

Pagination

xvii, 146 pages

Note about bibliography

Includes bibliographic references.

Rights

© 2020 Wenjin Tao, All rights reserved.

Document Type

Dissertation - Open Access

File Type

text

Language

English

Thesis Number

T 11759

Electronic OCLC #

1198499046

Download

Share

 
COinS