Design of a Real-Time Human-Robot Collaboration System using Dynamic Gestures
With the development of industrial automation and artificial intelligence, robotic systems are developing into an essential part of factory production, and the human-robot collaboration (HRC) becomes a new trend in the industrial field. In our previous work, ten dynamic gestures have been designed for communication between a human worker and a robot in manufacturing scenarios, and a dynamic gesture recognition model based on Convolutional Neural Networks (CNN) has been developed. Based on the model, this study aims to design and develop a new real-time HRC system based on multi-threading method and the CNN. This system enables the real-time interaction between a human worker and a robotic arm based on dynamic gestures. Firstly, a multi-threading architecture is constructed for high-speed operation and fast response while schedule more than one task at the same time. Next, A real-time dynamic gesture recognition algorithm is developed, where a human worker's behavior and motion are continuously monitored and captured, and motion history images (MHIs) are generated in real-time. The generation of the MHIs and their identification using the classification model are synchronously accomplished. If a designated dynamic gesture is detected, it is immediately transmitted to the robotic arm to conduct a real-time response. A Graphic User Interface (GUI) for the integration of the proposed HRC system is developed for the visualization of the real-time motion history and classification results of the gesture identification. A series of actual collaboration experiments are carried out between a human worker and a six-degree-of-freedom (6 DOF) Comau industrial robot, and the experimental results show the feasibility and robustness of the proposed system.
H. Chen et al., "Design of a Real-Time Human-Robot Collaboration System using Dynamic Gestures," Proceedings of the ASME International Mechanical Engineering Congress and Exposition, vol. 2B - 2020, American Society of Mechanical Engineers (ASME), Nov 2020.
The definitive version is available at https://doi.org/10.1115/IMECE2020-23650
ASME International Mechanical Engineering Congress and Exposition, IMECE 2020 (2020: Nov. 16-19, Virtual)
Mechanical and Aerospace Engineering
Keywords and Phrases
Human-robot collaboration; Motion history image; Multiple threads; Real-time system
International Standard Book Number (ISBN)
Article - Conference proceedings
© 2020 American Society of Mechanical Engineers (ASME), All rights reserved.
19 Nov 2020