Design of a Real-Time Human-Robot Collaboration System using Dynamic Gestures

Abstract

With the development of industrial automation and artificial intelligence, robotic systems are developing into an essential part of factory production, and the human-robot collaboration (HRC) becomes a new trend in the industrial field. In our previous work, ten dynamic gestures have been designed for communication between a human worker and a robot in manufacturing scenarios, and a dynamic gesture recognition model based on Convolutional Neural Networks (CNN) has been developed. Based on the model, this study aims to design and develop a new real-time HRC system based on multi-threading method and the CNN. This system enables the real-time interaction between a human worker and a robotic arm based on dynamic gestures. Firstly, a multi-threading architecture is constructed for high-speed operation and fast response while schedule more than one task at the same time. Next, A real-time dynamic gesture recognition algorithm is developed, where a human worker's behavior and motion are continuously monitored and captured, and motion history images (MHIs) are generated in real-time. The generation of the MHIs and their identification using the classification model are synchronously accomplished. If a designated dynamic gesture is detected, it is immediately transmitted to the robotic arm to conduct a real-time response. A Graphic User Interface (GUI) for the integration of the proposed HRC system is developed for the visualization of the real-time motion history and classification results of the gesture identification. A series of actual collaboration experiments are carried out between a human worker and a six-degree-of-freedom (6 DOF) Comau industrial robot, and the experimental results show the feasibility and robustness of the proposed system.

Meeting Name

ASME International Mechanical Engineering Congress and Exposition, IMECE 2020 (2020: Nov. 16-19, Virtual)

Department(s)

Mechanical and Aerospace Engineering

Comments

National Science Foundation, Grant CMMI-1954548

Keywords and Phrases

Human-robot collaboration; Motion history image; Multiple threads; Real-time system

International Standard Book Number (ISBN)

978-079188449-2

Document Type

Article - Conference proceedings

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2020 American Society of Mechanical Engineers (ASME), All rights reserved.

Publication Date

19 Nov 2020

Link to Full Text

Share

 
COinS