Smart Augmented Reality Instructional System for Mechanical Assembly towards Worker-Centered Intelligent Manufacturing

Abstract

Quality and efficiency are crucial indicators of any manufacturing company. Many companies are suffering from a shortage of experienced workers across the production line to perform complex assembly tasks. To reduce time and error in an assembly task, a worker-centered system consisting of multi-modal Augmented Reality (AR) instructions with the support of a deep learning network for tool detection is introduced. The integrated AR is designed to provide on-site instructions including various visual renderings with a fine-tuned Region-based Convolutional Neural Network, which is trained on a synthetic tool dataset. The dataset is generated using CAD models of tools and displayed onto a 2D scene without using real tool images. By experimenting the system to a mechanical assembly of a CNC carving machine, the result of a designed experiment shows that the system helps reduce the time and errors of the given assembly tasks by 33.2 % and 32.4 %, respectively. With the integrated system, an efficient, customizable smart AR instruction system capable of sensing, characterizing requirements, and enhancing worker's performance has been built and demonstrated.

Department(s)

Mechanical and Aerospace Engineering

Second Department

Computer Science

Comments

National Science Foundation, Grant 1954548

Keywords and Phrases

Augmented reality; Deep learning; Intellegent manufacturing; Mechanical assembly; Object detection; Region-based Convolutional Neural Networks (R-CNN)

International Standard Serial Number (ISSN)

0278-6125

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2020 Society of Manufacturing Engineers (SME), All rights reserved.

Publication Date

01 Apr 2020

Link to Full Text

Share

 
COinS