Abstract
Empowered by the advanced 3D sensing, computer vision and AI algorithm, autonomous robotics provide an unprecedented possibility for close-up infrastructure environment inspection in an efficient and reliable fashion. Deep neural network (DNN) learning algorithms, pretrained on the large database can empower real-time object detection as well as fully autonomous, safe robotic navigation in unstructured environments while avoiding the potential obstacle. However, the development and deployment of the robots, inspection planning and operation procedures are still tedious and segmented with tremendous manual intervention during environmental inspection and anomaly monitoring. The proposed digital twin approach is able to provide a virtual representation model of the target environment either from a build-design or from 3D scanning of the real-world physical assets at high resolution in the Unity simulation environment, a transverse drone robot model and test its Robotics Operating System(ROS) autonomous navigation and obstacle avoidance software stack, and the hardware-in-the-loop test can thus be conducted for the flight control algorithm effectiveness and real-time object detection performance evaluation. The preliminary result shows that VGG16-UNet deep learning algorithm was able to use only a small amount of guidance and time from experienced inspection pilots to successfully identify the critical elements and defects and real-time navigate around the unstructured environment. The proposed digital twin framework and methodology is promising to be utilized for developing and testing fully autonomous inspection robots and its path planning and navigation and detection operation with greater cost- and time-efficiency.
Recommended Citation
J. Li et al., "Hardware-in-the-loop And Digital Twin Enabled Autonomous Robotics-assisted Environment Inspection," 2023 6th International Symposium on Autonomous Systems, ISAS 2023, Institute of Electrical and Electronics Engineers, Jan 2023.
The definitive version is available at https://doi.org/10.1109/ISAS59543.2023.10164352
Department(s)
Civil, Architectural and Environmental Engineering
Keywords and Phrases
bridge inspection; Crack detection; Deep learning; Digital twin; hardware-in-the-loop
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2023 Institute of Electrical and Electronics Engineers, All rights reserved.
Publication Date
01 Jan 2023
Comments
National Science Foundation, Grant 2139025#