Abstract
This project aims to create a framework of training engineers and policy makers on robotic operation and image analysis for the inspection and preservation of transportation infrastructure. This framework includes a method for camera-based bridge inspection data collection, algorithms for data processing and pattern recognitions, and tools for inspection and preservation decision-making. Specifically, a Siamese Neural Network was developed to support bridge engineers in analyzing big video data. The network was initially trained by one-shot learning and fine-tuned iteratively with human in the loop. The neural network algorithm retrieved all related regions from the video based on the regions of interest defined by bridge inspectors. The neural network was evaluated on three bridge inspection videos with promising performances. In addition, an assistive intelligence system was developed to help inspectors efficiently and accurately detect and segment multiclass bridge elements from inspection videos. A Mask Region-based Convolutional Neural Network was transferred in the studied problem with a small initial training dataset labeled by the inspector. Then, the temporal coherence analysis was used to recover false negative detections of the transferred network. Finally, self-training with a guidance from experienced inspectors was used to iteratively refine the network. Results from a case study have demonstrated that the proposed method uses just a small amount of time and guidance from experienced inspectors to successfully build the assistive intelligence system with excellent outcomes.
Recommended Citation
Qin, Ruwen; Chen, Genda; Yin, Zhaozheng; Karim, Muhammad Monjurul; Zhao, Tianyi; Long, Suzanna; and Louis, Sushil J., "Final Report - A Training Framework of Robotic Operation and Image Analysis for Decision-Making in Bridge Inspection and Preservation" (2020). Project WD-1. 2.
https://scholarsmine.mst.edu/project_wd-1/2
Department(s)
Civil, Architectural and Environmental Engineering
Second Department
Engineering Management and Systems Engineering
Third Department
Computer Science
Research Center/Lab(s)
INSPIRE - University Transportation Center
Grant Number
USDOT # 69A3551747126
Sponsor(s)
Office of the Assistant Secretary for Research and Technology U.S. Department of Transportation 1200 New Jersey Avenue, SE Washington, DC 20590
Keywords and Phrases
Deep Learning, Computer Vision, Big Data Analytics, Bridge Inspection, Worker Assistance
Report Number
INSPIRE-007
Document Type
Technical Report
Document Version
Final Version
File Type
text
Rights
© 2025 Missouri University of Science and Technology, All rights reserved.
Publication Date
July 31, 2020
Comments
Principal Investigator: Ruwen Qin
Co-Principal Investigators: Genda Chen, Suzanna Long, Zhaozheng Yin, Sushil Louis
Grant Period: 11/30/2016 - 09/30/2024
Project Period: 01/01/2018 - 09/30/2020
The investigation was conducted in cooperation with the U. S. Department of Transportation