INSPIRE Archived Webinars

Assistive Intelligence (AI): Intelligent Data Analytics Algorithms to Assist Human Experts


Media is loading

Source Publication Title

INSPIRE-University Transportation Center Webinars

Webinar Date

30 Jan 2019, 11:00 am


Artificial Intelligence, particularly deep learning, has recently received increasing attention in many applications, such as image classification, speech recognition, and computer games. The success of deep learning algorithms requires big annotated datasets for training, gradient-based optimization algorithms, and powerful computational resources. In the case of civil infrastructure inspection, we can collect big data from different imaging sensors such as color, thermal, and hyperspectral cameras. Three issues encounter in this application. First, it is tedious and expensive to let human experts annotate the datasets to train deep learning algorithms. Second, the offline trained deep learning algorithms may not be able to adapt to new civil infrastructures. Third and lastly, the trained deep learning algorithm works like a black box on new data, without the domain knowledge from human experts. In this project, we investigate intelligent data analytics algorithms with human experts in the loop, called Assistive Intelligence (AI). Using the bridge inspection as a case study, we aim to find regions-of-interest (e.g., joints with damages) over long video sequences. The data analytics algorithm is initially trained from a small set of data. Given the dataset of a new bridge, bridge experts only need to annotate a few region-of-interest examples as the seed; our algorithm will retrieve corresponding examples in the rest of videos. Human experts can also return some incorrectly retrieved samples to the data analytics algorithm for further refinement. Thus, while the data analytics algorithm can assist human in an efficient way, bridge experts can leverage their domain knowledge in the adaptation of the computational tool in different scenarios.


Dr. Zhaozheng Yin received his PhD in Computer Science and Engineering from Penn State in 2009 and received his Master and Bachelor degrees at the University of Wisconsin-Madison and Tsinghua University. He is specialized in Computer Vision, Image Processing and Machine Learning, with broad applications in structure health monitoring. Dr. Yin has been a faculty member in Computer Science at Missouri S&T since September 2011. He is a recipient of CVPR Best Doctoral Spotlight Award (2009), MICCAI Young Scientist Award (2012, finalist in 2015 and 2010), NSF CAREER Award (2014), Department Outstanding Junior Faculty Research Award (2014) and Missouri S&T Faculty Research Award (2014, 2017), Best Paper Award of CVPR Workshop on Understanding Hands in Actions (2015). He is a Daniel St. Clair Faculty Fellow in the Computer Science department since 2015, and a Dean’s Scholar in the College of Engineering and Computing since 2016.


Civil, Architectural and Environmental Engineering

Second Department

Computer Science

Research Center/Lab(s)

INSPIRE - University Transportation Center

Document Type

Video - Presentation

Document Version

Final Version

File Type





© 2019 Missouri University of Science and Technology, All rights reserved.

This document is currently not available here.