Abstract
The role of human-machine teams in society is increasing, as big data and computing power explode. One popular approach to AI is deep learning, which is useful for classification, feature identification, and predictive modeling. However, deep learning models often suffer from inadequate transparency and poor explainability. One aspect of human systems integration is the design of interfaces that support human decision-making. AI models have multiple types of uncertainty embedded, which may be difficult for users to understand. Humans that use these tools need to understand how much they should trust the AI. This study evaluates one simple approach for communicating uncertainty, a visual confidence bar ranging from 0-100%. We perform a human-subject online experiment using an existing image recognition deep learning model to test the effect of (1) providing single vs. multiple recommendations from the AI and (2) including uncertainty information. For each image, participants described the subject in an open textbox and rated their confidence in their answers. Performance was evaluated at four levels of accuracy ranging from the same as the image label to the correct category of the image. The results suggest that AI recommendations increase accuracy, even if the human and AI have different definitions of accuracy. In addition, providing multiple ranked recommendations, with or without the confidence bar, increases operator confidence and reduces perceived task difficulty. More research is needed to determine how people approach uncertain information from an AI system and develop effective visualizations for communicating uncertainty.
Recommended Citation
H. V. Subramanian et al., "Communicating Uncertain Information from Deep Learning Models in Human Machine Teams," Proceedings of the American Society for Engineering Management 2020 International Annual Conference, American Society for Engineering Management (ASEM), Oct 2020.
Meeting Name
41st Annual Meeting of the American Society for Engineering Management (ASEM) and the Virtual International Annual Conference, vIAC (2020: Oct. 28-30, Virtual)
Department(s)
Engineering Management and Systems Engineering
Second Department
Psychological Science
Keywords and Phrases
Human Systems Integration; Recommendation System; Artificial Intelligence; Uncertainty; Human Machine Team
Document Type
Article - Conference proceedings
Document Version
Final Version
File Type
text
Language(s)
English
Rights
© 2020 American Society for Engineering Management (ASEM), All rights reserved.
Publication Date
30 Oct 2020
Included in
Artificial Intelligence and Robotics Commons, Operations Research, Systems Engineering and Industrial Engineering Commons, Psychology Commons
