Masters Theses

Keywords and Phrases

Artificial Intelligence; Human Machine Teams; Human Systems Integration; Recommendation System; Uncertainty

Abstract

“The use of Artificial Intelligence (AI) decision support systems is increasing in high-stakes contexts, such as healthcare, defense, and finance. Uncertainty information may help users better leverage AI predictions, especially when combined with domain knowledge. I conducted two human-subject experiments to examine the effects of uncertainty information with AI recommendations. The experimental stimuli are from an existing image recognition deep learning model, one popular approach to AI. In Paper I, I evaluated the effect of the number of AI recommendations and provision of uncertainty information. For a series of images, participants identified the subject and rated their confidence level. Results suggest that AI recommendations, especially multiple, increased accuracy and confidence. However, uncertainty information, which was represented visually with bars, did not significantly improve participants' performance. In Paper II, I tested the effect of AI recommendations in a within-subject comparison and the effect of more salient uncertainty information in a between-subject comparison in the context of varying domain knowledge. The uncertainty information combined both numerical (percent) and visual (color-coded bar) formats to make the information easier to interpret and more noticeable. Consistent with Paper I, results suggest that AI recommendations improved participants’ accuracy and confidence. In addition, the more salient uncertainty information significantly increased accuracy, but not confidence. Based on a subjective measure of domain knowledge, participants had higher domain knowledge for animals. In general, AI recommendations and uncertainty information had less of an effect as domain knowledge increased. Results suggest that uncertainty information, can improve accuracy and potentially decrease over-confidence”--Abstract, page iv.

Advisor(s)

Canfield, Casey I.

Committee Member(s)

Shank, Daniel Burton
Dagli, Cihan H., 1949-

Department(s)

Engineering Management and Systems Engineering

Degree Name

M.S. in Engineering Management

Comments

The authors would also like to thank the National Science Foundation (NSF) for partially funding this project.

Publisher

Missouri University of Science and Technology

Publication Date

Summer 2021

Journal article titles appearing in thesis/dissertation

  • Communicating Uncertain Information from Deep Learning Models in Human Machine Teams
  • Role of Uncertainty Information and Domain Knowledge in Use of AI Recommendations

Pagination

x, 68 pages

Note about bibliography

Includes bibliographic references.

Rights

© 2021 Harishankar Vasudevanallur Subramanian, All rights reserved.

Document Type

Thesis - Open Access

File Type

text

Language

English

Thesis Number

T 11927

Electronic OCLC #

1286687006

Download

Share

 
COinS