Heuristics and Errors in XAI-Augmented Clinical Decision-Making: Moving Beyond Algorithmic Appreciation and Aversion Moving Beyond Algorithmic Appreciation and Aversion

How do physicians integrate AI tools into medical decision-making? Prior research has analyzed extensively whether they exhibit AI algorithmic aversion or appreciation. Yet we argue that these behavioral outcomes arise from underlying decision-making heuristics such as pro-innovation bias, ambiguity aversion, or commitment bias. In this qualitative study, we examined 330 clinical decisions using “think aloud” protocols to identify heuristics employed with AI and explainable AI (XAI). We observed the presence of multiple heuristics, including a “mere exposure effect” and “false confirmation bias”. These heuristics were associated with decision-making errors. The “mere exposure effect” occurred commonly with XAI, when physicians, feeling uncertain about their diagnoses, altered their decision to an incorrect AI diagnosis. False confirmation errors also emerged when AI confirmed an erroneous diagnosis, precluding doctors from seeking alternative information. We also discuss how cognitive interventions could redress these heuristics in decision-making to better optimize accuracy.