Skip to Main content Skip to Navigation
Conference papers

How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review

Astrid Bertrand 1, 2, 3 Rafik Belloum 4, 2, 5 James Eagan 4, 2, 5 Winston Maxwell 6, 2, 3 
1 ECO-Télécom Paris - Equipe Eco Economie
I3, une unité mixte de recherche CNRS (UMR 9217) - Institut interdisciplinaire de l’innovation
4 DIVA - Design, Interaction, Visualization & Applications
LTCI - Laboratoire Traitement et Communication de l'Information
6 NOS - Numérique, Organisation et Société
I3, une unité mixte de recherche CNRS (UMR 9217) - Institut interdisciplinaire de l’innovation
Abstract : The field of eXplainable Artificial Intelligence (XAI) aims to bring transparency to complex AI systems. Although it is usually considered an essentially technical field, effort has been made recently to better understand users' human explanation methods and cognitive constraints. Despite these advances, the community lacks a general vision of what and how cognitive biases affect explainability systems. To address this gap, we present a heuristic map which matches human cognitive biases with explainability techniques from the XAI literature, structured around XAI-aided decision-making. We identify four main ways cognitive biases affect or are affected by XAI systems: 1) cognitive biases affect how XAI methods are designed, 2) they can distort how XAI techniques are evaluated in user studies, 3) some cognitive biases can be successfully mitigated by XAI techniques, and, on the contrary, 4) some cognitive biases can be exacerbated by XAI techniques. We construct this heuristic map through the systematic review of 37 papers-drawn from a corpus of 285-that reveal cognitive biases in XAI systems, including the explainability method and the user and task types in which they arise. We use the findings from our review to structure directions for future XAI systems to better align with people's cognitive processes.
Complete list of metadata

https://hal.telecom-paris.fr/hal-03684457
Contributor : Rafik Belloum Connect in order to contact the contributor
Submitted on : Wednesday, June 1, 2022 - 1:57:45 PM
Last modification on : Wednesday, August 10, 2022 - 2:53:08 PM
Long-term archiving on: : Friday, September 2, 2022 - 7:10:36 PM

File

How Cognitive Biases Affect XA...
Files produced by the author(s)

Identifiers

Collections

Citation

Astrid Bertrand, Rafik Belloum, James Eagan, Winston Maxwell. How Cognitive Biases Affect XAI-assisted Decision-making: A Systematic Review. AAAI/ACM Conference on Artificial intelligence, Ethics, and Society, Aug 2022, Oxford, United Kingdom. ⟨10.1145/3514094.3534164⟩. ⟨hal-03684457⟩

Share

Metrics

Record views

252

Files downloads

322