Position : explainable AI cannot advance without better user studies

Pičulin, Matej (avtor), Petek, Bernarda (avtor), Ograjenšek, Irena (avtor), Štrumbelj, Erik (avtor)

In this position paper, we argue that user studies are key to understanding the value of explainable AI methods, because the end goal of explainable AI is to satisfy societal desiderata. We also argue that the current state of user studies is detrimental to the advancement of the field. We support this argument with a review of general and explainable AI-specific challenges, as well as an analysis of 607 explainable AI papers featuring user studies. We demonstrate how most user studies lack reproducibility, discussion of limitations, comparison with a baseline, or placebo explanations and are of low fidelity to real-world users and application context. This, combined with an overreliance on functional evaluation, results in a lack of understanding of the value explainable AI methods, which hinders the progress of the field. To address this issue, we call for higher methodological standards for user studies, greater appreciation of high-quality user studies in the AI community, and reduced reliance on functional evaluation.

• Jezik: Angleški jezik
• Ključne besede: explainable artificial intelligence, explanation of artificial intelligence, evaluation of artificial intelligence, users, social aspects
• Tipologija: 1.08 - Objavljeni znanstveni prispevek na konferenci
• Organizacija: FRI - Fakulteta za računalništvo in informatiko
• Status publikacije: Objavljeno
• Različica publikacije: Objavljena publikacija
• Leto izida: 2025
• Št. strani: 18 str.
• PID: 20.500.12556/RUL-183622
• UDK: 004.8
• COBISS.SI-ID: 255898115

Avtorske pravice

Informacija o licenci CC BY 4.0 je bila pridobljena s strani avtorjev članka. (Datum opombe: 22. 6. 2026)

• Datum objave v RUL: 22.06.2026

Gradivo je del monografije

• Naslov: International Conference on Machine Learning, 13-19 July 2025, Vancouver Convention Center, Vancouver, Canada
• Uredniki: Aarti Singh, Maryam Fazel, Daniel Hsu, Simon Lacoste-Julien, Felix Berkenkamp, Tegan Maharaj, Kiri Wagstaff, Jerry Zhu
• Kraj izida: Cambridge (MA)
• Založnik: PMLR
• Leto izida: 2025
• COBISS.SI-ID: 255897859
• Naslov zbirke: Proceedings of machine learning research
• Številčenje v zbirki: Vol. 267
• ISSN zbirke: 2640-3498

Licence

• Licenca: CC BY 4.0, Creative Commons Priznanje avtorstva 4.0 Mednarodna
• Povezava: http://creativecommons.org/licenses/by/4.0/deed.sl
• Opis: To je standardna licenca Creative Commons, ki daje uporabnikom največ možnosti za nadaljnjo uporabo dela, pri čemer morajo navesti avtorja.

Sekundarni jezik

• Jezik: Slovenski jezik
• Ključne besede: razlaga umetne inteligence, razložljiva umetna inteligenca, vrednotenje umetne inteligence, uporabniki, družbeni vidiki

Projekti

• Financer: ARRS - Agencija za raziskovalno dejavnost Republike Slovenije
• Številka projekta: P2-0442
• Naslov: Podatkovne vede in digitalna preobrazba