Assessing accuracy and consistency in intracranial aneurysm sizing : human expertise vs. artifcial intelligence

Planinc, Andrej (author), Špegel, Nina (author), Podobnik, Zala (author), Šinigoj, Uroš (author), Skubic, Petra (author), Choi, June Ho (author), Park, Wonhyoung (author), Robič, Tina (author), Tabor, Nika (author), Jarabek, Leon (author), Špiclin, Žiga (author), Bizjak, Žiga (author)

Intracranial aneurysms (IAs) are a common vascular pathology and are associated with a risk of rupture, which is often fatal. Aneurysm growth of more than 1 mm is considered a surrogate of rupture risk, therefore, this study presents a comprehensive analysis of intracranial aneurysm measurements utilizing a dataset comprising 358 IA from 248 computed tomography angiography (CTA) scans measured by four junior raters and one senior rater. The study explores the variability in sizing assessments by employing both human raters and an Artificial Intelligence (AI) system. Our findings reveal substantial inter- and intra-rater variability among junior raters, contrasting with the lower intra-rater variability observed in the senior rater. Standard deviations of all raters were above the threshold for IA growth (1 mm). Additionally, the study identifies a systemic bias, indicating a tendency for human experts to measure aneurysms smaller than the AI system. Our findings emphasize the challenges in human assessment while also showcasing the capacity of AI technology to improve the precision and reliability of intracranial aneurysm assessments, especially beneficial for junior raters. The potential of AI was particularly evident in the task of monitoring IA at various intervals, where the AI-based approach surpassed junior raters and achieved performance comparable to senior raters.

• Language: English
• Keywords: intrarater variability, interrater variability, deep learning, intracranial aneurysms
• Work type: Article
• Typology: 1.01 - Original Scientific Article
• Organization: FE - Faculty of Electrical Engineering; MF - Faculty of Medicine
• Publication status: Published
• Publication version: Version of Record
• Publisher: Springer Nature
• Year: 2024
• Number of pages: 11 str.
• Numbering: 14, art. 16080
• PID: 20.500.12556/RUL-160094
• UDC: 004.8:616-007.64
• ISSN on article: 2045-2322
• DOI: 10.1038/s41598-024-65825-4
• COBISS.SI-ID: 204579587
• Publication date in RUL: 20.08.2024

Record is a part of a journal

• Title: Scientific reports
• Shortened title: Sci. rep.
• Publisher: Nature Publishing Group
• ISSN: 2045-2322
• COBISS.SI-ID: 18727432

Licences

• License: CC BY 4.0, Creative Commons Attribution 4.0 International
• Link: http://creativecommons.org/licenses/by/4.0/
• Description: This is the standard Creative Commons license that gives others maximum freedom to do what they want with the work as long as they credit the author.

Secondary language

• Language: Slovenian
• Keywords: variabilnost ocenjevalcev, globoko učenje, intrakranialne anevrizme

Projects

• Funder: ARIS - Slovenian Research and Innovation Agency
• Project number: J2-3059
• Name: Sprotno prilagajanje načrta protonske in radioterapije