Uporabnost umetne inteligence pri obravnavi kliničnih primerov pacientov

Vodopivc, Žan (author), Horvat, Nejc (mentor), Knez, Lea (co-mentor)

Umetna inteligenca je opredeljena kot »sposobnost stroja, računalnika, da rešuje umske probleme«. Vključuje dele človeške inteligence, kot so sklepanje, reševanje problemov, načrtovanje, učenje, odzivanje, razumevanje in ustvarjanje jezika. Zaradi svojih zmogljivosti bi lahko bila uporabna tudi pri farmacevtskih kognitivnih storitvah, kot je npr. farmakoterapijski pregled. Namen magistrske naloge je bil ovrednotiti uporabnost umetne inteligence pri prepoznavi in reševanju težav, povezanih z zdravili. Zanimalo nas je predvsem, kako vsebinsko pravilni, razumljivi, celoviti, jezikovno pravilni in argumentirani so po mnenju kliničnih farmacevtov odgovori umetne inteligence ter kakšen je odnos kliničnih farmacevtov do uporabe umetne inteligence. Razvili smo pet kliničnih primerov pacientov z različnimi boleznimi in zdravili. Za vsak primer smo pripravili dve splošni vprašanji in šest do deset specifičnih vprašanj. Namen splošnih vprašanj je bil, da ugotovimo, ali umetna inteligenca prepozna težave v zdravljenju z zdravili in potrebne ukrepe brez naših podvprašanj. Specifična vprašanja pa so vezana na določeno terapevtsko področje. V času izvedbe raziskave je bil za namen naše študije uporaben samo veliki jezikovni model ChatGPT. Temu smo zastavili primere in vprašanja. S pomočjo spletnega orodja 1ka so klinične farmacevtke ocenjevale pravilnost, razumljivost, celovitost, jezik in argumentiranost odgovorov ChatGPT na Likertovi lestvici strinjanja 1–5. Vsak primer sta ocenjevali dve klinični farmacevtki. Rezultati nakazujejo, da je pravilnost odgovorov ChatGPT podobna pri odgovarjanju na splošna in na specifična vprašanja. Ko so klinične farmacevtke ocenjevale pravilnost odgovorov ChatGPT na splošna vprašanja, je bila povprečna ocena 3,40. Povprečna ocena odgovorov na specifična vprašanja je bila 3,17. Na splošno je ChatGPT dobro poznal zdravilne učinkovine (indikacije, odmerjanje, neželene učinke), medtem ko je imel težave s prepoznavanjem nekaterih generičnih zdravil. V pomoč je bil pri osnovnem prepoznavanju težav, povezanih z zdravljenjem. Pri splošnih vprašanjih je navajal precej univerzalne nasvete v povezavi z ohranjanjem zdravja. V odgovorih je zelo pogosto poudarjal, da je priporočljivo poiskati nasvet zdravnika ali farmacevta. Razumljivost odgovorov ChatGPT je bila ocenjena s povprečno oceno 3,83, celovitost z 2,36, jezikovna pravilnost s 3,61 in argumentiranost z 2,47. Na koncu anket je sledilo pet vprašanj v povezavi z odnosom do umetne inteligence. S pomočjo teh smo ugotovili, da ni bilo statističnih razlik v ocenah pravilnosti odgovorov ChatGPT med kliničnimi farmacevtkami, ki podpirajo, in tistimi, ki ne podpirajo uporabe umetne inteligence. Pokazalo se je, da je tudi ChatGPT zmotljiv, zaradi česar je pomembno pridobljene informacije obravnavati kritično in jih potrditi še z drugimi viri.

• Language: Slovenian
• Keywords: umetna inteligenca, ChatGPT, klinična farmacija, pregled zdravljenja z zdravili, obravnava kliničnih primerov pacientov
• Work type: Master's thesis/paper
• Organization: FFA - Faculty of Pharmacy
• Year: 2024
• PID: 20.500.12556/RUL-155918
• Publication date in RUL: 24.04.2024

Secondary language

• Language: English
• Title: The usefulness of artificial intelligence in the management of clinical patient cases

Abstract

Artificial intelligence is defined as the ability of machines to learn from experience, adjust to new inputs and perform human-like tasks. The aim of the master thesis was to evaluate the usefulness of artificial intelligence at identifying and solving drug-related problems. Its capabilities could also make it useful for pharmaceutical cognitive services. Specifically, we were interested in how correct, understandable, comprehensive, linguistically correct and well-reasoned were artificial intelligence answers as perceived by clinical pharmacists. Additionally, we sought to gather insights from clinical pharmacists regarding their attitudes toward the use of artificial intelligence. We developed five patient cases from different medical fields. For each case, we formulated two general questions and six to ten specific questions. We designed general questions to find out whether artificial intelligence can identify problems in drug treatment without our sub-questions. The specific questions were linked to a particular therapeutic area. During our research, only ChatGPT proved useful for our purposes. We posed questions to ChatGPT and utilized its responses. Using the web tool 1ka clinical pharmacists rated the correctness, understandability, comprehensiveness, language and reasoning on a Likert scale from one to five. Two clinical pharmacists assessed each case. The results suggest that the correctness of ChatGPT responses were similar when answering general and specific questions. When clinical pharmacists rated the rightness of ChatGPT responses to general questions on the average score was 3.40. The mean score for answers to specific questions was 3.17. In general, ChatGPT had a good knowledge about active substances (indications, dosages, side effects), while it had more difficulties identifying some generic names. It was helpful in the basic identification of treatment-related problems. On general issues, it mostly gave universal advices in connection with maintaining health. ChatGPT often emphasized the importance of consulting a doctor or pharmacist. The average score for the understandability of ChatGPT answers was 3.83, for the comprehensiveness 2.36, for the linguistic correctness 3.61, and the average score for the reasoning was 2.47. At the end of the survey, we included questions for clinical pharmacists about their attitudes towards artificial intelligence. We found no significant difference in the correctness ratings of ChatGPT responses between clinical pharmacists who supported and those who did not support the use of artificial intelligence. It turned out that ChatGPT is also fallible, which is why it is important to look critically at the information obtained and corroborate it with other sources.

• Keywords: artificial intelligence, ChatGPT, clinical pharmacy, drug treatment review, clinical case management