Statistični kazalniki kakovosti in učinkovitosti oskrbe pacientov s koronarno boleznijo

Bijec, Janez (avtor), Ograjenšek, Irena (mentor), Došenović Bonča, Petra (komentor)

V magistrskem delu predstavljam možnost uporabe podatkov o obračunanih zdravstvenih obravnavah iz podatkovnih skladišč Zavoda za zdravstveno zavarovanje Slovenije (ZZZS) za sekundarni namen oblikovanja prototipa nadzorne plošče, na kateri je oblikovan sistem ključnih statističnih kazalnikov za merjenje kakovosti in učinkovitosti oskrbe pacientov s koronarno boleznijo v slovenskem zdravstvenem sistemu. Cilj zastavljenega dela je bil oblikovati nadzorno ploščo s kazalniki kakovosti in učinkovitosti oskrbe pacientov s koronarno boleznijo na podlagi administrativnih podatkov tako, da je sistem kazalnikov v skladu s teoretičnimi izhodišči paradigme na vrednosti temelječe zdravstvene oskrbe in Donabedijanovega modela merjenja kakovosti zdravstvene oskrbe, obenem pa so kazalniki prikazani na uporabniku prijazen, pregleden, razumljiv in interaktiven način, ki ga omogoča R-paket Shiny. Raziskovalni proces izgradnje prototipa nadzorne plošče je bil zastavljen v štirih korakih, pri katerih sem uporabil tako kvantitativne statistične metode kot kvalitativne metode polstrukturiranega intervjuja ter fokusne skupine. V okviru kvalitativnih metod dela sem sodeloval tudi z znanstvenim panelom domenskih strokovnjakov, da bi odgovoril tudi na tri raziskovalna vprašanja. Ker so pridobljeni podatki ZZZS administrativne narave, katerih prvotni namen je plačevanje opravljenih obravnav izvajalcem zdravstvene dejavnosti, se je prvo raziskovalno vprašanje nanašalo na oceno kakovosti, primernosti in morebitnih pristranskosti podatkov za sekundarni namen oblikovanja sistema kazalnikov. Drugo raziskovalno vprašanje (in tudi motivacija za izgradnjo prototipa nadzorne plošče) je, ali med opazovanimi izvajalci zdravstvene oskrbe obstajajo pomembne strokovne razlike v kakovosti in učinkovitosti zdravljenja, merjenimi z zastavljenim sistemom kazalnikov. Ker pa pridobljeni podatki to omogočajo, se je tretje raziskovalno vprašanje nanašalo na preučevanje, ali obstaja povezava med dejavniki pacientovega lokalnega okolja - občin prebivališča in kakovostjo in učinkovitostjo celotnega procesa zdravljenja, merjenega z omenjenimi kazalniki. Raziskovalni proces se je nanašal na paciente z diagnozo koronarne bolezni, hospitalizirane v obdobju med 1. 1. 2015 in 30. 6. 2021 v slovenskem zdravstvenem sistemu. Na nadzorni plošči sem primerjal 14 slovenskih izvajalcev zdravstvene oskrbe, povezanost med kazalniki in dejavniki prebivališča pa preučeval za 212 slovenskih občin. Raziskovalni proces sem začel z oblikovanjem končnega izbora 13 kazalnikov kakovosti in učinkovitosti, ki jih prikazujem na nadzorni plošči. V začetku sem izvedel metodo polstrukturiranega intervjuja s predstavnikoma ZZZS, kjer sem se seznanil z osnovami oblikovanja in izračunavanja kazalnikov, ki jih spremljajo na ZZZS. Po izvedeni metodi polstrukturiranega intervjuja sem izvedel metodo fokusne skupine, kjer sem kot vodilo za izvedbo pripravil dva nabora kazalnikov. Prvi, širši nabor kazalnikov je bil oblikovan predvsem na podlagi spletnih virov, drugi oziroma ožji nabor pa je bil oblikovan v skladu s paradigmo merjenja vrednosti za pacienta, Donabedianovega modela in tudi v skladu z zmožnostmi, ki jih podatki o obračunanih obravnavah ZZZS sploh omogočajo. Po izvedeni prvi fokusni skupini sem končni izbor prikazanih kazalnikov oblikoval v sodelovanju z domenskimi strokovnjaki z uporabo kombinacije metod spletne ankete ter druge ponovitve metode fokusne skupine. V drugem koraku raziskovalnega procesa sem se posvetil ocenjevanju kakovosti in primernosti podatkov o obračunanih zdravstvenih obravnavah ZZZS oblikovanje prototipa nadzorne plošče. Kvalitativni raziskovalni metodi, ki sem ju uporabil v tem segmentu raziskovalnega procesa, sta polstrukturirani intervju ter fokusna skupina. Z metodo intervjuja s predstavnikoma ZZZS sem raziskoval ozadje nastajanja, hrambe in možnih tehničnih težav pri ravnanju z njimi ter vire pristranskosti, ki izhajajo iz dejstva, da gre za podatke za administrativni namen. Z metodo fokusne skupine pa sem validiral rezultate ocenjevanja kakovosti podatkov na podlagi dveh pristopov, ki sem ju izbral po zgledu relevantne literature. Pri prvem pristopu sem presojal kakovost podatkov tako, da sem na podlagi predlaganih definicij in metodologije ocenjeval tri dimenzije kakovosti podatkov: skladnost, celovitost in prepričljivost. Drugi način ocene kakovosti pa je temeljil na medicinskih konceptih (sprejemi, odpusti, smrti pacientov) in predpostavki, da je njihova variabilnost (statistična stabilnost) skozi čas konstantna, morebitna večja odstopanja, ki jih ni mogoče pojasniti, pa pripišem morebitnim tehničnim težavam pri nastajanju administrativnih podatkov. Za oceno kakovosti podatkov po obeh pristopih sem uporabil metodi grafičnih vizualizacij in tabelaričnih povzetkov. Tretji korak raziskovalnega procesa je bil namenjen izračunu 13 kazalnikov kakovosti in učinkovitosti tako, da so prikazane vrednosti na nadzorni plošči čim bolj primerljive med obravnavanimi izvajalci zdravstvene oskrbe, kar pomeni uravnotežene oziroma kontrolirane za različne ravni tveganja, ki izhajajo iz osebne ravni pacientov (spol, starost, sočasne bolezni ...) ter ravni bolnišnice in pacientovega lokalnega okolja slovenskih občin. Omenjeni način mi je omogočil tako odgovor na raziskovalno vprašanje, ali med opazovanimi izvajalci obstajajo pomembne razlike v kakovosti in učinkovitosti obravnave, kot tudi odgovor, ali obstaja povezava med dejavniki pacientovega lokalnega okolja - občine prebivališča, in izidi in drugimi kazalniki procesa zdravstvene obravnave pacientov. Zaradi narave teh raziskovalnih vprašanj ter tipov kazalnikov sem se tudi po zgledu relevantne literature odločil, da za izračun vrednosti kazalnikov uporabim kvantitativno metodo statistične regresije. Za modeliranje vseh 13 kazalnikov sem uporabil pristop posplošenih linearnih mešanih modelov. Za kazalnike v obliki deleža oziroma binarnega izida (npr. Bolnišnična umrljivost) sem uporabil metodo večinivojske logistične regresije, za kazalnike numeričnega tipa pa bodisi večnivojske negativne binomske bodisi večnivojske gama regresije (npr. Celokupna ležalna doba), kjer sem uporabil navkrižni tip modela ... Postopek sem začel s pripravo ustreznih odvisnih in pojasnjevalnih modelskih spremenljivk. 13 odvisnih spremenljivk je predstavljalo vsak opazovani kazalnik, 14 pojasnjevalnih spremenljivk je predstavljalo značilnosti pacienta, 2 pojasnjevalni spremenljivki sta predstavljali značilnosti bolnišnic, 18 pojasnjevalnih spremenljivk pa značilnosti slovenskih občin. Za zmanjšanje dimenzijskosti podatkov sem na 17 spremenljivkah o značilnostih občin uporabil metodo eksploratorne faktorske analize, ki je nakazala uporabo treh faktorskih spremenljivk, ki so poleg spremenljivke Povprečni čas vožnje v bolnišnico postale nove pojasnjevalne modelske spremenljivke značilnosti občin. Pri 12 obravnavanih kazalnikih sem opazovano populacijo pacientov, ki so izpolnjevali vključitvene kriterije, naključno in stratificirano razdelil na učno in testno množico v razmerju 50 : 50. Na učni množici sem ocenjeval modelske koeficiente treh tipov modelov, kjer sem zaporedno dodajal sklope pojasnjevalnih spremenljivk. Najprej sem začel z modelom brez pojasnjevalnih spremenljivk, nadaljeval sem z vključitvijo pojasnjevalnih spremenljivk značilnosti pacienta ter pri tretjem tipu modelov še pojasnjevalnih spremenljivk o značilnostih bolnišnic in občin. Naključni spremenljivki odklonov od presečišča za bolnišnice in občine, ki sta predstavljali naključni del modela, sta bili vedno vključeni. Pri enem izmed kazalnikov se je metodologija zaradi majhnega vzorca razlikovala zgolj toliko, da nisem uporabil razdelitve na učno in testno množico podatkov. Za iskanje optimalnih in manj kompleksnih modelov sem uporabil metodo stepwise, ki postopno vključuje in izključuje spremenljive s ciljem minimizacije Bayesovega informacijskega kriterija (BIC). Z uporabo metode stepwise sem pri vsakem kazalniku posebej zmanjšal število vključenih modelskih pojasnjevalnih spremenljivk o značilnostih pacienta. Rezultat postopka modeliranja so bile ocene modelskih koeficientov in njihovi pripadajoči 95-odstotni Waldovi intervali zaupanja, ki sem jih za vse tri tipe modelov podal v tabelah. Posplošeni linearni mešani modeli omogočajo tudi izračun koeficienta razdelitve variance (VPC) ter medianskega razmerja obetov oziroma incidenčne stopnje (MOR/MIR), ki tudi nudijo informirano oceno povezanosti kontekstov (pacienta, bolnišnice in občine) z odvisno spremenljivko, ki je predstavljala posamezni kazalnik. Za oceno kakovosti prileganja modelov podatkom sem glede na tip kazalnika uporabil mere psevdokoeficienta determinacije Nakagawa R^2, površine pod krivuljo karakteristike delovanja sprejemnika (AUC) in mero korenske srednje kvadratne napake (RMSE). Za končni izračun vrednosti kazalnikov, ki so prikazani na nadzorni plošči, sem uporabil metodo standardizacije kazalnika. Na testni množici podatkov sem za vsakega pacienta naredil napovedi vrednosti kazalnikov, za katere sem uporabil najkompleksnejši model z vsemi vključenimi spremenljivkami. Za vsakega izvajalca sem izračunal kvocient med opazovano in za paciente tega izvajalca pričakovano (z modelom napovedano) vrednostjo kazalnika. Ta kvocient sem poimenoval utež, ki je specifična za vsakega izvajalca. Na nadzorni plošči (poleg ocenjenih 95-odstotnih intervalov zaupanja) prikazujem standardizirane vrednosti vsakega kazalnika, ki je zmnožek specifične uteži na izvajalca in ustrezne mere središčnosti (povprečje pri 12 kazalnikih, mediana pri 1 kazalniku) vseh pacientov v analizi. Četrti in zadnji korak raziskovalnega dela je bil zasnova in izgradnja prototipa nadzorne plošče. Zasnoval sem jo na podlagi spoznanj kvalitativne metode polstrukturiranega intervjuja s predstavnikoma ZZZS in zgledov iz relevantne literature. Nadzorna plošča je s teoretičnega vidika strateškega tipa, saj uporabniku nudi splošen pregled kakovosti in učinkovitosti zdravstvene obravnave pacientov skozi daljše obdobje od 1. 1. 2015 do 30. 6. 2021. Uporabil sem modernejše in interaktivne grafične prikaze in intuitivni grafični vmesnik za njihovo dinamično prilagajanje. Nadzorna plošča vsebuje tri strani. Na prvi, vstopni strani so z grafičnim elementom vrednostnih polj prikazani opazovani kazalniki na populacijskih ravni. Prikazi na prvi strani se lahko dinamično modificirajo glede na obdobje hospitalizacije, spol, starost in vrsto koronarne bolezni. Na drugi strani nadzorne plošče je omogočena primerjava med izvajalci. Na razsevnih diagramih so prikazane standardizirani vrednosti kazalnikov za vsakega izvajalca, dodana pa je tudi linija središčnosti ter 95-odstotni intervali zaupanja, izgrajeni z uporabo metode zankanja. Prikazi so razdeljeni na podstrani glede na sklope zdravstvene oskrbe pacientov. Na tretji strani nadzorne plošče so na zemljevidu interaktivno prikazane opazovane in modelsko napovedane vrednosti kazalnikov po občinah prebivališča pacientov. Na podlagi rezultatov metode spletne ankete in fokusne skupine sem v prvem koraku raziskovalnega procesa oblikoval končni izbor 13 kazalnikov kakovosti in učinkovitosti, ki so prikazani na nadzorni plošči. Skupno so razdeljeni v 5 različnih sklopov zdravstvene oskrbe pacienta. Te sklope poimenujem umrljivost pacientov, ponovne hospitalizacije pacientov, proces zdravljenja med primarno (indeksno) hospitalizacijo, čas po odpustu oziroma rehabilitacija pacienta in višina izdatkov zdravstvenih obravnav. Glavno spoznanje drugega koraka raziskovalnega procesa je, da so podatki o obračunanih obravnavah, ocenjeni na podlagi opredeljenih dimenzij kakovosti in konceptualnega načina, primerni in kakovostni. Odstopanj od predpisane in pričakovane skladnosti ni ali so te redke, celovitost podatkov po vrsticah je prav zelo blizu 100-odstotni, s stališča vrednosti, ki jih zavzemajo spremenljivke, pa sem tudi na podlagi spoznaj izvedene metode fokusne skupine oblikoval mnenje, da ni dokazov za trditev, da so neprepričljivi ali da jim ni mogoče verjeti oziroma da ne odražajo realnega stanja. Zaznal sem nekaj težav s celovitostjo podatkov pri konceptualnem načinu ocenjevanja, kar sem pripisal vplivu motnje epidemije covida-19 in drugim razlogom, kot je tehnična nadgradnja podatkovnih skladišč, ki je bila verjetni vzrok tudi za edini zaznan večji izpad podatkov. Splošna ocena je oblikovana tudi na podlagi polstrukturiranega intervjuja s predstavnikoma ZZZS, da so podatki primerni za analizo in izračun kazalnikov v magistrskem delu. Proces statističnega modeliranja oziroma izračuna vrednosti kazalnikov z metodologijo posplošenih linearnih mešanih modelov je pokazal, da je bila izbira te metode statistične regresije upravičena, saj ocenjena varianca naključnih spremenljivk odklonov od presečišča za bolnišnice in občine ni bila enaka 0. Na podlagi Bayesovega informacijskega kriterija (BIC) in Nakagawa R^2 sem ugotovil, da so se modeli bolje prilegali podatkom pri tistih kazalnikih, ki so se nanašali na indeksno hospitalizacijo, s časovnim oddaljevanjem od te pa je bilo prileganje podatkom v splošnem slabše. Mera BIC se je najbolj zmanjšala z vključitvijo pojasnjevalnih spremenljivk o značilnostih pacienta, ob vključitvi spremenljivk bolnišnic in občin pa ne več bistveno. Tudi napovedna oziroma razlagalna moč modelov, merjena s površino pod krivuljo karakteristike delovanja sprejemnika (AUC) in korenske srednje kvadratne napake (RMSE), je bila boljša pri modelih za tiste kazalnike, ki so bili časovno bližje indeksni hospitalizaciji. Poleg BIC izračunane mere VPC sicer nakazujejo, da je ključni dejavnik tveganja oziroma da je največja povezanost med odvisno spremenljivko, ki predstavlja kazalnik, pri večini obravnavanih kazalnikov prav raven pacienta. Sledijo dejavniki bolnišnice, najmanjši delež variabilnosti pa je odpadel na raven dejavnika lokalnega okolja - občine. Rezultati dela tudi nakazujejo, da ni mogoče trditi, da obstaja pomembna in močna povezanost med kazalniki izidov in procesov zdravljenja in dejavniki lokalnega okolja občin v Sloveniji. To trdim na podlagi nizke vrednosti koeficienta VPC za občine, mere BIC, ki se v večini primerov ni znižala z vključitvami pojasnjevalnih spremenljivk o značilnosti občin prebivališča, in eksponentno transformiranih modelskih koeficientov spremenljivk o značilnosti občin, ki so povečini zavzeli vrednosti blizu 1, kar pomeni, da so spremembe bodisi v razmerju obetov, bodisi v relativnem razmerju, zanemarljive. Glede na rezultate vrednosti 13 kazalnikov, ki jih prikazuje nadzorna plošča, ugotavljam, da se potrdi pričakovana razlika v kakovosti in učinkovitosti zdravstvene oskrbe tako glede na spol, starost in vrsto koronarne bolezni pri pacientih, saj imajo ti različne izhodiščne možnosti za uspešno zdravljenje. Rezultati primerjanja izvajalcev pa kažejo, da med njimi obstajajo razlike tako v kakovosti zdravljenja, merjeni z doseženimi izidi, kot po učinkovitosti merjeni s kazalniki procesa, tudi ko vrednosti kazalnikov uravnotežim za različne ravni tveganj, ki izhajajo iz dejavnikov pacienta, bolnišnice in lokalnega okolja z metodo standardizacije vrednosti. Pri kazalnikih v obliki deleža so opažene razlike standardiziranih vrednosti pri izvajalcih reda velikosti od 4 pa do 10 odstotnih točk, odvisno od kazalnika. Tudi na podlagi ocenjenih medianskega razmerja obetov/incidenčne stopnje (MOR/MIR) in koeficienta razdelitve variance (VPC) se potrdi, da med izvajalci obstajajo razlike, ki se sicer pomembno zmanjšajo ob dodajanju neodvisnih spremenljivk v modele, nekaj variabilnosti pa ostane nepojasnjene. Delo je retrospektivna opazovalna študija, ki temelji na podatkih, zbranih za drug namen. Glavna omejitev tega tipa raziskav so nezaznane sovplivajoče, tudi moteče spremenljivke, katerih značilnost je povezanost z odvisno in pojasnjevalno spremenljivko v modelih in vplivajo na modelske rezultate, če so izpuščene. Omejitev raziskovanja je tudi sama uporaba podatkov o obračunanih obravnavah ZZZS, ki lahko vsebujejo neopažene pristranskosti in izpade pri nastajanju podatkov, kar potencialno vpliva na dobljene rezultate študije. Dodatno poenostavitev predstavlja določitev indeksne hospitalizacije kot prve, ki se pojavi v podatkih, lahko pa gre tudi že za drugo, če se je prva zgodila pred začetkom opazovanja 1. 1. 2015. Omejitev je tudi operacionalizacija študije povezanosti tveganj, ki izhajajo iz lokalnega območja občin s kazalniki, kjer je velika poenostavitev predpostavka, da imajo vsi pacienti iz iste občine enako raven tveganja. Omejitve pri uporabi rezultatov in pazljivost pri interpretaciji pa so potrebne tudi zaradi omejitev uporabljenih statističnih metod, saj je vzorec 14 bolnišnic na drugi ravni relativno majhen pri uporabi posplošenih linearnih mešanih modelov.

• Jezik: Slovenski jezik
• Ključne besede: nadzorna plošča, koronarna bolezen, ključni kazalniki, kakovost podatkov, posplošeni linearni mešani modeli
• Vrsta gradiva: Magistrsko delo/naloga
• Organizacija: FE - Fakulteta za elektrotehniko
• Leto izida: 2024
• PID: 20.500.12556/RUL-155798
• COBISS.SI-ID: 193752323
• Datum objave v RUL: 18.04.2024

Sekundarni jezik

• Jezik: Angleški jezik
• Naslov: Statistical indicators of patient care quality and efficiency for patients with coronary artery disease

Izvleček

The master thesis presents a possible case of using reimbursement data for the provided treatments in the healthcare system, which are being collected by the Health Insurance Institute of Slovenia (ZZZS), to build a dashboard that displays a system of key performance indicators of quality and efficiency of care for patients with coronary artery disease in the Slovenian healthcare system. The goal of the work was to design a system of performance indicators of quality and efficiency of care for patients with coronary artery desease based on administrative data in a way that follows the paradigm of Value-Based Healthcare and Donabedian's model of measuring the quality and efficiency of healthcare. Performance indicators should be presented on the dashboard in a user-friendly, transparent, understandable, and interactive manner, which was possible to achieve with the R package Shiny. The research process of building the dashboard consisted of four steps in which a combination of qualitative and quantitative scientific methods was used in collaboration with a panel of domain experts to answer three research questions. Because the data used in this work are administrative, meaning their primary purpose is assuring payment for the provided healthcare services and products, the goal of the first research question was an assessment of the quality, fitness for use and possible biases for the secondary use case of designing the system of key performance indicators. The second research question, and also the motivation for building the dashboard, is to determine if there is a significant observed difference in the quality and efficiency of healthcare among providers, measured with the designed system of key performance indicators. Since the data enables this, the goal of the third research question is to examine if there is a connection between healthcare outcomes and processes measured with the designed system of key performance indicators and the context of patients' local area of residence - municipality. The research process included patients with a diagnosis of coronary artery disease, hospitalized between 1 January 2015 and 30 June 2021 in the Slovenian healthcare system. On the dashboard, 14 Slovenian hospitals are compared, and the estimation of the connection between performance indicators and contexts of patients' local area of residence is analyzed for 212 Slovenian municipalities. I started my research process by designing the final selection of 13 performance indicators of quality and efficiency presented on the dashboard. First, I conducted semi-structured interview with two representatives of the ZZZS during which I gained basic insights into designing and calculating the indicators they use. After the method of semi-structured interview was completed, I conducted a focus group for which I prepared two selections of possible candidates for the indicators. The first, broader set was prepared using mostly online sources, while the second set was developed on the theoretical basis of the Value-Based Healthcare paradigm, Donabedian's model, and also the feasibility of calculation based on data received was taken into account. After the first iteration of the focus group method was completed, the final selection of indicators presented on the dashboard, was made in collaboration with a panel of domain experts, using methods of online survey and second iteration of the focus group method. In the second stage of the research process, I focused on assessing the quality and the question if the data for the paid treatments are fit for designing the prototype of the dashboard. The qualitative research methods I used at this stage were a semi-structured interview and a focus group. I interviewed two ZZZS representatives to gain insights into creation, storage, and possible problems that may arise during creation of the data. For estimating the quality of data, I used two different approaches, both recommended by the relevant literature. Using the first methodology I assessed the quality of data by estimating three data quality dimensions - conformance, completeness, and plausibility -, all defined by strict methodological criteria. The second approach to estimating the quality of data was based on concepts in medicine (admissions, discharges, deaths, etc.), and the assumption that these concepts have a mostly constant variability (statistical stability), which means that large and unexplained differences from that assumption could point to data quality issues. For both of these approaches, I used methods of graphical visualizations and tabular summarization. The purpose of the third stage in the research process was to calculate the 13 indicators of quality and efficiency of care in such a way that the values will enable the best possible comparison of the healthcare providers, which means they will be controlled for different levels of risks that are present at the level of patients (age, sex, etc.), hospitals and patients' local areas of residence - municipalities in Slovenia. This approach enabled me to address the question if there is a significant difference in quality and efficiency of care among providers and also enabled me to assess if there is a connection between healthcare outcomes and process indicators with contexts of patients' local area of residence - municipalities. Based on the nature of these research questions, the type of performance indicators I was working with, and literature recommendation, I decided to use a quantitative method of statistical regression. For analysing and calculating all 13 performance indicators, I used the generalized linear mixed models methodology. For performance indicators that are represented as a ratio (Hospital mortality), I used multilevel logistic regression, and for performance indicators of a numerical type, I used either multilevel negative binomial, or multilevel Gamma regression. I started the calculation procedure by preparing the model variables. Thirteen variables were dependent and represented each performance indicator, fourteen explanatory variables represented the characteristics of patients, two explanatory variables represented the characteristics of hospitals and eighteen explanatory variables represented the properties of Slovenian municipalities. For the purspose of dimensionality reduction, I conducted the method of explanatory factor analysis on 17 variables, which represented characteristics of municipalities. The method indicated the use of three factor variables, which together with the variable Average time of transportation to the hospital, became new municipality level explanatory variables in the models. For 12 out of 13 performance indicators of quality and efficency of care, I randomly and stratified split the data with patients that met inclusion criteria into training and test data sets in a 50:50 ratio. Due to small sample size, I did not use the split of the data in case of one indicator. On the training set, I estimated three types of regression models. The first type had no explanatory variables included, the second type had only patient-level explanatory variables included, and for the third type the variables that represented hospital and municipality characteristics were also included in the model. Variables of hospitals and municipalities contexts, which represent the random part of the generalized linear mixed effects models, were always included. To find more optimal and less complex models, I used the Stepwise method, which uses the principle of minimazing the Bayes information criterion (BIC). By using the Stepwise method, I was able to reduce the number of patient characteristics explanatory variables included in the models. The results of statistical modeling were the coefficient estimates and their corresponding 95-percent Wald's confidence intervals, which were set out in the tables. Generalized linear mixed effects models allow the estimation of the so-called Variance partition coefficient (VPC) and Median odds/incidence ratio (MOR/MIR), both used to assess the connection between the characteristics of contexts (patients, hospitals, municipalities) with the dependent variable that represents every performance indicator. For the assessment of models fit to the data, I used BIC and Nakagawa R^2, a pseudo coefficient of determination. For the assessment of the model's predictive power, I used Area under receiver operating characteristic (ROC) curve (AUC) and Root mean square error (RMSE) measures. The final calculation of the performance indicators values displayed on the dashboard was made using the standardization methodology. I used the most complex model to make predictions of the dependent variable for every patient on the test data. For every provider, I calculated the quotient between the observed and expected (model-predicted) value of the performance indicator. I called this quotient a weight, which is provider-specific. On the dashboard, this standardized value of the performance indicator is displayed, which is just a product of the provider-specific weight and an appropriate central tendency value (mean for 12 indicators, median for 1 indicator) for all the patients included in the analysis. The fourth and final step in the research process for this master thesis was building the prototype of the dashboard itself. I designed it based on the insights from the method of semi-structured interview with representatives of ZZZS and good practices from relevant literature. Theoretically speaking, the dashboard is of a strategic type, meaning it offers a general overview of the quality and efficiency of care between 1 January 2015 and 30 June 2021. I used modern and interactive visualizations and an intuitive graphical user interface for their dynamic adjusting. The dashboard itself consists of three pages. The first page uses Value Boxes to display the observed values of the performance indicators on a population level. The values can be modified based on the date of hospitalization, sex, age, and type of coronary artery disease. On the second page of the dashboard, I provide a comparison between observed hospitals. Using scatterplots, I show the standardized values of the performance indicators for every healthcare provider, while also adding a corresponding 95-confidedence intervals and a central tendency line on scatterplots. The scatterplots are divided into sub-pages based on the stage of the healthcare process they represent. The last page of the dashboard displays observed and model-predicted values of indicators by municipalities. Based on the collaboration with domain experts and the results of online survey and focus group methods, the first step of the research process resulted in a final selection of 13 key performance indicators of quality and efficiency of care that are displayed on the dashboard. They are divided into five sets based on stages of healthcare for patients. I named these sets as follows: mortality, re-hospitalizations, process of care, patient care after discharge or rehabilitation, and economic aspects of healthcare. The second stage of the research process resulted in an insight that the quality of the data for the paid healthcare treatments, estimated using both data quality dimensions and conceptual approaches, is generally high. There is very little to no noncompliance to established variable conformance rules. Row completeness of data is also very close to 100 percent. Based on the analysis of the values that variables take, there is very little reason to believe they are not plausible or believable, which was also validated with the focus group method in collaboration with the panel of domain experts. I did find some possible data quality issues using the conceptual approach, but these problems were explained by the influence of the covid-19 pandemic, and one bigger outage I detected was explained by data warehouse procedure changes at the ZZZS. The general assessment, formed also from the insights of a semi-structured interview, is that data are mostly fit for my purpose of building the dashboard. The main research result in the third stage of the process of building the dashboard, which consisted of calculating the performance indicator values using statistical modelling, shows that applying generalized linear mixed models methodology was justified since the estimated variances of the random intercept variables for hospitals and municipalities contexts were not close or equal to zero. BIC and Nakagawa R^2 measures seemed to indicate that models fitted the data better for those performance indicators that related closer to the time of index hospitalization. BIC generally improved the most with the inclusion of patient characteristics explanatory variables in the models and did not improve much further with the inclusion of hospital and municipality characteristics independent variables. The predictive power of the models measured with AUC and RMSE was also generally better with models for those performance indicators that were closer to the time of index hospitalization. In addition to BIC, observed Variance partition coefficients (VPC) seemed to indicate that the main source of risk lies at the patient individual level, followed by hospital and patient local area of residence - municipalities contexts. For the latter, no significant connection with the outcomes or process indicators is observed. I make this claim based on the detected VPC values, no significant improvement on BIC or Nakagawa R^2 when adding variables of patients' local area contexts, and also based on the exponentially transformed estimated model coefficients for these variables, which are in most cases very close to 1, meaning no significant change in odds or relative ratios. Based on the observed values of 13 performance indicators displayed on the dashboard, I observe the expected difference in quality and efficiency of care that stems from different patients' characteristics (age, sex, type of coronary artery disease) baseline risks. The comparison of providers seems to indicate differences both in the quality measured with the relevant outcomes, but also in the efficiency measured with process indicators of care provided to the patients. Some differences persist even after I control for different levels of risks that stem from patient, hospital, and patients' local area contexts using the adjusted values approach. For performance indicators of the type of ratio, the difference between providers is on average 4 to 10 percentage points, depending on the indicator itself. The differences between hospitals are also observed based on VPC and MOR/MIR values, but they generally decrease with adding explanatory variables in models, although some observed variability remains unexplained. This master thesis is a retrospective observational study. The main limitation of this type of research stems from unknown confounding variables, which have the characteristic of being associated with both dependent and independent variables and can affect the model results if left unaccounted for. The limitation of my work is also based on the use of data that are primarily intended for other purposes, so I need to emphasize all observed and unobserved biases and limitations of such data that can influence the results. There is also quite a big simplification in the research of the association between local area contexts and performance indicators, which is the assumption that all the patients from the same municipality have the same local area characteristics risk level. I also have to emphasize the limitation of determining the index hospitalizations, which for this work was assumed as every first hospitalization noted in the data, but this could in fact be second or third hospitalization and the index hospitalization happened before 1 January 2015 but is of course not present in the data. I would advise some caution when interpreting results due to statistical methods being used since having only 14 hospitals on the models' second level makes quite a small sample size when using generalized linear mixed effects models methodology.

• Ključne besede: dashboard, coronary artery disease, key performance indicators, data quality, generalized linear mixed effects models