Bootstrap as a one-size-fits-all approach to quantifying uncertainty

Zrimšek, Urša (author), Demšar, Jure (mentor)

Quantifying uncertainty is a vital part of every statistical study. There are many different methods, but in the hands of an inexperienced user, most of them can lead to big mistakes in the interpretation. Bootstrap is a favorable method for this task because of its robustness, versatility, ease of understanding and lack of stringent distributional assumptions. But even after 40 years of existence, it is not clear if this general method is accurate enough to substitute the traditional methods specialized for specific parameters of interest. To answer this, we designed an extensive simulation study that assesses the methods' confidence interval prediction for six different parameters on samples of multiple sizes, generated from seven diverse distributions. We chose the double bootstrap as the best general bootstrap method, additionally recommending the standard bootstrap for confidence intervals for extreme percentiles. We compared the best bootstrap methods to the traditional methods and found out that for almost all of the parameters no traditional method is practically better. Moreover, bootstrap gives good predictions even on the distributions where traditional methods fail because of broken assumptions. Our work thus suggests that estimates generated by the proposed bootstrap methods are comparable to or even better than the ones made by the traditional methods.

• Language: English
• Keywords: Uncertainty, confidence intervals, bootstrap.
• Work type: Master's thesis/paper
• Typology: 2.09 - Master's Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2023
• PID: 20.500.12556/RUL-153314
• COBISS.SI-ID: 178901507
• Publication date in RUL: 22.12.2023

Secondary language

• Language: Slovenian
• Title: Bootstrap kot splošen pristop k merjenju negotovosti

Abstract

Merjenje negotovosti je pomemben del vsake statistične raziskave. Obstaja veliko metod, vendar pa lahko ob nerazumevanju njihovega ozadja večina privede do velikih napak pri interpretaciji. Metoda bootstrap bi lahko zaradi svoje robustnosti, vsestranskosti, preprostosti in pomanjkanja strogih predpostavk o porazdelitvi podatkov zmanjšala število takih napak. Toda tudi po štiridesetih letih obstoja ni povsem jasno, če je ta splošna metoda dovolj natančna, da bi lahko nadomestila tradicionalne metode, specializirane za posamezne parametre. Da bi odgovorili na to vprašanje, smo izvedli obsežno simulacijo, ki ocenjuje napovedi intervalov zaupanja za šest parametrov na vzorcih različnih velikosti, vzorčenih iz sedmih raznolikih porazdelitev. Za najboljšo splošno metodo bootstrap smo izbrali dvojni bootstrap, dodatno pa priporočamo uporabo standardnega bootstrapa za izračun intervalov zaupanja za ekstremne percentile. Predlagane bootstrap metode smo primerjali s tradicionalnimi metodami in ugotovili, da za skoraj vse parametre ne obstaja praktično boljša tradicionalna metoda. Poleg tega nam bootstrap zagotavlja dobre napovedi celo na porazdelitvah, kjer tradicionalne metode zaradi napačnih predpostavk zatajijo. Zaključujemo, da so ocene intervalov pridobljene s predlaganimi bootstrap metodami primerljive, če ne celo boljše od ocen tradicionalnih metod.

• Keywords: Negotovost, intervali zaupanja, bootstrap.