Popravki pri večkratnem testiranju: primer gibanja cen emisijskih kuponov

Žavbi Kunaver, Anja (author), Ograjenšek, Irena (mentor), Cugmas, Marjan (comentor)

Pri rabi statistike v praksi se vse pogosteje srečujemo z izzivom večkratnega testiranja, saj lahko brez ustreznih popravkov dobimo preveliko število zavrnjenih resničnih ničelnih domnev (napaka prve vrste). Izziv je med drugim prisoten tudi na področju energetike oziroma trgovanja z emisijskimi kuponi, ki ga naslavljamo v pričujočem delu. Namen emisijskih kuponov je zmanjšanje izpustov ogljikovega dioksida, trgovanje s kuponi pa poteka na več ravneh: na primarnem trgu in na sekundarnem trgu. Podjetja se morajo vključevati na trg emisijskih kuponov, da si zagotovijo dovolj pravic za izpuste ogljikovega dioksida. Primarni trg predstavljajo avkcije, ki potekajo vsak delovni dan ob 11. uri, sekundarni trg pa je aktiven vsak delovni dan po 10 ur dnevno. Trgovanje na sekundarnem trgu poteka neprekinjeno, trgovce pa zanima, kateri dan in ob kateri uri se jim najbolj splača kupiti ali prodati emisijske kupone (kdaj je cena višja ali nižja od dnevnega ali tedenskega povprečja). Za preverjanje statistične značilnosti razlik med povprečji cen lahko uporabimo različne teste, kot so t-test, Wilcoxonov test predznačenih rangov, Tukeyev test in permutacijski test. V tem magistrskem delu se osredotočimo na primerjavo t-testa in bolj splošnega pristopa - permutacijskega testa. Prednosti slednjega sta v njegovi vsestranski uporabnosti in dejstvu, da ne predpostavlja porazdelitve testne statistike. Če preverjamo domneve o povprečnih cenah na ravni dni ali ur z uporabo t-testa ali permutacijskega testa, naletimo na problem večkratnega testiranja. K problemu lahko pristopamo z uporabo različnih metod. Najstarejša je Bonferronijeva metoda, ki kontrolira verjetnost ene ali več zavrnitev ničelnih domnev (skupna stopnja napake), sodobnejše metode pa kontrolirajo pričakovani delež lažno zavrnjenih domnev izmed vseh zavrnjenih (stopnja napačnega odkritja). Primer slednje metode so popravki Benjamini-Hochberg (BH). V magistrskem delu preučujemo, kateri test (permutacijski ali t-test) je bolj primeren za analizo gibanja cene emisijskih kuponov. V ta namen simuliramo vrednosti, ki so podobne dejanskim cenam emisijskih kuponov na sekundarnem trgu in primerjamo moč testa s permutacijskim in t-testom. Izkaže se, da sta oba testa ustrezna in imata podobno moč. Glede na to, da je parni t-test enostavnejši in hitrejši za uporabo, ga izberemo kot bolj ustreznega. Nadalje nas zanima, kateri popravki p-vrednosti izmed bolj poznanih so najboljši - s pomočjo katerih dobimo največjo moč testa ter kako velika je razlika med njimi. Izkaže se, da ima pristop BH, ki kontrolira stopnjo napačnega odkritja, očitno večjo moč testa v primerjavi z ostalimi uporabljenimi metodami. Na koncu na danih vrednostih cen emisijskih kuponov izvedemo analizo s parnim t-testom in BH popravki s ciljem oblikovanja priporočene strategije trgovanja. Ugotovimo, da se trgovcem v povprečju splača kupovati med 8. in 10. uro zjutraj (takrat je cena v povprečju nižja) ter prodajati med 10. in 12. uro (ko je cena v povprečju višja kot zjutraj). Med dnevi v tednu na danih podatkih sicer ni statistično značilnih razlik v povprečni ceni pri stopnji tveganja manj kot 5 %.

• Language: Slovenian
• Keywords: večkratno testiranje, permutacijski test, t-test, Benjamini-Hochberg, emisijski kuponi, p-vrednosti
• Work type: Master's thesis/paper
• Organization: FE - Faculty of Electrical Engineering
• Year: 2021
• PID: 20.500.12556/RUL-133171
• COBISS.SI-ID: 82566147
• Publication date in RUL: 15.11.2021

Secondary language

• Language: English
• Title: Multiple Testing Corrections: Case of Emissions Coupons' Price Movements

Abstract

When applying statistics in practice, we often face the challenge of multiple testing, which, without appropriate corrections, may lead to too many rejected true null hypothesis (type I error rate). The challenge also needs to be dealt with in the energy sector of the economy. One such case is trading with emission coupons which we address in this master's thesis. The purpose of emission coupons is to reduce the emissions of carbon dioxide. Trading with coupons takes place at two levels: on the primary and secondary market. Here, companies which actively trade on the emission markets, strive to provide themselves with enough allowance for carbon dioxide emissions. On the primary market, there are auctions every working day at 11 o'clock. The secondary market is active ten hours a day every working day. There is trading going on continuously on the secondary market and the traders want to know which day and which hour are optimal for selling or buying emission coupons (when is the actual price higher or lower than daily or weekly average). There are various possible approaches for testing statistical significance of differences between average prices such as t-test, Wilcoxon sig-rank test, Tukey test and permutation test. In this master's thesis we focus on comparison between paired t-test and a more general approach – permutation test. The advantage of the latter is its general applicability and the fact that it is not based on any assumption about the test statistic distribution. If we compare daily and hourly averages with t-test or with permutation test, we come across the multiple testing problem. We can attempt to solve it with various methods. The oldest is Bonferroni’s method which controls the probability of one or more rejections (family-wise error rate - FWER) but the later methods control the expected proportion of erroneously rejected null hypotheses (false discovery rate - FDR). Example of such a method is Benjamini-Hochberg's method (BH). In this master's thesis we study which test (permutation or t-test) is more appropriate for analysis of emission coupons’ price trends. We therefore simulate values which are similar to the actual prices of emission coupons on the secondary market and compare power of permutation test to power of t-test. It turns out that both tests are appropriate and have about the same test power. Because the paired t-test is faster and easier to use we choose it as more appropriate. We also attempt to find out which multiple testing procedure among the better-known ones is the most appropriate – with which procedure do we get the best test power and how big are differences between procedures. It turns out that the BH procedure which controls the FDR has the best power comparing to other applied procedures. Finally, based on the real-life data on the emission coupons’ prices we carry out analysis with paired t-test and BH procedure with the goal of proposing a relevant trading strategy. We determine that traders should in general buy between 8 and 10 AM (when the price is lower on average) and sell between 10 and 12 (when the price on average is higher than in the morning). However, based on our data there seem to be no statistical significance between day price averages at risk level less than 5 %.

• Keywords: multiple testing, permutation test, t-test, Benjamini-Hochberg, emission coupons, p-values