Feature selection by combining feature importance and redundancy estimates

Najdova, Elena (author), Džeroski, Sašo (mentor)

Feature selection is a crucial task in the process of building predictive models in machine learning. Understanding which data is relevant for our problem and what feature selection method is most compatible with our data is of vital importance to achieve good results. The goal of this thesis was to develop and implement software for feature selection in the machine learning task of classification, based on the maximum relevance minimum redundancy (MRMR) approach. We aimed to create a software which can be used with different measures of relevance and redundancy. This software is then used to compare the performance of the developed approach (choosing k features with the MRMR approach) to the performance of the classical filtering approach, where the top k features with highest relevance are selected (and their redundancy is ignored). We analyze the results and draw conclusions from them. With appropriate relevance and redundancy measures, the MRMR approach can perform much better than the filtering approach.

• Language: English
• Keywords: feature selection, machine learning, maximum relevance minimum redundancy, classification, predictive model
• Work type: Bachelor thesis/paper
• Typology: 2.11 - Undergraduate Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2023
• PID: 20.500.12556/RUL-148393
• COBISS.SI-ID: 158150403
• Publication date in RUL: 21.08.2023

Secondary language

• Language: Slovenian
• Title: Izbiranje značilk s kombiniranjem ocen njihove pomembnosti in redundance

Abstract

Izbira značilk je ključna naloga v procesu izgradnje vsakega napovednega modela v strojnem učenju. Razumevanje, kateri podatki so pomembni za naš problem in katera metoda izbire značilk je najbolj kompatibilna z našimi podatki, je bistvenega pomena za doseganje dobrih rezultatov pri vsakem problemu nadzorovanega učenja. Cilj te naloge je bil razviti in implementirati programsko opremo za izbiro značilk pri nalogi strojnega učenja, natančneje klasifikaciji, na podlagi pristopa največje pomembnosti in najmanjše redundance (MRMR). Naš cilj je ustvariti programsko opremo, ki se lahko uporablja za izbiro značilk različnimi merami relevantnosti in redundance. To programsko opremo smo nato uporabili za primerjavo uspešnosti razvitega pristopa, kjer k značilk, izberemo z MRMR, z uspešnostjo pristopa filtriranja, pri katerem se izbere k najpomembnejših značilk z največjo pomembnostjo (in se ne upošteva njihova redundanca). Analiziramo rezultate, primerjamo in iz njih potegnemo zaključke. z ustrezno izbiro mer pomembnosti in redundance lahko metoda MRMR deluje veliko boljše.

• Keywords: izbira značilk, strojno učenje, največja pomembnost najmanjša redundanca, klasifikacija, napovedni modeli