Quadratic mutual information feature selection

Sluga, Davor (author), Lotrič, Uroš (author)

We propose a novel feature selection method based on quadratic mutual information which has its roots in Cauchy–Schwarz divergence and Renyi entropy. The method uses the direct estimation of quadratic mutual information from data samples using Gaussian kernel functions, and can detect second order non-linear relations. Its main advantages are: (i) unified analysis of discrete and continuous data, excluding any discretization; and (ii) its parameter-free design. The effectiveness of the proposed method is demonstrated through an extensive comparison with mutual information feature selection (MIFS), minimum redundancy maximum relevance (MRMR), and joint mutual information (JMI) on classification and regression problem domains. The experiments show that proposed method performs comparably to the other methods when applied to classification problems, except it is considerably faster. In the case of regression, it compares favourably to the others, but is slower.

• Language: English
• Keywords: feature selection, information-theoretic measures, quadratic mutual information, Cauchy-Schwarz divergence
• Work type: Article
• Typology: 1.01 - Original Scientific Article
• Organization: FRI - Faculty of Computer and Information Science
• Publication status: Published
• Publication version: Version of Record
• Year: 2017
• Number of pages: 16 str.
• Numbering: Vol. 19, iss. 4, art. 157
• PID: 20.500.12556/RUL-131260
• UDC: 004
• ISSN on article: 1099-4300
• DOI: 10.3390/e19040157
• COBISS.SI-ID: 1537405123
• Publication date in RUL: 24.09.2021

Record is a part of a journal

• Title: Entropy
• Shortened title: Entropy
• Publisher: MDPI
• ISSN: 1099-4300
• COBISS.SI-ID: 515806233

Licences

• License: CC BY 4.0, Creative Commons Attribution 4.0 International
• Link: http://creativecommons.org/licenses/by/4.0/
• Description: This is the standard Creative Commons license that gives others maximum freedom to do what they want with the work as long as they credit the author.
• Licensing start date: 01.04.2017

Secondary language

• Language: Slovenian
• Keywords: izbira značilk, informacijsko-teoretične mere, kvadratna medsebojna informacija, Cauchy-Schwarzova divergenca

Projects

• Funder: ARRS - Slovenian Research Agency
• Project number: P2-0241
• Name: Sinergetika kompleksnih sistemov in procesov