Approximate multiple kernel learning with least-angle regression

Stražar, Martin (author), Curk, Tomaž (author)

Kernel methods provide a principled way for general data representations. Multiple kernel learning and kernel approximation are often treated as separate tasks, with considerable savings in time and memory expected if the two are performed simultaneously. Our proposed Mklaren algorithm selectively approximates multiple kernel matrices in regression. It uses Incomplete Cholesky Decomposition and Least-angle regression (LAR) to select basis functions, achieving linear complexity both in the number of data points and kernels. Since it approximates kernel matrices rather than functions, it allows to combine an arbitrary set of kernels. Compared to single kernel-based approximations, it selectively approximates different kernels in different regions of the input spaces. The LAR criterion provides a robust selection of inducing points in noisy settings, and an accurate modelling of regression functions in continuous and discrete input spaces. Among general kernel matrix decompositions, Mklaren achieves minimal approximation rank required for performance comparable to using the exact kernel matrix, at a cost lower than 1% of required operations. Finally, we demonstrate the scalability and interpretability in settings with millions of data points and thousands of kernels.

• Language: English
• Keywords: kernel methods, kernel approximation, multiple kernel learning, least-angle regression
• Work type: Article
• Typology: 1.01 - Original Scientific Article
• Organization: FRI - Faculty of Computer and Information Science
• Publication status: Published
• Publication version: Version of Record
• Year: 2019
• Number of pages: Str. 245-258
• Numbering: Vol. 340
• PID: 20.500.12556/RUL-125566
• UDC: 004
• ISSN on article: 0925-2312
• DOI: 10.1016/j.neucom.2019.02.030
• COBISS.SI-ID: 1538162883
• Publication date in RUL: 25.03.2021

Record is a part of a journal

• Title: Neurocomputing
• Shortened title: Neurocomputing
• Publisher: Elsevier
• ISSN: 0925-2312
• COBISS.SI-ID: 172315

Licences

• License: CC BY 4.0, Creative Commons Attribution 4.0 International
• Link: http://creativecommons.org/licenses/by/4.0/
• Description: This is the standard Creative Commons license that gives others maximum freedom to do what they want with the work as long as they credit the author.
• Licensing start date: 25.03.2021

Secondary language

• Language: Slovenian
• Keywords: jedrne metode, aproksimacija jeder, učenje z več jedrnimi funkcijami, regresija najmanjših kotov