Efficient generator of mathematical expressions for symbolic regression

Mežnar, Sebastian (author), Džeroski, Sašo (author), Todorovski, Ljupčo (author)

We propose an approach to symbolic regression based on a novel variational autoencoder for generating hierarchical structures, HVAE. It combines simple atomic units with shared weights to recursively encode and decode the individual nodes in the hierarchy. Encoding is performed bottom-up and decoding top-down. We empirically show that HVAE can be trained efficiently with small corpora of mathematical expressions and can accurately encode expressions into a smooth low-dimensional latent space. The latter can be efficiently explored with various optimization methods to address the task of symbolic regression. Indeed, random search through the latent space of HVAE performs better than random search through expressions generated by manually crafted probabilistic grammars for mathematical expressions. Finally, EDHiE system for symbolic regression, which applies an evolutionary algorithm to the latent space of HVAE, reconstructs equations from a standard symbolic regression benchmark better than a state-of-the-art system based on a similar combination of deep learning and evolutionary algorithms.

• Language: English
• Keywords: symbolic regression, equation discovery, generative models, variational autoencoders, evolutionary algorithms
• Work type: Article
• Typology: 1.01 - Original Scientific Article
• Organization: FMF - Faculty of Mathematics and Physics
• Publication status: Published
• Publication version: Version of Record
• Year: 2023
• Number of pages: Str. 4563-4596
• Numbering: Vol. 112, iss. 11
• PID: 20.500.12556/RUL-153002
• UDC: 004.8
• ISSN on article: 1573-0565
• DOI: 10.1007/s10994-023-06400-2
• COBISS.SI-ID: 176785923
• Publication date in RUL: 14.12.2023

Record is a part of a journal

• Title: Machine learning
• Shortened title: Mach. learn.
• Publisher: Springer Nature
• ISSN: 1573-0565
• COBISS.SI-ID: 513211417

Licences

• License: CC BY 4.0, Creative Commons Attribution 4.0 International
• Link: http://creativecommons.org/licenses/by/4.0/
• Description: This is the standard Creative Commons license that gives others maximum freedom to do what they want with the work as long as they credit the author.

Secondary language

• Language: Slovenian
• Keywords: simbolna regresija, odkrivanje enačb, generativni modeli, variacijski samokodirniki, evolucijski algoritmi