Meta-optimization of dimension adaptive parameter schema for Nelder–Mead algorithm in high-dimensional problems

Rojec, Žiga (author), Tuma, Tadej (author), Olenšek, Jernej (author), Bűrmen, Árpád (author), Puhan, Janez (author)

Although proposed more than half a century ago, the Nelder–Mead simplex search algorithm is still widely used. Four numeric constants define the operations and behavior of the algorithm. The algorithm with the original constant values performs fine on most low-dimensional, but poorly on high-dimensional, problems. Therefore, to improve its behavior in high dimensions, several adaptive schemas setting the constants according to the problem dimension were proposed in the past. In this work, we present a novel adaptive schema obtained by a meta-optimization procedure. We describe a schema candidate with eight parameters subject to meta-optimization and define an objective function evaluating the candidate’s performance. The schema is optimized on up to 100-dimensional problems using the Parallel Simulated Annealing with Differential Evolution global method. The obtained global minimum represents the proposed schema. We compare the performance of the optimized schema with the existing adaptive schemas. The data profiles on the Gao–Han modified quadratic, Moré–Garbow–Hilstrom, and CUTEr (Constrained and Unconstrained Testing Environment, revisited) benchmark problem sets show that the obtained schema outperforms the existing adaptive schemas in terms of accuracy and convergence speed.

• Language: English
• Keywords: meta-optimization, Nelder-Mead algorithm, adaptive parameter schema, high-dimensional optimization problems
• Work type: Article
• Typology: 1.01 - Original Scientific Article
• Organization: FE - Faculty of Electrical Engineering
• Publication status: Published
• Publication version: Version of Record
• Year: 2022
• Number of pages: 16 str.
• Numbering: Vol. 10, iss. 13, art. 2288
• PID: 20.500.12556/RUL-145174
• UDC: 004
• ISSN on article: 2227-7390
• DOI: 10.3390/math10132288
• COBISS.SI-ID: 118971139
• Publication date in RUL: 12.04.2023

Record is a part of a journal

• Title: Mathematics
• Shortened title: Mathematics
• Publisher: MDPI AG
• ISSN: 2227-7390
• COBISS.SI-ID: 523267865

Licences

• License: CC BY 4.0, Creative Commons Attribution 4.0 International
• Link: http://creativecommons.org/licenses/by/4.0/
• Description: This is the standard Creative Commons license that gives others maximum freedom to do what they want with the work as long as they credit the author.

Secondary language

• Language: Slovenian
• Keywords: metaoptimizacija, algoritem Nelder-Mead, adaptivni parametri, visokodimenzionalni optimizacijski problemi

Projects

• Funder: ARRS - Slovenian Research Agency
• Project number: P2-0246
• Name: ICT4QoL - Informacijsko komunikacijske tehnologije za kakovostno življenje