On simple EM acceleration schemes suitable for mixture modelling with high overlap between components

Panić, Branislav (author), Klemenc, Jernej (author), Nagode, Marko (author), Oman, Simon (author)

The Expectation-Maximisation (EM) algorithm is widely used for maximum likelihood estimation in incomplete data problems such as mixture modelling, but it often converges slowly, particularly when mixture components overlap substantially. This study presents a comprehensive empirical evaluation of simple EM acceleration schemes for Gaussian mixture models, comparing linear (STEM), quadratic (SQUAREM), and greedy (line search, golden section) methods across 240 simulated mixture configurations spanning three dimensionalities, four component counts, five overlap levels, and four sample sizes. A key contribution is the first systematic comparison of the three acceleration parameter estimates (▫$\alpha$▫▫$_1$▫, ▫$\alpha$▫▫$_2$▫, ▫$\alpha$▫▫$_3$▫) in the mixture modelling context: we show that only ▫$\alpha$▫▫$_3$▫, which is derived as the geometric mean estimate of ▫$\alpha$▫▫$_1$▫ and ▫$\alpha$▫▫$_2$▫, provides genuine acceleration, while ▫$\alpha$▫▫$_1$▫ and ▫$\alpha$▫▫$_2$▫ consistently increase iteration counts by 50–110% relative to ▫$\alpha$▫▫$_3$▫, effectively acting as deceleration. With ▫$\alpha$▫▫$_3$▫, SQUAREM reduces iterations by up to 48% with negligible computational overhead, while greedy methods achieve similar iteration reductions but at 50–110% greater wall-clock time due to repeated log-likelihood evaluations. Crucially, acceleration does not degrade parameter estimation quality under any tested combination of initialisation, overlap, dimensionality, or number of components. We further examine the interaction between acceleration and initialisation, finding that k-means benefits most from acceleration (up to 50% time savings), while the REBMIX (Rough-Enhanced-Bayes MIXture estimation) algorithm benefits least as it already starts near the optimum. Among REBMIX configurations, histogram preprocessing with the outliers mode traversing strategy offers the best trade-off between quality and computational cost. The findings are validated on a real-world Backblaze hard drive failure dataset, confirming the practical utility of EM acceleration. All methods are implemented in the free and open-source R package rebmix, accompanied by full source code.

• Language: English
• Keywords: mixture modelling, expectation-maximisation, acceleration, parameter estimation
• Work type: Article
• Typology: 1.01 - Original Scientific Article
• Organization: FS - Faculty of Mechanical Engineering
• Publication status: Published
• Publication version: Version of Record
• Year: 2026
• Number of pages: 21 str.
• Numbering: Vol. 14, issue 9, art. 1543
• PID: 20.500.12556/RUL-185098
• UDC: 510.5:004.414.23
• ISSN on article: 2227-7390
• DOI: 10.3390/math14091543
• COBISS.SI-ID: 285731331
• Publication date in RUL: 22.07.2026

Record is a part of a journal

• Title: Mathematics
• Shortened title: Mathematics
• Publisher: MDPI AG
• ISSN: 2227-7390
• COBISS.SI-ID: 523267865

Licences

• License: CC BY 4.0, Creative Commons Attribution 4.0 International
• Link: http://creativecommons.org/licenses/by/4.0/
• Description: This is the standard Creative Commons license that gives others maximum freedom to do what they want with the work as long as they credit the author.

Secondary language

• Language: Slovenian
• Keywords: mešani modeli, EM algoritem, pospeševanje, ocena parametrov

Projects

• Funder: ARIS - Slovenian Research and Innovation Agency
• Project number: P2-0182
• Name: Razvojna vrednotenja