On simple EM acceleration schemes suitable for mixture modelling with high overlap between components

Panić, Branislav (avtor), Klemenc, Jernej (avtor), Nagode, Marko (avtor), Oman, Simon (avtor)

The Expectation-Maximisation (EM) algorithm is widely used for maximum likelihood estimation in incomplete data problems such as mixture modelling, but it often converges slowly, particularly when mixture components overlap substantially. This study presents a comprehensive empirical evaluation of simple EM acceleration schemes for Gaussian mixture models, comparing linear (STEM), quadratic (SQUAREM), and greedy (line search, golden section) methods across 240 simulated mixture configurations spanning three dimensionalities, four component counts, five overlap levels, and four sample sizes. A key contribution is the first systematic comparison of the three acceleration parameter estimates (▫$\alpha$▫▫$_1$▫, ▫$\alpha$▫▫$_2$▫, ▫$\alpha$▫▫$_3$▫) in the mixture modelling context: we show that only ▫$\alpha$▫▫$_3$▫, which is derived as the geometric mean estimate of ▫$\alpha$▫▫$_1$▫ and ▫$\alpha$▫▫$_2$▫, provides genuine acceleration, while ▫$\alpha$▫▫$_1$▫ and ▫$\alpha$▫▫$_2$▫ consistently increase iteration counts by 50–110% relative to ▫$\alpha$▫▫$_3$▫, effectively acting as deceleration. With ▫$\alpha$▫▫$_3$▫, SQUAREM reduces iterations by up to 48% with negligible computational overhead, while greedy methods achieve similar iteration reductions but at 50–110% greater wall-clock time due to repeated log-likelihood evaluations. Crucially, acceleration does not degrade parameter estimation quality under any tested combination of initialisation, overlap, dimensionality, or number of components. We further examine the interaction between acceleration and initialisation, finding that k-means benefits most from acceleration (up to 50% time savings), while the REBMIX (Rough-Enhanced-Bayes MIXture estimation) algorithm benefits least as it already starts near the optimum. Among REBMIX configurations, histogram preprocessing with the outliers mode traversing strategy offers the best trade-off between quality and computational cost. The findings are validated on a real-world Backblaze hard drive failure dataset, confirming the practical utility of EM acceleration. All methods are implemented in the free and open-source R package rebmix, accompanied by full source code.

• Jezik: Angleški jezik
• Ključne besede: mixture modelling, expectation-maximisation, acceleration, parameter estimation
• Vrsta gradiva: Članek v reviji
• Tipologija: 1.01 - Izvirni znanstveni članek
• Organizacija: FS - Fakulteta za strojništvo
• Status publikacije: Objavljeno
• Različica publikacije: Objavljena publikacija
• Leto izida: 2026
• Št. strani: 21 str.
• Številčenje: Vol. 14, issue 9, art. 1543
• PID: 20.500.12556/RUL-185098
• UDK: 510.5:004.414.23
• ISSN pri članku: 2227-7390
• DOI: 10.3390/math14091543
• COBISS.SI-ID: 285731331
• Datum objave v RUL: 22.07.2026

Gradivo je del revije

• Naslov: Mathematics
• Skrajšan naslov: Mathematics
• Založnik: MDPI AG
• ISSN: 2227-7390
• COBISS.SI-ID: 523267865

Licence

• Licenca: CC BY 4.0, Creative Commons Priznanje avtorstva 4.0 Mednarodna
• Povezava: http://creativecommons.org/licenses/by/4.0/deed.sl
• Opis: To je standardna licenca Creative Commons, ki daje uporabnikom največ možnosti za nadaljnjo uporabo dela, pri čemer morajo navesti avtorja.

Sekundarni jezik

• Jezik: Slovenski jezik
• Ključne besede: mešani modeli, EM algoritem, pospeševanje, ocena parametrov

Projekti

• Financer: ARIS - Javna agencija za znanstvenoraziskovalno in inovacijsko dejavnost Republike Slovenije
• Številka projekta: P2-0182
• Naslov: Razvojna vrednotenja