Learning with noisy labels [~re]visited

Hudovernik, Valter (avtor), Rot, Žiga (avtor), Vovk, Klemen (avtor), Škodnik, Luka (avtor), Čehovin Zajc, Luka (avtor)

Learning with noisy labels (LNL) is a subfield of supervised machine learning investigating scenarios in which the training data contain errors. While most research has focused on synthetic noise, where labels are randomly corrupted, real-world noise from human annotation errors is more complex and less understood. Wei et al. (2022) introduced CIFAR-N, a dataset with human-labeled noise and claimed that real-world noise is fundamentally more challenging than synthetic noise. This study aims to reproduce their experiments on testing the characteristics of human-annotated label noise, memorization dynamics, and benchmarking of LNL methods. We successfully reproduce some of the claims but identify some quantitative discrepancies. Notably, our attempts to reproduce the reported benchmark reveal inconsistencies in the reported results. To address these issues, we develop a unified framework and propose a refined benchmarking protocol that ensures a fairer evaluation of LNL methods. Our findings confirm that real-world noise differs structurally from synthetic noise and is memorized more rapidly by deep networks. By open-sourcing our implementation, we provide a more reliable foundation for future research in LNL.

• Jezik: Angleški jezik
• Ključne besede: noisy learning, deep learning, repeatability of experiments
• Vrsta gradiva: Članek v reviji
• Tipologija: 1.01 - Izvirni znanstveni članek
• Organizacija: FRI - Fakulteta za računalništvo in informatiko
• Status publikacije: Objavljeno
• Različica publikacije: Objavljena publikacija
• Leto izida: 2026
• Št. strani: 22 str.
• Številčenje: Vol. 11, iss. 1, Art. 2
• PID: 20.500.12556/RUL-179046
• UDK: 004.85
• ISSN pri članku: 2430-3658
• DOI: 10.5281/zenodo.18401497
• COBISS.SI-ID: 267165699
• Datum objave v RUL: 04.02.2026

Gradivo je del revije

• Naslov: The Rescience journal
• Založnik: The ReScience journal
• ISSN: 2430-3658
• COBISS.SI-ID: 525809177

Licence

• Licenca: CC BY 4.0, Creative Commons Priznanje avtorstva 4.0 Mednarodna
• Povezava: http://creativecommons.org/licenses/by/4.0/deed.sl
• Opis: To je standardna licenca Creative Commons, ki daje uporabnikom največ možnosti za nadaljnjo uporabo dela, pri čemer morajo navesti avtorja.

Sekundarni jezik

• Jezik: Slovenski jezik
• Ključne besede: učenje na šumnih podatkih, globoko učenje, ponovljivost eksperimentov