Measuring catastrophic forgetting in cross-lingual classification : transfer paradigms and tuning strategies

Koloski, Boshko (avtor), Škrlj, Blaž (avtor), Robnik Šikonja, Marko (avtor), Pollak, Senja (avtor)

Cross-lingual transfer leverages knowledge from a resource-rich source language, commonly English, to enhance performance in less-resourced target languages. Two widely used strategies are: Cross-Lingual Validation (CLV), which involves training on the source language and validating on the target language, and Intermediate Training (IT), where models are first fine-tuned on the source language and then further trained on the target language. While both strategies have been studied, their effects on encoder-based models for classification tasks remain underexplored. In this paper, we systematically compare these strategies across six multilingual classification tasks, evaluating downstream performance, catastrophic forgetting, and both zero-shot and full-shot scenarios. Additionally, we contrast parameter-efficient adapter methods with full-parameter fine-tuning. Our results show that IT generally performs better in the target language, whereas CLV more effectively preserves source-language knowledge across multiple cross-lingual transfers. These findings underscore the trade-offs between optimizing target performance and mitigating catastrophic forgetting.

• Jezik: Angleški jezik
• Ključne besede: cross-lingual transfer, cross-lingual learning, catastrophic-forgetting, document classification
• Vrsta gradiva: Članek v reviji
• Tipologija: 1.01 - Izvirni znanstveni članek
• Organizacija: FRI - Fakulteta za računalništvo in informatiko
• Status publikacije: Objavljeno
• Različica publikacije: Objavljena publikacija
• Leto izida: 2025
• Št. strani: Str. 33509-33520
• Številčenje: Vol. 13
• PID: 20.500.12556/RUL-178163
• UDK: 004.8
• ISSN pri članku: 2169-3536
• DOI: 10.1109/ACCESS.2025.3543608
• COBISS.SI-ID: 227115523
• Datum objave v RUL: 20.01.2026

Gradivo je del revije

• Naslov: IEEE access
• Založnik: Institute of Electrical and Electronics Engineers
• ISSN: 2169-3536
• COBISS.SI-ID: 519839513

Licence

• Licenca: CC BY 4.0, Creative Commons Priznanje avtorstva 4.0 Mednarodna
• Povezava: http://creativecommons.org/licenses/by/4.0/deed.sl
• Opis: To je standardna licenca Creative Commons, ki daje uporabnikom največ možnosti za nadaljnjo uporabo dela, pri čemer morajo navesti avtorja.

Sekundarni jezik

• Jezik: Slovenski jezik
• Ključne besede: medjezikovno učenje, medjezikovni prenos, globoko učenje, strojno učenje