Improving Large Language Models for Machine Translation Using Synthetic Preference DataVajda, Dario (Avtor) Robnik Šikonja, Marko (Mentor) Vreš, Domen (Komentor) machine learningmachine translationlarge language modelsLarge language models have emerged as effective machine translation systems. We explore how a general instruction-tuned large language model can be improved for machine translation using relatively few easily produced data resources. With Slovene as our primary use case, we improve the GaMS-9B-Instruct model using Direct Preference Optimization (DPO) training on a programmatically curated and enhanced subset of a public dataset. As DPO requires pairs of quality-ranked instances, we generated its training dataset by translating English Wikipedia articles using two LLMs, GaMS-9B-Instruct and EuroLLM-9B-Instruct. We ranked the resulting translations based on heuristics coupled with automatic evaluation metrics such as COMET. The evaluation shows that our fine-tuned model outperforms both models involved in the dataset generation. In comparison to the baseline models, the fine-tuned model achieved a COMET score gain between 0.02 and 0.04 on translating a wide variety of texts. It also avoids language and formatting errors more consistently.20262026-07-13 13:15:04Diplomsko delo/naloga184701sl