Details

Large Language Models for Japanese–Croatian translation : human evaluation and macroeconomic implications
ID Karlović, Ratomir (Author), ID Bobanović Dasko, Mieta (Author), ID Srdanović, Irena (Author)

.pdfPDF - Presentation file, Download (781,42 KB)
MD5: C38C5E30959C4017B68B6252DD0B76CD
URLURL - Source URL, Visit http://journals.uni-lj.si/ala/article/view/27227 This link opens in a new window

Abstract
Large Language Models (LLMs) are increasingly used for translation, yet their value depends on preserving meaning rather than producing fluent output. This study evaluates seven LLMs on Japanese–Croatian translation, a low-resource, typologically distant language pair. Using rubric-based human evaluation of adequacy, fluency, terminology, and register, we compare model performance. Results show a stable ranking: qwen3 performs best, followed by phi4 and gemma3, while qwen2 performs worst. Performance differences reflect structural reconstruction, particularly argument recovery, aspectual mapping, lexical precision, and register. Qualitative analysis also reveals limited differentiation within the South Slavic continuum and pragmatic inconsistencies. Although productivity effects were not measured, improved translation adequacy may reduce post-editing and verification effort.

Language:English
Keywords:LLM, Japanese-Croatian machine translation, human evaluation, macroeconomic implications
Work type:Article
Typology:1.01 - Original Scientific Article
Organization:FF - Faculty of Arts
Publication status:Published
Publication version:Version of Record
Year:2026
Number of pages:Str. 237-261
Numbering:Vol. 16, no. 2
PID:20.500.12556/RUL-190497 This link opens in a new window
ISSN on article:2232-3317
DOI:10.4312/ala.16.2.237-261 This link opens in a new window
Publication date in RUL:04.11.2026
Views:161
Downloads:45
Metadata:XML DC-XML DC-RDF
:
Copy citation
Share:Bookmark and Share

Record is a part of a journal

Title:Acta linguistica asiatica
Publisher:Založba Univerze v Ljubljani
ISSN:2232-3317
COBISS.SI-ID:255902464 This link opens in a new window

Licences

License:CC BY-SA 4.0, Creative Commons Attribution-ShareAlike 4.0 International
Link:http://creativecommons.org/licenses/by-sa/4.0/
Description:This Creative Commons license is very similar to the regular Attribution license, but requires the release of all derivative works under this same license.

Secondary language

Language:Slovenian
Title:Veliki jezikovni modeli za prevajanje med japonščino in hrvaščino : človeško vrednotenje in makroekonomske implikacije
Abstract:
Veliki jezikovni modeli (LLM) se vse pogosteje uporabljajo za prevajanje, vendar je njihova vrednost odvisna predvsem od ohranjanja pomena. Prispevek vrednoti sedem LLM-jev pri prevajanju med japonščino in hrvaščino, jezikovnim parom z omejenimi viri in veliko tipološko oddaljenostjo. Na podlagi ocenjevalne rubrike za prevodno ustreznost, tekočnost, terminološko natančnost in register primerjamo uspešnost modelov. Rezultati kažejo stabilno razvrstitev: qwen3 se je izkazal najbolje, sledita phi4 in gemma3, najslabši pa je qwen2. Razlike v uspešnosti odražajo predvsem strukturno rekonstrukcijo, zlasti obnovo vezljivostne strukture, preslikavo glagolskega vida, natančnost izbire besed in register. Kvalitativna analiza razkriva tudi omejeno razlikovanje znotraj južnoslovanskega jezikovnega kontinuuma ter pragmatične nedoslednosti. Čeprav produktivnost ni bila neposredno merjena, bi večja prevodna ustreznost lahko zmanjšala obseg popravkov in napor pri preverjanju prevodov.

Keywords:veliki jezikovni modeli, LLM, strojno prevajanje med japonščino in hrvaščino, človeško vrednotenje, makroekonomske implikacije

Projects

Funder:EC - European Commission
Funding programme:NextGeneration
Project number:IIP_UNIPU_010138
Name:From Dynamic Vocabulary to Virtual Interaction: Development of a Foreign Language Learning Model in Tourism Using Corpora, Artificial Intelligence, and Extended Reality – Language for Real and Virtual Worlds
Acronym:VIRAI

Funder:Other - Other funder or multiple funders
Funding programme:SPIN
Project number:IP.1.1.03.0120
Name:Konverzacijski Order Management
Acronym:INFOBIP

Funder:Other - Other funder or multiple funders
Funding programme:SPIN
Project number:IP.1.1.03.0158
Name:Projektiranje i razvoj nove generacije laboratorijskog informacijskog sustava
Acronym:iLIS

Funder:Other - Other funder or multiple funders
Funding programme:SPIN
Project number:IP.1.1.03.0039
Name:Istraživanje i razvoj inovativnog sustava preporuka za napredno gostoprimstvo u turizmu
Acronym:InnovateStay

Funder:EC - European Commission
Funding programme:Digital Europe
Project number:101083838
Name:European Digital Innovation Hub Adriatic Croatia
Acronym:EDIH Adria

Similar documents

Similar works from RUL:
Similar works from other Slovenian collections:

Collection

This document is a part of these collections:
  1. Acta linguistica asiatica

Back