20.500.12556/RUL-174280 Retrieval-augmented code completion for local projects using large language models The use of large language models (LLMs) is becoming increasingly widespread among software developers. However, privacy and computational requirements are problematic with commercial solutions and the use of LLMs. In this work, we focus on using relatively small and efficient LLMs with 160M parameters that are suitable for local execution and augmentation with retrieval from local projects. We train two open transformer-based models, the generative GPT-2 and the retrieval-adapted RETRO, on open-source Python files, and empirically compare them, confirming the benefits of embedding-based retrieval. Furthermore, we improve our models’ performance with In-context retrieval-augmented generation (RAG), which retrieves code snippets using the Jaccard similarity of tokens. We evaluate In-context RAG on larger models and determine that, despite its simplicity, the approach is more suitable than using the RETRO architecture. Experimental results indicate that In-context RAG improves the code completion baseline by over 26 %, while RETRO improves over the similarly sized GPT-2 baseline by 12 %. We highlight the key role of proper tokenization in achieving the full potential of LLMs in code completion. large language models code completion retrieval-augmented generation in-context retrieval obdelava naravnega jezika veliki jezikovni modeli programsko inženirstvo generiranje kode true false true Angleški jezik Slovenski jezik Članek v reviji 2025-09-30 14:22:10 2025-09-30 14:22:12 2026-06-15 15:39:19 0000-00-00 00:00:00 2025 0 0 15 str. Vol. 292, art. 128596 Nov. 2025 0000-00-00 Zaloznikova Objavljeno NiDoloceno 0000-00-00 0000-00-00 0000-00-00 004.85:81'322 0957-4174 10.1016/j.eswa.2025.128596 242180867 RAZ_Hostnik_Marko_2025.pdf RAZ_Hostnik_Marko_2025.pdf 1 B2EEDC5CC010223F3B81D36A1CFE2040 bf73fbc53d072e776be9e970b684e6c164493bdf0b6cbfd7e52308118e1fb410 2dc4980c-9dfc-11f0-9328-0050569b8976 https://repozitorij.uni-lj.si/Dokument.php?lang=slv&id=219070 https://www.sciencedirect.com/science/article/pii/S0957417425022158 1 e46cb86d-9df7-11f0-9328-0050569b8976 https://repozitorij.uni-lj.si/Dokument.php?lang=slv&id=219065 Fakulteta za računalništvo in informatiko Fakulteta za matematiko in fiziko 0 0 0