This thesis examines whether a simple, fully local Retrieval-Augmented Generation (RAG) pipeline, composed exclusively of open-source components and executed on central processing units, can serve as a chatbot supporting teaching. The aim of the thesis is to determine whether such a RAG pipeline is capable of retrieving and browsing documentation and answering questions with sufficient speed and quality for real-time use in the educational process. The work was carried out in collaboration with a company where,
due to strict data protection requirements and limited infrastructure, the use of large language models (LLMs) via cloud services is not acceptable. We implement a naive RAG pipeline using the ChromaDB vector database, three small general-purpose embedding models, and three smaller instruction tuned LLMs, which are run via the Ollama inference server. Individual components are first evaluated separately and then within the full pipeline. We find that the RAG pipeline, which satisfies our architectural criteria, does not achieve the quality and performance required for use in an educational setting.
|
