Employees of large enterprises need fast access to information about products,
services, and procedures. Because this information is scattered across multiple
sources, manual searching is slow and often yields incomplete answers. This
thesis addresses the problem by designing, implementing, and evaluating
an agent system for the employees of Telekom Slovenije, based on retrievalaugmented generation (RAG). Its knowledge base is built by a data-ingestion
pipeline that converts the telekom.si website content into Markdown, splits it
into chunks, and stores them as vector embeddings in Azure AI Search. On
top of it runs an agent application, built on the LlamaIndex framework using
the ReAct pattern, which performs hybrid search and cites sources in every
answer. Employees access the system through a conversational interface with
single sign-on. Alongside a cloud language model, the system also supports
locally hosted models within the company infrastructure. We evaluated the
system on 105 real user questions: the answer success rate is 94.3%, and the
first part of the answer typically appears after about five seconds. A survey
among the test users confirms the time savings. Knowledge-base coverage
remains the main limitation and priority for future work.
|