In this work, we address the challenge of efficient persistent storage and retrieval of embedded data for large language model–based agents. Since continuously expanding the context window degrades performance and increases costs, we propose a cloud architecture built on a vector database. By examining the fundamental concepts of vector databases and analyzing Milvus and Qdrant, we establish the foundational knowledge needed to design the system architecture.
Our solution implements a multi-tenant architecture that uses Qdrant, an Envoy proxy, a Quarkus interface, and OpenFGA for security. Through extensive empirical testing of various index configurations, cluster setups, and quantization methods (such as rotational quantization), our results show that optimizing the use of tenant subgraphs and selecting appropriate quantization techniques can significantly reduce resource consumption and improve database throughput without sacrificing query accuracy.
|
