Text similarity comparison is an important tehnique in many modern systems, from search engines, recommendation systems to RAG technologies, where text similarity is used during the retrieval phase to select the most relevant documents for a user's query. Despite the widespread use of these approaches, no existing solution or tool was found that would allow users to directly compare two texts or an entire corpus of texts based on a selection from a range of vector models. This served as the motivation for developing the solution presented in this thesis. We developed a web application using the React library and Flask framework that allows users to enter two texts, select a vector model and additional text preprocessing options, such as case sensitivity, punctuation removal, and the simulation of typographical errors, noise and shuffled words. The application calculates similarity using selected metrics and additionally provides a visual representation of the vector embedding space using PCA, t-SNA and UMAP methods. In a separate corpus mode view, the user is allowed to enter a text corpus, select a vector model and a query, after which the application ranks the texts according to their relevance to the query, demonstrating the retrival phase of a RAG system.
|
