Bilingual inquiry classification using cross-lingual word embeddingsTRAJKOVA, ELENA (Avtor) Dobrišek, Simon (Mentor) Justin, Tadej (Komentor) multilingual text classificationparallel corporacross-lingual embeddingsLLMemail classificationThe ability to process diverse textual content is crucial as people communicate and share information across multiple languages. Multilingual text classification allows organizations to effectively analyze and comprehend customer feedback, inquiries, and sentiments expressed in multiple languages. It provides valuable advantages for businesses looking to enhance customer service and optimize their CRM (Customer Relationship Management) systems. By leveraging this capability, enterprises can address a variety of customer needs, providing customized and efficient support in their preferred languages, and at the same time saving costs and optimizing resources. Multilingual text classification presents challenges in representing and analyzing text data from different languages, due to differences in vocabulary, grammar, and semantics. Moreover, uneven data distribution among languages makes training a model more challenging, as some languages face the problem of limited data availability. Cross-lingual word embeddings provide a solution to the aforementioned challenges, as they represent words from different languages in a shared space. With this, we can analyze and classify text in multiple languages more effectively, using a single classification model. In this study, we assess the task of automatically classifying customer inquiries into target departments, such as legal, IT, accounting, etc. We conduct this problem as a bilingual text classification task, using Slovenian and English text data. We develop a pipeline for classifying texts from either English or Slovenian source language. We focus on implementing a parallel corpus and representing both languages in a shared space. With the use of a single but bilingual feature space, we include typical text patterns from both languages. This allows us to classify any incoming text sample without needing information about its source language. The feature space is generated using a statistical feature extraction method, TF-IDF (Term Frequency-Inverse Document Frequency). Alongside TF-IDF, we evaluate a pre-trained sentence-transformers model for extracting the embeddings and provide a side-by-side comparison of the results. We provide a qualitative and quantitative comparison of two classification algorithms, Logistic Regression and Random Forest, in bilingual and monolingual experimental scenarios. Logistic Regression outperforms Random Forest in all experimental setups. In classifying the bilingual data, Logistic Regression correctly classifies 97,6% with TF-IDF embeddings and 92,7% using pre-trained embeddings. In contrast, Random Forest achieves an accuracy of 93,8% using TF-IDF embeddings and 90,6% with the pre-trained embeddings. Additionally, we suggest incorporating Large Language Models (LLMs) into the workflow to provide an immediate response. This study focuses on evaluating the bilingual text classification using a suitable dataset, so we do not conduct a performance evaluation of the LLM itself. However, we propose utilizing the capabilities of LLM in Q\&A tasks. In this context, key information needed to generate the response is provided to the LLM. We use the training documents as a search database for defining the prompt context. Using cosine similarity, we obtain the most similar documents from the database based on their bilingual embeddings. The findings demonstrate that the bilingual embeddings group the documents with a clear distinction between the languages. Consequently, the retrieved relevant documents are in the same language as the input sample, making this process language-invariant.20232023-10-09 14:03:43Magistrsko delo/naloga151568sl