Topic analysis of Slovenian news and social media

HLADNIK, JUŠ (author), Robnik Šikonja, Marko (mentor)

Topic modeling is an unsupervised machine learning technique that aims to discover hidden semantic structures within large collections of text documents, thus facilitating the exploration and understanding of vast textual data. We conduct a comprehensive comparison of four popular topic modeling algorithms, namely LDA, NMF, Top2vec and BERTopic, in the context of the Slovenian language. To assess the performance of these algorithms we use topic coherence and topic diversity quantitative evaluation and additionally manually interpret extracted topics. Our results demonstrate that all models achieve higher topic coherence on the news corpus compared to tweets. While BERTopic is the only algorithm to produce satisfactory results on the tweets corpus, all models perform well on the news corpus. Furthermore, we introduce a novel method, MBTS (Maximum Bipartite Topic Similarity), for comparing the similarity of topic models and evaluating their stability. This method relies on semantic similarity and maximum graph bipartite matching. Our findings have important implications for the selection and application of topic modeling algorithms in the context of the Slovenian language. Moreover, the MBTS method opens up a new and important area of topic model stability evaluation.

• Language: English
• Keywords: topic modeling, language models, Slovene language, topic model stability and similarity, natural language processing
• Work type: Master's thesis/paper
• Typology: 2.09 - Master's Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2023
• PID: 20.500.12556/RUL-146738
• COBISS.SI-ID: 158061571
• Publication date in RUL: 09.06.2023

Secondary language

• Language: Slovenian
• Title: Tematska analiza slovenskih novic in družbenih omrežij

Abstract

Modeliranje tem je nenadzorovana metoda strojnega učenja, ki si prizadeva odkriti skrite semantične strukture znotraj velikih zbirk dokumentov, s čimer omogoča raziskovanje in razumevanje obsežnih besedilnih podatkov. Celovito primerjamo štiri priljubljene algoritme za modeliranje tem, in sicer LDA, NMF, Top2vec in BERTopic, v kontekstu slovenskega jezika. Modele kvantitativno ovrednotimo z metrikama koherentnost tem in raznolikost tem, poleg tega odkrite teme tudi ročno pregledamo in interpretiramo. Naši rezultati kažejo, da vsi modeli dosegajo višjo koherenco tem na korpusu novic v primerjavi s tviti. Medtem ko algoritem BERTopic edini dosega zadovoljive rezultate na korpusu tvitov, na korpusu novic vsi modeli dosegajo dobre rezultate. Poleg tega predstavimo novo metodo, MBTS (največja dvostranska podobnost tem), za primerjavo podobnosti modelov za modeliranje tem in ocenjevanje njihove stabilnosti. Ta metoda temelji na semantični podobnosti in maksimalnem dvostranskem ujemanju grafov. Naše ugotovitve imajo pomembne posledice za izbiro in uporabo algoritmov za modeliranje tem v kontekstu slovenskega jezika. Poleg tega metoda MBTS odpira novo in pomembno področje evalvacije stabilnosti modelov za modeliranje tem.

• Keywords: modeliranje tem, jezikovni modeli, slovenščina, stabilnost in podobnost modelov za modeliranje tem, obdelava naravnega jezika