Natural language processing (NLP) has advanced significantly in recent years. An important part of NLP is sarcasm detection which enables better understanding of sentiment in texts. There is little research on sarcasm detection in Slovene.
In this thesis we explore sarcasm detection in Slovene using monolingual Slovene (SloBERTa) and multilingual (Llama3 8B and GPT-4o) large language models. We used two datasets in English, namely the Reddit comment and news headlines datasets, which we first translated using machine translation. We used fine-tuning and prompt engineering and evaluated their success rate. We also tried to improve the models' success rate by adding explanations to our examples.
The best results were achieved through fine-tuning with Llama3 8B, followed by SloBERTa. Results from prompt engineering with GPT-4o were slightly worse. While adding explanations led to a significant improvement of GPT-4o, the results did not surpass those obtained by fine-tuning the significantly smaller SloBERTa model and Llama3 8B model. The results with Llama3 8B using prompt engineering were only slightly better than random guessing and adding explanations provided minimal improvements.
|
