Sarcasm detection with transfer learning from multiple sourcesĐoković, Lazar (Avtor) Robnik Šikonja, Marko (Mentor) natural language processinglarge language modelssarcasm detectionneural machine translationBERT modelGPT modelLlama modelSarcasm detection is a natural language processing task of classifying whether an utterance is sarcastic or not. It is closely related to sentiment analysis since it often inverts surface sentiment. Despite the great interest and research done by the sentiment analysis community, it remains a challenging problem. This is because sarcastic sentences are highly dependent on context, and they are often accompanied by various non-verbal cues. Recent work in sarcasm detection mostly focuses on the Transformer architecture of neural networks and its application in high-resourced languages like English. To build a sarcasm detection dataset for Slovene, we leverage two modern techniques in machine translation and language modeling. The first approach uses a medium-size Transformer model trained specifically for neural machine translation, while the second method utilizes a very large generative model. We explore the viability of such datasets and how the size of a pretrained Transformer affects its ability to detect sarcasm. We use this data to train ensembles of Transformer-based models. We evaluate model performance using established methodologies. Our results show that larger models generally outperform smaller ones, and that ensembling can slightly improve sarcasm detection performance. Our best ensemble approach achieves an $\text{F}_1$-score of 0.765.20242024-08-27 14:10:01Diplomsko delo/naloga160384sl