Language models and task-driven learning for sarcasm detection

DIMITRIEVIKJ, ALEKSANDAR (author), Robnik Šikonja, Marko (mentor)

In natural language processing, sarcasm detection determines whether a given text is sarcastic or not. It can have many real-world applications such as machine translation. In this work, we present three language modelling approaches and adapt them to the task of sarcasm detection. Two approaches are pretrained language models, BERT uses the encoder part of the transformer architecture and GPT-3 uses the decoder part of the transformer. The third method uses a newly-proposed task-driven learning technique TLM. We evaluated the methods using well-known metrics such as classification accuracy, precision and recall. GPT-3 performed the best in almost every aspect, with BERT being a close second. Our findings showed that TLM is very dependent on the task data and is therefore not suitable for a general task such as sarcasm detection.

• Language: English
• Keywords: natural language processing, language models, sarcasm detection, transformer architecture
• Work type: Bachelor thesis/paper
• Typology: 2.11 - Undergraduate Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2022
• PID: 20.500.12556/RUL-138840
• COBISS.SI-ID: 125453827
• Publication date in RUL: 23.08.2022

Secondary language

• Language: Slovenian
• Title: Jezikovni modeli in učenje s prilagajanjem nalogi za prepoznavanje sarkazma

Abstract

Zaznavanje sarkazma je postopek ugotavljanja, ali je besedilo sarkastično ali ne. Avtomatsko prepoznavanje sarkazma je pomemben vidik obdelave naravnega jezika in ima lahko veliko aplikacij, npr strojno prevajanje. V delu predstavljamo tri pristope jezikovnega modeliranja in jih prilagajamo nalogi odkrivanja sarkazma. Dva pristopa sta vnaprej naučena jezikovna modela, BERT uporablja kodirni del transformerske arhitekture, GPT-3 pa uporablja dekodirni del transformerja. Tretja metoda, TLM, uporablja novo predlagano tehniko učenja, ki temelji na ekstrakciji podatkov glede na dano nalogo. Metode smo ovrednotili z uporabo dobro znanih metrik, kot so klasifikacijska točnost, natančnost in priklic. Metoda GPT-3 se je izkazala za najboljšo v skoraj vseh vidikih, BERT pa je bil na drugem mestu. Naše ugotovitve so pokazale, da je TLM zelo odvisen od podatkov dane naloge in zato ni primeren za splošno nalogo, kot je odkrivanje sarkazma.

• Keywords: obdelava naravnega jezika, jezikovni modeli, prepoznavanje sarkazma, arhitektura transformer