Appraisal analysis and AI chatbots : do we even need humans?

Križan, Agata (author), Barbič, Aja (author)

Artificial intelligence (AI) is rapidly transforming various fields, including linguistics, by offering new tools for the analysis and generation of human language. As AI tools, particularly chatbots, have become increasingly sophisticated, questions have arisen about their capacity to replicate complex human linguistic processes, such as those covered by the appraisal framework developed by Martin and White (2005). The appraisal framework examines how three main categories – attitude, graduation, and engagement – are expressed in discourse at the semantic level. This paper investigates how AI chatbots, MS Copilot, ChatGPT, and Claude approach appraisal analysis in a selected text, highlighting similarities and notable differences in comparison to human analysis. The findings, although based on analysis of a single text, provide valuable insights into the advantages and drawbacks of AI in mimicking human-like appraisal analysis, which might be beneficial when conducting appraisal research.

• Language: English
• Keywords: appraisals, human and AI comparative analyses, ChatGPT, MS Copilot, Claude
• Work type: Article
• Typology: 1.01 - Original Scientific Article
• Organization: FF - Faculty of Arts
• Publication status: Published
• Publication version: Version of Record
• Publication date: 01.01.2025
• Year: 2025
• Number of pages: Str. 35-52
• Numbering: Vol. 22, no. 1
• PID: 20.500.12556/RUL-180370
• UDC: 81:004.89
• ISSN on article: 1581-8918
• DOI: 10.4312/elope.22.1.35-52
• COBISS.SI-ID: 241708035
• Publication date in RUL: 06.03.2026

Record is a part of a journal

• Title: ELOPE : English language overseas perspectives and enquiries
• Shortened title: ELOPE
• Publisher: Slovensko društvo za angleške študije, Slovensko društvo za angleške študije, Znanstvena založba Filozofske fakultete Univerze v Ljubljani, Založba Univerze v Ljubljani
• ISSN: 1581-8918
• COBISS.SI-ID: 212890112

Licences

• License: CC BY-SA 4.0, Creative Commons Attribution-ShareAlike 4.0 International
• Link: http://creativecommons.org/licenses/by-sa/4.0/
• Description: This Creative Commons license is very similar to the regular Attribution license, but requires the release of all derivative works under this same license.

Secondary language

• Language: Slovenian
• Title: Analiza jezika vrednotenja in pogovorni sistemi : ali ljudi sploh potrebujemo?

Abstract

Umetna inteligenca hitro preoblikuje različna področja, vključno z jezikoslovjem, tako da ponuja nova orodja za analizo in ustvarjanje človeškega jezika. Ker postajajo orodja umetne inteligence, zlasti pogovorni sistemi, vse bolj izpopolnjena, se pojavljajo vprašanja o njihovi sposobnosti ponovitve kompleksnih človeških jezikovnih procesov, kot so tisti zajeti v jeziku vrednotenja, ki sta ga razvila Martin in White (2005). Okvir jezika vrednotenja preučuje, kako se v diskurzu izražajo tri glavne kategorije – odnos, stopnjevanje odnosov in vključenost – na semantični stopnji. Članek raziskuje kako pogovorni sistemi, MS Copilot, ChatGPT in Claude pristopijo k analizi jezika vrednotenja v izbranem besedilu, tako da osvetli podobnosti kot tudi pomembne razlike skozi primerjavo s človeško analizo. Ugotovitve, čeprav temeljijo na enem izbranem besedilu, omogočijo dragoceni vpogled v prednosti in pomanjkljivosti umetne inteligence pri posnemanju človeške jezikovne analize, kar je lahko koristno pri raziskovanju jezika vrednotenja.

• Keywords: jezik vrednotenja, primerjalne analize, umetna inteligenca, ChatGPT, MS Copilot, Claude