Implementacija sistema za ocenjevanje esejev na podlagi koherence in semantične skladnosti

Simončič, Žiga (author), Bosnić, Zoran (mentor)

Cilj magistrskega dela je implementirati sistem za ocenjevanje esejev v angleškem jeziku. Zgledujemo se po metodologiji obstoječega sistema, ki poleg ocenjevanja sintakse uporablja tudi mere koherentnosti in semantične skladnosti. Metodologijo implementiramo v grafičnem okolju Orange, s prijaznim vmesnikom, opcijsko uporabo vektorskih vložitev za predstavitev besedila in možnostjo nadaljnjega razvoja sistema. Sistem evalviramo na podatkih, dostopnih na spletnem mestu Kaggle in, kolikor je mogoče, rezultate primerjamo z rezultati dosedanje metodologije in jih podrobno analiziramo. Poglobimo se tudi v izbiranje atributov za izboljšanje rezultatov. Glavni prispevki dela obsegajo (1) implementacijo sistema, (2) preprostost uporabe in (3) izboljšave dosedanjega dela, vključno z dodatnimi računskimi možnostmi in podrobno analizo izbiranja atributov za izboljšanje rezultatov.

• Language: Slovenian
• Keywords: samodejno ocenjevanje esejev, semantična skladnost, Orange
• Work type: Master's thesis/paper
• Typology: 2.09 - Master's Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2020
• PID: 20.500.12556/RUL-120555
• COBISS.SI-ID: 32227843
• Publication date in RUL: 22.09.2020

Secondary language

• Language: English
• Title: Implementation of an automated essay evaluation system based on coherence and semantic consistency

Abstract

The goal of this thesis is the implementation of an essay grading system. We lean heavily on the methodology of an existing system, which, besides using syntactical measurements, also uses coherence and semantic cosistency measures. We implement the methodology in the Orange data mining tool, with a firendly user interface, optional use of word embeddings for word representation and the possibility for further developments of the system. The system is evaluated on public datasets from the Kaggle website. The results are to the most possible extent compared with the results of the existing methodology and analyzed in detail. We also compare several attribute selection methods, which improve our results. Main contributions of this work are comprised of (1) implementation of the system, (2) ease of use and (3) improvements upon previous work, including additional computing options and detailed attribute selection analysis.

• Keywords: automatic essay evaluation, semantic consistency, Orange