In the thesis we have tested a method for improved classification of deep neural networks with prior knowledge of negation. State of the art language models, such as ELMo and BERT, are successful at text classification, but fail when there is negation involved. We adjusted pre-trained language models to work better with negation in Slovene. We modified the loss function of the neural networks and retrained the models. We have tested the method on a modified corpus with added negations of original sentences. The method successfully reduced the error in the negated sentences for masked language models, and it increased the accuracy for some tasks from the Slovene version of the SuperGLUE benchmark but decreased for others.
|
