Cross-lingual approach to abstractive summarization

Žagar, Aleš (author), Robnik Šikonja, Marko (mentor), Farkaš, Igor (comentor)

Automatic text summarization is a process of extracting important information from texts and presenting that information in the form of a summary. Abstractive summarization approaches progressed using deep neural networks, but results are not yet satisfactory, especially for languages where large training sets do not exist. In several natural language processing tasks, cross-lingual model transfers are succesfully applied for low-resource languages where large enough datasets are not available. For summarization such cross-lingual transfer was so far not attempted due to non-reusable decoder side of neural models. In our work, we used a pretrained English summarization model based on deep neural networks and sequence-to-sequece architecture to summarize Slovene news articles. We solved the problem with inadequate decoder by using an additional language model for target language text generation. We developed five models with different training sample sizes. The results were assessed by automatic and human evaluation. Our cross-lingual model performance is similar to the existing Slovene abstractive summarizer. We also discuss some interdisciplinary aspects, raised by our work.

• Language: English
• Keywords: automatic summarization
• Work type: Master's thesis/paper
• Typology: 2.09 - Master's Thesis
• Organization: PEF - Faculty of Education
• Year: 2020
• PID: 20.500.12556/RUL-117026
• COBISS.SI-ID: 20240131
• Publication date in RUL: 23.06.2020

Secondary language

• Language: Slovenian
• Title: Medjezikovni pristop k abstraktivnemu povzemanju

Abstract

Avtomatsko povzemanje besedil označuje proces pridobivanja pomembnih informacij iz besedila in njihovo predstavitev v obliki povzetka. Pristopi k abstraktivnemu povzemanju so precej napredovali z uporabo globokih nevronskih mrež, a so rezultati še vedno lahko nezadovoljivi, kar še posebej velja za jezike brez velikih učnih množic. Pri mnogih nalogah obdelave naravnega jezika se medjezikovni prenosi kažejo za uspešne tudi pri jezikih, ki za rešitev problema nimajo ustrezno velikih učnih množic. Doslej še ni bilo poskusa medjezikovnega prenosa povzemanja zaradi neenostavne ponovne uporabe dekodirnika nevronskih modelov. V delu smo za povzemanje slovenskih novičarskih člankov uporabili vnaprej naučen model za povzemanje v angleškem jeziku, ki temelji na globokih nevronskih mrežah in arhitekturi zaporedje v zaporedje. Problem neustreznega dekodirnika smo rešili z uporabo dodatnega jezikovnega modela za generiranje besedil v ciljnem jeziku. Razvili smo pet modelov, ki se med seboj ločijo po številu učnih primerov. Rezultate smo evalvirali z avtomatsko in človeško evalvacijo. Z našim medjezikovnim modelom smo dosegli primerljiv rezultat z obstoječim abstraktivnim povzemalnikom za slovenski jezik. V delu obravnavamo tudi relevantne interdisciplinarne vidike.

• Keywords: avtomatsko povzemanje