Summarization of web commentsMilačić, Katarina (Avtor) Robnik Šikonja, Marko (Mentor) word embeddingscross-lingual embeddingslow-resource languagesabstractive summarizationextractive summarizationdeep neural networkslanguage modelstransfer learningText summarization is a process of reducing a given text to a concise and fluent shorter version. With rapidly increasing amounts of textual data, automatic text summarization could save time and reduce work. This task is non-trivial since it requires knowledge of vocabulary, semantics and cognitive processing. Pre-trained language models such as BERT contain extensive language knowledge. They can be used to transfer models trained in resource-rich languages to low-resource languages. In this work, we leverage knowledge of two BERT models: CroSloEngual BERT and multilingual BERT for transfer learning. We test extractive and abstractive summarization approaches that extend BERT architecture. We test the proposed approach on dataset of Croatian comments without summaries. We evaluate models using ROUGE and BERTScore and perform human evaluation. Trained abstractive models are able to detect keywords and a general topic, but struggle with the languages not present in the training data and produce false information. Extractive summarization models are reliable and have a good coverage of topics and contain important sentences.20202020-11-13 10:25:00Magistrsko delo/naloga121983sl