Izdelava orodja za shranjevanje in analizo spletnih medijskih vsebin

KAFERLE, MARKO (author), Stančin, Sara (mentor)

Diplomska naloga z naslovom Izdelava orodja za shranjevanje in analizo spletnih medijskih vsebin je bila zastavljena s ciljem implementacije hitrega in sistematičnega preučevanja javnih medijskih vsebin. V diplomski nalogi smo si zadali sledeče cilje: ustvariti program za pridobivanje spletnih vsebin; vzpostaviti povezavo med omenjenim programom in strežnikom za shranjevanje podatkov; beležiti in analizirati pridobljeno vsebino. Naš začetni program za pridobivanje vsebin smo implementirali v programskem jeziku Java. Sprva smo se želeli posvetiti analizi slovenskih besedil, vendar smo se zaradi težav pri implementaciji preusmerili na angleški jezik. Uporabili smo NLP (Natural Language Processing) tehnike, ki so temeljile na knjižnici Stanford CoreNLP. Podatke smo shranjevali v SQL (Structured Query Language) bazo v njihovi osnovni in lematizirani obliki, pri čemer smo pridobljene vsebine razdelili na več sklopov. Temu je sledila analiza s pomočjo specifično zastavljenih funkcij. V glavnem delu analize smo se ukvarjali s testiranjem hitrosti poizvedovanja glede na uporabljeno metodo. Začeli smo z enostavnimi klici na celotni bazi, nadaljevali z uporabo pogledov, kasneje pa dodali še indekse. Rezultati so bili skladni s pričakovanji. Uporaba pogledov in indeksov je znatno skrajšala čas poizvedb.

• Language: Slovenian
• Keywords: Stanford CoreNLP, SQL, lematizacija, indeksiranje
• Work type: Undergraduate thesis
• Organization: FE - Faculty of Electrical Engineering
• Year: 2023
• PID: 20.500.12556/RUL-151812
• COBISS.SI-ID: 169455363
• Publication date in RUL: 20.10.2023

Secondary language

• Language: English
• Title: Online media content retrieval and analysis

Abstract

The thesis was conceived with the idea of implementing a fast and systematic way of analysing media coverage, namely web articles. The main focus of the thesis addresses: the implementation of a program for obtaining web content; bridging said program with a server for data storage; tracking and analysing the obtained data. Our starter program was implemented in Java. At the beginning, we wanted to focus on data in Slovene, but due to problems with the implementation of a working analyser, we shifted our focus to English. Using Stanford CoreNLP, we utilized various NLP (Natural Language Processing) techniques. The data was restructured to its most basic form using lemmatization and then stored in a SQL (Structured Query Language) server. What followed were experiments on said data using specific functions for specific subgroups. The main focus of the analysis was testing the speed of querying based on different factors. The first step used just a normally written query. The second step was focused on views, and the last optimization included the use of indexing. As predicted, the runtime significantly decreased with each additional step.

• Keywords: Stanford CoreNLP, SQL, lemmatization, indexing