Relational Deep Learning over Bibliographic Data

Stevanović, Đorđe (author), Šubelj, Lovro (mentor)

Research–information systems such as SICRIS record publications, collaborations, and projects over long periods, and form relational, time–stamped data that are naturally represented as heterogeneous temporal graphs. In this thesis, a leakage–safe forecasting pipeline is developed for mentorship outcomes by casting the SICRIS schema into a heterogeneous graph and evaluating models under seed–time splits with a five–year prediction horizon. The core of the model consists of: a Heterogeneous Graph Transformer, pre–norm residual layers, Jumping Knowledge aggregation, and stochastic graph regularization (DropEdge/Dropout). We compare against a strong tabular baseline, XGBoost. Tasks are defined at researcher–seed times and cover both binary outcomes (whether a doctoral or master mentorship occurs) and count outcomes (how many such mentorships occur) within the forward window. Empirical results on the SICRIS–derived graph demonstrate that the heterogeneous, time–aware model provides consistent gains over the tabular baseline. The analysis indicates that typed attention with relative temporal encoding and Jumping Knowledge aggregation improves the use of multi–hop, multi–relation evidence, and that stochastic graph regularization helps stabilize deeper stacks.

• Language: English
• Keywords: Relational Deep Learning, Heterogeneous Temporal Graphs, SICRIS, RelBench, Mentorship Prediction, Explainability
• Work type: Bachelor thesis/paper
• Typology: 2.11 - Undergraduate Thesis
• Organization: FRI - Faculty of Computer and Information Science
• Year: 2025
• PID: 20.500.12556/RUL-173293
• COBISS.SI-ID: 253761795
• Publication date in RUL: 15.09.2025

Secondary language

• Language: Slovenian
• Title: Globoko relacijsko učenje nad bibliografskimi podatki

Abstract

Raziskovalno-informacijski sistemi, kot je SICRIS, dolgotrajno beležijo publikacije, sodelovanja in projekte ter s tem tvorijo relacijske, časovno označene podatke, ki jih je smiselno predstaviti kot heterogene časovne grafe. V tej diplomski nalogi je razvit napovedni model za napovedovanje števila mentorstev, pri čemer je shema SICRIS preslikana v heterogeni graf, model pa je ovrednoten z delitvami po petletnem napovednem horizontu. Jedro modela sestavljajo: heterogeni grafovski transformer, rezidualne plasti, agregacija ”Jumping Knowledge” in stohastična regularizacija grafa (navključno odstranjevanje povezav v grafu in navključno odstranjevanje aktivacij). Primerjava je izvedena z močnim tabelaričnim modelom, XGBoost. Naloge so definirane za posamezne raziskovalce v različnih časovnih točkah in zajemajo tako binarne izide (vsaj eno mentorstvo doktoranda ali magistranta) kot številske izide (število takih mentorstev) znotraj izbranega časovnega okna. Empirični rezultati na SICRIS grafu kažejo, da heterogeni, časovno občutljivi model presega tabelarično izhodišče. Analiza nakazuje, da tipizirana pozornost z relativnim časovnim kodiranjem ter agregacija Jumping Knowledge bolj učinkovito modelira večskokovne in večrelacijske podatke, stohastična regularizacija grafa pa pomaga stabilizirati globlje plasti.

• Keywords: Relacijsko globoko učenje, Heterogeni časovni grafi, SICRIS, RelBench, Napovedovanje mentorstva, Razložljivost