Reinforcement learning of large language models based on information propagation in graphsMihevc, Anže (Avtor) Žitnik, Slavko (Mentor) Bonchi, Francesco (Komentor) Artificial IntelligenceLarge Language ModelsReinforcement LearningAlignmentGraphsLarge Language Models (LLMs) are rapidly transforming information systems, increasingly contributing to the generation and propagation of online content. In social media environments, where AI-generated posts and agents are becoming pervasive, understanding how LLMs behave and adapt within social graph structures remains largely unexplored, particularly in dynamic, process-based settings studied in network science. Existing research on diffusion and influence processes typically assumes overly simplified, one-dimensional models of user behavior, overlooking the rich social and linguistic dynamics of real interactions. This thesis introduces a novel socially aware alignment framework for LLMs, designed to integrate language modeling with network-based information propagation dynamics. We first analyze the embedding space of social dimensions, demonstrating that several dimensions, such as trust, conflict, and support, are moderately correlated, and we extend this space with additional classifiers for stance detection. Building on this foundation, we propose a generalization of the Binary Cascade Model (BCM) that incorporates multi-dimensional, socially grounded user representations. An efficient reward formulation is developed to exploit the structure of the active set during propagation, enabling scalable optimization. The complete reinforcement learning pipeline employs Proximal Policy Optimization (PPO) to align the LLM’s generation behavior with the social context of the network. The framework is extensively evaluated on both synthetic graphs, generated with controllable community and feature structures, and real-world cascade data from the Brexit Twitter($X$) discourse. Results show that the proposed alignment method successfully models realistic propagation dynamics, revealing inherent biases in LLMs toward specific stances and instabilities in highly polarized settings. The findings have dual implications: technically, the framework advances socially informed alignment of language models for applications such as public communication and information campaigns; ethically, it raises concerns about potential misuse for opinion manipulation. Future work will focus on improving PPO stability, enhancing reward predictability, and exploring larger model architectures for more robust socially aligned behavior.20252025-11-11 09:40:02Magistrsko delo/naloga175853VisID: 37761COBISS_ID: 259379459sl