20.500.12556/RUL-190730 Učenje kooperativnih bojnih strategij z večagentnim globokim učenjem Learning cooperative combat strategies with multi-agent deep learning Učenje usklajenega vedenja in sodelovanja agentov v bojnih simulacijah, kjer več avtonomnih agentov deluje kot ekipa, predstavlja zahteven problem zaradi delnih opažanj agentov, komunikacije med njimi, nestacionarnosti večagentnega okolja, velikega prostora možnih stanj ter težkega pripisovanja nagrad posameznim agentom. V tem delu smo razvili simulacijsko okolje, kjer 2 ekipi treh agentov tekmujeta na zemljevidih z ovirami. Agente učimo z algoritmom MAPPO po načelu centraliziranega učenja in decentraliziranega izvajanja. V eksperimentu primerjamo vplive različnih tehnik pomnjenja, spodbujanja raziskovanja, sestave opažanj agentov, dolžine učenja, predstavitve okolja, arhitekture nevronskih mrež ter hierarhičnega upravljanja agentov. Dobljene politike ovrednotimo z neposrednimi medsebojnimi boji ter boji proti hevrističnim nasprotnikom na več zemljevidih, kjer smo poleg deleža zmag spremljali tudi ocene Elo (merilo relativne tekmovalne uspešnosti) ter ostale bojne in vedenjske metrike agentov. V zadnjem delu naloge smo problem ter modele prenesli še v 2.5-dimenzionalni prostor in ovrednotili njihovo uspešnost. Končni rezultat naloge je več uspešnih modelov bojevanja, simulacijsko okolje ter tridimenzionalna vizualizacija poteka bojev, ki omogoča dodaten vpogled in razlago nekaterih odločitev naučenih agentov. Rezultati kažejo, da najuspešnejše politike združujejo kratkoročni spomin zgodovine opažanj, pomnilniški pregledni zemljevid in doučenje z nižjo stopnjo učenja. Najboljša politika v turnirju več modelov doseže povprečni točkovni izkupiček 66,08 %. Learning coordinated behavior and cooperation among agents in combat simulations—where multiple autonomous agents operate as a team—poses a significant challenge. This complexity arises from factors such as partial observability, inter-agent communication, the non-stationary nature of multi-agent environments, a vast state space, and the difficulty of attributing rewards to individual agents. In this work, we developed a custom simulation environment in which two teams of three agents compete on maps containing obstacles. We train the agents using the MAPPO algorithm, following the paradigm of centralized training and decentralized execution. Through experiments, we evaluate the impact of various techniques—including memory mechanisms, exploration incentives, the composition of agent observations, extended training durations, environment representations, neural network architectures, and hierarchical agent control. The resulting policies are evaluated through direct head-to-head matches and contests against heuristic-based opponents across multiple maps. In addition to win rates, we tracked Elo ratings (a measure of relative competitive performance) as well as various combat and behavioral metrics. Finally, we extended the problem and the models to a 2.5-dimensional space and evaluated their performance there as well. The results of the project are several successful combat models, a simulation environment, and a 3D visualization of the combat sequences, which provide further insight into the rationale behind certain decisions made by the trained agents. The results show that the most successful policies combine a short-term memory of the history of observations, a memory map, and fine-tuning with a lower learning rate. The best policy in the multi-model tournament achieves an average score of 66.08 %. Računalniška grafika Večagentno spodbujevano učenje Kooperativne bojne strategije Simulacijska okolja Computer graphics Multiagent reinforcement learning Cooperative combat strategies Simulation environments true false false Slovenski jezik Angleški jezik Magistrsko delo/naloga 2026-11-02 13:10:05 2026-11-02 13:10:12 2026-11-03 04:08:28 0000-00-00 00:00:00 2026 0 0 0000-00-00 NiDoloceno NiDoloceno NiDoloceno 0000-00-00 0000-00-00 0000-00-00 1970-01-01 38597 Markelj_Bine_-_Ucenje_kooperativnih_bojnih_strategij_z_vecagentnim_globokim_ucenjem.pdf Markelj_Bine_-_Ucenje_kooperativnih_bojnih_strategij_z_vecagentnim_globokim_ucenjem.pdf 1 D5E5E6312BE3DA8BE549244B70672EF8 e43f47b6404eb7e6bb60b623a5503432e81b20373bc96a4ac61ad1c8afb2a6f5 4c8a4abb-d6b6-11f1-aea2-0050569b8976 https://repozitorij.uni-lj.si/Dokument.php?lang=slv&id=252311 Fakulteta za računalništvo in informatiko 0 0 0