20.500.12556/RUL-186491 Uporaba spodbujevanega učenja za krmiljenje simuliranega robotskega manipulatorja Application of Reinforcement Learning to the Control of a Simulated Robotic Manipulator V diplomskem delu smo zasnovali in implementirali simulacijsko okolje za spodbujevano učenje (angl. reinforcement learning) za robotski manipulator Universal Robots UR10e. Obravnavali smo razvoj okolja, ki omogoča učenje robotske manipulacije v simulaciji. Okolje temelji na simulatorju MuJoCo, knjižnicah Gymnasium in Gymnasium Robotics ter ogrodju PyTorch z uporabo knjižnice Stable-Baselines3. Razvili smo prilagojeni okolji za nalogi doseganja cilja (angl. Reach) ter prijemanja in dvigovanja predmeta (angl. Pick-and-Place). Implementirali smo kartezično vodenje robotskega manipulatorja z uporabo inverzne kinematike po metodi dušenih najmanjših kvadratov (angl. Damped Least Squares), prilagodili smo opazovalni in akcijski prostor, dekompozicijo zahtevnejše naloge ter ustrezni funkciji nagrajevanja. Ovrednotili smo algoritme SAC (angl. Soft Actor-Critic), PPO (angl. Proximal Policy Optimization) in TQC (angl. Truncated Quantile Critics). Pri nalogi doseganja cilja sta algoritma SAC in TQC dosegla 100 % uspešnost, PPO pa približno 80 %. Naloga prijemanja in dvigovanja predmeta je bila zahtevnejša. SAC je dosegel približno 40 % uspešnost, TQC največ približno 30 %, PPO pa naloge ni uspešno dokončal. Rezultati kažejo, da razvito okolje omogoča uspešno učenje osnovne manipulacijske naloge in je osnova za nadaljnji razvoj zahtevnejših nalog ter prenos naučenih strategij na fizične robotske sisteme. In this thesis, we designed and implemented a reinforcement learning simulation environment for the Universal Robots UR10e robotic manipulator. We focused on developing an environment that enables learning robotic manipulation tasks within a simulation. The environment is built upon the MuJoCo simulator, the Gymnasium and Gymnasium Robotics libraries, and the PyTorch framework, utilizing the Stable-Baselines3 library. We developed custom environments for the tasks of reaching a target and grasping and lifting an object. We implemented Cartesian control of the robotic manipulator using inverse kinematics based on the Damped Least Squares method, defined custom observation and action spaces, decomposed the more complex task, and formulated appropriate reward functions. We evaluated the SAC (Soft Actor-Critic), PPO (Proximal Policy Optimization), and TQC (Truncated Quantile Critics) algorithms. For the reaching task, the SAC and TQC algorithms achieved a 100 % success rate, while PPO achieved approximately 80 %. The grasping and lifting task proved more challenging. SAC achieved a success rate of approximately 40 %, TQC reached a maximum of around 30 %, and PPO failed to successfully complete the task. The results demonstrate that the developed environment enables successful learning of a basic manipulation task and serves as a foundation for developing more complex tasks and transferring learned strategies to physical robotic systems. spodbujevano učenje MuJoCo UR10e PPO SAC TQC reinforcement learning MuJoCo UR10e PPO SAC TQC true false false Slovenski jezik Angleški jezik Diplomsko delo/naloga 2026-09-02 12:06:37 2026-09-02 12:06:49 2026-09-09 10:48:04 0000-00-00 00:00:00 2026 0 0 0000-00-00 NiDoloceno NiDoloceno NiDoloceno 0000-00-00 0000-00-00 0000-00-00 63428 290472963 Pestotnik_Zan_-_Uporaba_spodbujevanega_ucenja_za_krmiljenje_simuliranega_robotskega_manipulatorj.pdf Pestotnik_Zan_-_Uporaba_spodbujevanega_ucenja_za_krmiljenje_simuliranega_robotskega_manipulatorj.pdf 1 BCDC6AD0E83F79ACA325AB0C2DE4E821 cab182061173855b010e2ed7ba9885ca34d701b262c1d5c729ff401682189276 96a1d606-a6b5-11f1-9b0d-0050569b8976 https://repozitorij.uni-lj.si/Dokument.php?lang=slv&id=245756 Fakulteta za elektrotehniko 0 0 0