In this thesis, we designed and implemented a reinforcement learning simulation environment for the Universal Robots UR10e robotic manipulator. We focused on developing an environment that enables learning robotic manipulation tasks within a simulation. The environment is built upon the MuJoCo simulator, the Gymnasium and Gymnasium Robotics libraries, and the PyTorch framework, utilizing the Stable-Baselines3 library.
We developed custom environments for the tasks of reaching a target and grasping and lifting an object. We implemented Cartesian control of the robotic manipulator using inverse kinematics based on the Damped Least Squares method, defined custom observation and action spaces, decomposed the more complex task, and formulated appropriate reward functions. We evaluated the SAC (Soft Actor-Critic), PPO (Proximal Policy Optimization), and TQC (Truncated Quantile Critics) algorithms.
For the reaching task, the SAC and TQC algorithms achieved a 100 % success rate, while PPO achieved approximately 80 %. The grasping and lifting task proved more challenging. SAC achieved a success rate of approximately 40 %, TQC reached a maximum of around 30 %, and PPO failed to successfully complete the task. The results demonstrate that the developed environment enables successful learning of a basic manipulation task and serves as a foundation for developing more complex tasks and transferring learned strategies to physical robotic systems.
|
