Controlling a reusable rocket stage involves translation, rotation, changing mass, and actuator limits. In this thesis, a configurable Unity simulator for investigating this problem using reinforcement learning is presented. The model includes six degrees of freedom, propellant consumption, aerodynamic forces, and actuator response dynamics. Configurations, training artifacts, and telemetry ensure experiment traceability. Policies are trained using Proximal Policy Optimization (PPO).
The system is evaluated on hovering above a fixed point, tracking a sequence of target points, and landing on four legs. Each policy is tested in 250 episodes with deterministic action selection. For hovering, all episodes reach the 60~s time limit; the policy stabilizes altitude and motion but retains a horizontal offset. The tracking policy achieves a 76.4\% success rate, and the landing policy achieves 84.8\% across five equally represented difficulty levels. At the highest difficulty, the landing success rate is 36\%. The development runs link unwanted behavior to deficiencies in the reward function and simulator, which are investigated using telemetry and component tests. The remaining limitations concern control accuracy and the difference between training and deterministic policy execution.
|
