MODELING OF A UAV CONTROL SYSTEM USING THE REINFORCEMENT LEARNING MODEL

Abstract

The article investigated the most common methods of UAV control using both traditional methods using the proportional- integral-differential (PID) control law and intelligent systems. The use of artificial intelligence (AI) algorithms, in particular reinforcement learning algorithms (Reinforcement Learning), ensures adaptability to changing dynamics and the environment. When creating a UAV model, the main indicators of the UAV control system were taken into account – Lift Amplitude and Lift Difference. The use of these parameters in modeling allows for realistic behavior of the drone in two-dimensional space, which makes it possible to accurately assess the effectiveness of control algorithms. The Lift Amplitude is responsible for basic stability and the performance of vertical tasks, while the Lift Difference provides the ability to maneuver and reach specified waypoints. A performance evaluation system (scoring) was proposed, which is based on the task of navigating to random points in space, represented in the form of «balloons». During the research, a mathematical model and control algorithms were developed that take into account the dynamics of UAV movement in two-dimensional space, taking into account the influence of inertia, gravity and propeller thrust. Simulation was carried out using the DQN, SAC and SAC algorithms with a modification of the differential thrust level. A comparative analysis of the effectiveness of the above approaches in different scenarios was performed. To test the effectiveness of the algorithms, a simulation environment based on the Python programming language was created using the NumPy, Matplotlib, Pygame and Stable-Baselines3 libraries. The environment allows you to model the tasks of flight stabilization, navigation and obstacle avoidance. In order to create a universal platform for researching control systems, a simulation environment based on the Python language was developed. This environment allows you to test algorithms in conditions close to real ones, without the need to use expensive equipment.