An algorithm for operational navigation in urban development using reinforcement learning

Abstract. This paper presents a navigation system for unmanned aerial vehicles (UAVs) operating in urban environments, where the main challenges include changes to the city layout that may not be reflected on existing maps. To effectively address this issue, we have employed semantic segmentation algorithms based on deep convolutional neural networks, which enable accurate identification and classification of urban objects from visual data. This segmentation plays a crucial role in real-time environmental perception, allowing UAVs to distinguish objects such as buildings and vehicles. Initial routes are calculated using an enhanced Dijkstra’s algorithm, which determines the shortest path through the urban landscape. However, these routes may require adjustments due to the absence of certain objects on the maps. Along with semantic segmentation and the enhanced Dijkstra’s algorithm, reinforcement learning methods are utilized to adjust the navigation routes generated by the algorithms. The reinforcement learning model continuously learns from the UAV’s interactions with the environment, optimizing the route by considering safety and efficiency factors. The training and debugging of the algorithm were conducted in a developed synthetic 3D scene. Through simulation and testing in the constructed scene, the proposed navigation system demonstrated improvements in route safety and adaptability compared to routes generated by the enhanced Dijkstra’s algorithm.