Real-Time Landform Feature Detection and Segmentation Based on Heterogeneous MPSoCs for Lunar Robotic Exploration

Abstract. Real-time feedback and operation are crucial for the next generation of lunar rovers and robots, enabling more powerful and intelligent exploration. Traditional ground control methods face challenges in meeting the demands of real-time navigation and exploration for lunar rovers and robots due to issues such as latency and communication inefficiencies. The electronic systems of lunar rovers and robots are constrained by limited power supply, necessitating energy-efficient solutions. Multiprocessor System-on- Chips (MPSoCs) are considered capable of balancing the power consumption and performance requirements for lunar robotic exploration missions. MPSoCs integrate multiple processor cores and other components, such as memory and peripheral interfaces, and can include various processors such as ARM cores and Field-Programmable Gate Arrays (FPGAs). FPGA-based MPSoCs are highly customizable and energy-efficient, making them advantageous for addressing the challenges of edge computing tasks in lunar robotic exploration. In this paper, we present neural network inference accelerators using Vitis-AI on an FPGA heterogeneous MPSoC for lunar landform analysis, including landform feature detection and segmentation. Feature detection is implemented by the YOLOv5s-MobileNetV2 network, and segmentation is based on a Feature Pyramid Network (FPN). The dataset consists of images of the lunar surface taken by various lunar exploration missions. By utilizing inference acceleration and an improved neural network design, our accelerators demonstrate superior performance in energy efficiency, inference speed, and accuracy. The developments have been deployed on an edge computing device, the Xilinx Kria KV260 MPSoC platform, and experimental results show that the accelerators achieved a feature detection frame rate of 52 FPS and a segmentation frame rate of 27 FPS. The accuracy of the project meets or exceeds that of popular deep-learning models. The system operates at around 6.3 W of power for the detection task and 7.6 W for segmentation, making it energy efficient. The results suggest that such a system can be deployed in future lunar rovers and robots to enhance the effectiveness of exploration tasks.