3D Robotics and LMM for Vineyard Inspection

Abstract. Autonomous mobile robotic solutions are increasingly being explored in precision agriculture to aid human workers in labour-intensive or repetitive tasks. Moreover, the emergence of foundation models in vision-based AI domain presents an opportunity to perform automated interpretation of in-field collected data. This study presents a cost-effective mobile robotic research platform designed for autonomous vineyard inspection: it integrates mission planning, real-world navigation and a post-processing pipeline of multimodal data. The system, based on the Leo rover, is equipped with LiDAR, RGB cameras and GNSS-visual-inertial positioning, ensuring reliable operation in GNSS-degraded vineyard environments. We propose a novel methodology for automating several stages of the workflow using various open and in-situ collected data. The robotic platform and processing pipeline were validated through simulation and field experiments, demonstrating its capability for autonomous navigation, 3D reconstruction, AI-based fruit detection and an initial plant health assessment through Large Multimodal Models (LMM). Results show that while 3D mapping provides highresolution spatial data, AI-driven object detection and vision models require further domain adaptation for reaching reliable and trustable operation. The study highlights the feasibility of cost-effective mobile robotic solutions in vineyard monitoring and the potential of integrating AI to enhance agricultural automation.