UAV-assisted Multi-Object Tracking and Segmentation of Apples under Occlusions in Orchard Settings

Abstract. In precision agriculture, orchards present unique challenges for automated monitoring due to the dense foliage, complex tree structures, and frequent occlusions caused by branches, leaves, and overlapping fruits. To address these challenges, multi-object tracking and segmentation (MOTS) has been explored in general computer vision domains, aiming to simultaneously track and segment instance-level objects and maintain consistent identities across video frames. However, most existing studies focus on object-level detection without considering the temporal continuity and spatial consistency required for robust fruit monitoring over time. In this work, we implement one of the state-of-the-art MOTS methods, Grounded-SAM2, in an orchardian environment for tracking apples. In addition, four different UAV flight modes were conducted to explore the optimal solution for UAV-assisted MOTS. Our proposed evaluation framework, which relies on spatio-temporal consistency metrics and instance association heuristics, enabled the assessment of tracking performance without prior annotations.