Trait-focused low-cost and stereo-based 3D plant modeling for phenotyping via deep neural detection

Abstract. Phenotyping, the measurement of plants physical traits, plays a pivotal role in advancing sustainable agricultural practices. Therefore, developing efficient, low-cost, means to generate such measures is vital. Though image based 2D-driven methods are commonly applied for that purpose due to their processing simplicity, it is clear that only 3D information can offer the necessary plant details. Notwithstanding, the generation of such data is challenged by the requirement to acquire a large set of images or the use of active sensors, which exhibit sensitivity to illumination and require lengthy acquisition campaigns. Consequently, 3D plant phenotyping is presently limited to controlled laboratory conditions and is hardly applied in actual growth setups. To address this shortcoming, this paper argues that by focusing on relevant plant traits, modeling can be simplified and the need for detailed plant reconstruction can be relieved. Accordingly, only a minimal set of images, specifically a stereo pair, can suffice for the reconstruction, thereby providing a low-cost sensing solution. To facilitate the reconstruction, we adapt an anchor-free detection deep neural network and integrate low- and high-level features to accurately detect our plant traits of interest. As the paper demonstrates our adapted network facilitates a robust 3D reconstruction of the entities of interest. Performance analysis demonstrates how our detection is reliable and accurate compared to standard anchor-free frameworks, translating to accurate reconstruction, as we validate against 3D plant scans.