2D tree crops training system improve computer vision application in field: A case study

In the last decade computer vision had an enormous evolution and its application in agriculture is expanding quickly with a lot of research done and several solutions already available on the market. This fact is due to the willing and the necessity to robotize agricultural process so to ease the spread of smart agriculture approaches and techniques to be more precise in responding to plants, environment, and human needs. Fruit crops sector is one of the most difficult agricultural sectors on which apply robotization because of its high level of complexity both at orchard and tree level. It is recognized that a simplification of the tree and orchard environment will certainly help in automate activity in fruit production so, lately the diffusion of less complex two-dimensional tree shapes is happening. This study wants to evaluate improvement in computer vision application for fruit detection problems, that 2D training systems should bring with them. To the knowledge of the authors this could be the first paper trying to quantify that. In the trial a YOLOv3 neural network was trained on three datasets containing 2D, 3D and mixed apple training system images. Two model specialized on 2D and 3D training system, and one specialized in mixed situation were obtained. These models were then cross evaluated to define their performances in each training system condition (2D, 3D and mixed). In add to that a ground truthing dataset, with a known number of real fruits, was utilized to evaluate which percentage of the real fruit number can be directly detected by the models and how much the different training system affect this capability. Results show that the developed models present generally poor performance for field application with max F1-score of 0.68. For all the model*dataset combination, mixed model resulted always the best, followed by 2D or 3D model when applied to their relative training system. Bests performances were achieved by two models out of three in 2D training system dataset suggesting that this shape improve fruit detection. 2D model performed better than 3D in mixed situation suggesting better training phase done with 2D system images. From ground truthing analysis, 2D training system improved models result from 2.4 to 11.5%.