Fruit recognition, task plan, and control for apple harvesting robots¹

Reconhecimento de frutas, plano de tarefas e controle para robôs de colheita de maçãs

ABSTRACT

Intelligent apple-harvesting robots use a staggered distribution of branches and leaves during operation, causing problems such as slow motion planning, low operational efficiency, and high path cost for multi-degrees-of-freedom (DOF) harvesting manipulators. This study presents an autonomous apple-harvesting robotic arm-hand composite system that aims to improve the operational efficiency of intelligent harvesting in dwarf anvil-planted apple orchards. The machine vision system for fruit detection uses the deep learning convolutional neural network (CNN) YOLOv7 and RGB-D camera online detection coupling technology to rapidly recognise apples. The spatial depth information of the fruit area was then extracted from the aligned depth image for precise positioning. Coordinate transformation was used to obtain the coordinates of the fruit under the coordinate system of the manipulator. Based on the informed rapid-exploration random tree (Informed-RRT*) algorithm and path-planning model, the identified target apples were harvested without collision path planning. In an apple-harvesting test, the recognition accuracy of the visual system was 89.4%, and the average time to harvest a single apple was 9.69 s, which was 4.8% faster than the mainstream general harvesting technology. Moreover, the harvesting time for a single apple was reduced by 1.7%. Thus, the proposed system enabled accurate and efficient fruit harvesting.

Key words:
apple harvesting robotic arm-hand composite; manipulator; deep learning; path planning; harvesting sequence planning

HIGHLIGHTS:

The labour-intensive harvesting process leads to high production costs of apples.

Automatic mechanized harvesting reduces labour costs and enhances competitiveness.

Deep learning and collision-free path planning to achieve lossless harvesting of apples can re-place manual harvesting.