Development of an autonomous kiwifruit harvester : a thesis presented in partial fulfilment of the requirements for the degree of Doctor of Philosophy in Industrial Automation at Massey University, Manawatu, New Zealand.

Abstract

The already intensive labour requirements within the New Zealand kiwifruit industry are increasing. Furthermore, ZESPRI Group Limited is targeting a threefold increase in industry return by 2025 (from approximately $NZ1Billion to $NZ3Billion). Development of autonomous mechanised solutions to assist manual labour is emerging as a strategic necessity. The objective of this research was to develop a commercially viable autonomous kiwifruit harvester (AKH). The AKH must be capable of operating within variable and complex on-orchard environments to minimise manual labour requirements. Successful completion required development and integration of autonomous: 1. Fruit identification and localisation 2. Custom robotic arms with soft fruit extraction harvesting hands 3. Custom robotic arm for soft fruit handling 4. Transportation platform with navigational sensing and strategies 5. Storage bin collection and drop-off The AKH has four robotic harvesting arms with hands specifically designed to mimic the human fruit harvesting action. Remotely mounted stereoscopic vision identifies and localises fruit. The fruit locations are mapped into the harvesting arms’ coordinate space allowing fruit extraction. The presented system configuration resolves the slow harvest rates experienced by other systems. Practical on-orchard testing identified additional environmental complexities that present the greatest challenge to consistent fruit identification. These are mainly from natural lighting effects. Stereoscopic machine vision (SMV) was investigated as the primary navigation sensor. However, diverse environmental conditions (lighting and structure appearance) made consistent object detection unreliable. Consequently, a light detection and ranging/SMV combination was used to achieve reliable navigational object detection and fruit storage bin identification. Practical on-orchard testing and analysis verified AKH operational ability (testing was limited due to a vine killing bacterial (Psa-V) outbreak restricting orchard access): 1. Fruit identification (83.6% of crop) with combined localisation and extraction accuracy of 3.6mm in three-dimensional space 2. More gentle fruit harvesting and handling than humans harvesting 3. Reliable object detection and path planning for navigation. Over the twenty metre scanning range 96% of the in-row objects were correctly classified to reliably determine the drive path 4. Reliable fruit storage bin identification and localisation (98% correct classification) 5. Commercially viable manufacture cost less than $130,000 per unit 6. Although full commercial operation was not achieved, modifications are identified to rectify the limitations Key system improvements are presented for: 1. High intensity artificial lighting for increased fruit identification rates. Natural sunlight variations affected identification ability, minimising this affect will increase identification rates 2. Alter the storage bin filling arm geometry to permit complete storage bin filling 3. Sensing the robotic arms’ position to resolve positioning errors Furthermore, ZESPRI Group Limited is targeting a threefold increase in industry return by 2025 (from approximately $NZ1Billion to $NZ3Billion). Development of autonomous mechanised solutions to assist manual labour is emerging as a strategic necessity. The objective of this research was to develop a commercially viable autonomous kiwifruit harvester (AKH). The AKH must be capable of operating within variable and complex on-orchard environments to minimise manual labour requirements. Successful completion required development and integration of autonomous: 1. Fruit identification and localisation 2. Custom robotic arms with soft fruit extraction harvesting hands 3. Custom robotic arm for soft fruit handling 4. Transportation platform with navigational sensing and strategies 5. Storage bin collection and drop-off The AKH has four robotic harvesting arms with hands specifically designed to mimic the human fruit harvesting action. Remotely mounted stereoscopic vision identifies and localises fruit. The fruit locations are mapped into the harvesting arms’ coordinate space allowing fruit extraction. The presented system configuration resolves the slow harvest rates experienced by other systems. Practical on-orchard testing identified additional environmental complexities that present the greatest challenge to consistent fruit identification. These are mainly from natural lighting effects. Stereoscopic machine vision (SMV) was investigated as the primary navigation sensor. However, diverse environmental conditions (lighting and structure appearance) made consistent object detection unreliable. Consequently, a light detection and ranging/SMV combination was used to achieve reliable navigational object detection and fruit storage bin identification. Practical on-orchard testing and analysis verified AKH operational ability (testing was limited due to a vine killing bacterial (Psa-V) outbreak restricting orchard access): 1. Fruit identification (83.6% of crop) with combined localisation and extraction accuracy of 3.6mm in three-dimensional space 2. More gentle fruit harvesting and handling than humans harvesting 3. Reliable object detection and path planning for navigation. Over the twenty metre scanning range 96% of the in-row objects were correctly classified to reliably determine the drive path 4. Reliable fruit storage bin identification and localisation (98% correct classification) 5. Commercially viable manufacture cost less than $130,000 per unit 6. Although full commercial operation was not achieved, modifications are identified to rectify the limitations Key system improvements are presented for: 1. High intensity artificial lighting for increased fruit identification rates. Natural sunlight variations affected identification ability, minimising this affect will increase identification rates 2. Alter the storage bin filling arm geometry to permit complete storage bin filling 3. Sensing the robotic arms’ position to resolve positioning errors