湖北农业科学 ›› 2022, Vol. 61 ›› Issue (7): 145-151.doi: 10.14088/j.cnki.issn0439-8114.2022.07.027

• 信息工程 • 上一篇    下一篇

自然场景下多目标苹果识别定位融合算法研究

曹春卿, 张吴平, 李富忠, 韩冀皖, 余廷熙, 刘帅   

  1. 山西农业大学软件学院,山西 晋中 030801
  • 收稿日期:2021-12-04 出版日期:2022-04-10 发布日期:2022-05-04
  • 通讯作者: 张吴平(1973-),男,山西长治人,教授,研究生导师,主要从事植物表型学、旱作有机农业研究,(电子信箱)zwping@126.com。
  • 作者简介:曹春卿(1996-),男,河北秦皇岛人,在读硕士研究生,研究方向为农业苹果采摘机器人,(电话)19933630401(电子信箱)895663938@qq.com
  • 基金资助:
    山西省重点研发计划重点项目(201703D211002-2); 农业大数据创新平台(K481811076); 山西省重点研发计划项目(201803D221008-4)

Fusion algorithm in multi-target apple recognition and localization in natural scene

CAO Chun-qing, ZHANG Wu-ping, LI Fu-zhong, HAN Ji-wan, YU Ting-xi, LIU Shuai   

  1. College of Software, Shanxi Agricultural University, Jinzhong 030801, Shanxi,China
  • Received:2021-12-04 Online:2022-04-10 Published:2022-05-04

摘要: 自然场景下苹果采摘对目标的精准识别和三维定位是苹果智能采摘设备的关键技术。融合YOLOv3算法和双目视觉技术,通过YOLOv3算法对多种自然场景下的样本进行训练,构建识别模型,利用双目视觉获取苹果图像,运用YOLOv3模型得到图像中目标苹果的二维坐标,再利用双目视觉视差原理得到深度坐标信息,从而实现对目标苹果的三维空间定位。将该算法应用于不同自然场景下苹果的识别和定位,并进行识别效果和定位精度的评价。结果表明,在光照不均、果实上存在阴影并且存在相互遮挡的情况下,最小相对误差为0.193%,最大相对误差为3.670%;在夜晚光照不足且存在相互遮挡的情况下,最小相对误差为0.831%,最大相对误差为4.417%;有露水在苹果表面形成反射并且果实存在相互遮挡的情况下,最小相对误差为0.176%,最大相对误差为4.205%;在光线较弱、阴影小、存在遮挡时,最小相对误差为0.168%,最大相对为误差3.776%。研究中所运用的算法只需适量样本就可以满足不同场景下的识别和定位训练,在不同场景下的mAP(mean Average Precision)达96.60%。该算法具有较强的稳定性,能够识别重叠、夜间、光照折射等不同应用场景下的目标苹果,能够较好地满足智能机器人在识别定位方面的精度需求。

关键词: 苹果智能采摘, 双目视觉, YOLOv3, 深度信息, 算法

Abstract: The accurate recognition and three-dimensional positioning of apple picking targets in natural scenes are the key technologies of apple intelligent picking equipment. Integrating YOLOv3 algorithm and binocular vision technology, the samples in a variety of natural scenes were trained through YOLOv3 algorithm, the recognition model was constructed, apple imageswere obtained by binocular vision, the two-dimensional coordinates of the target apple in the image were obtained by using the YOLOv3 model, and then the depth coordinate information was obtained by using the binocular vision parallax principle to realize the three-dimensional spatial positioning of the target apple. The algorithm was applied to apple recognition and location in different natural scenes, and the recognition effect and location accuracy were evaluated. The results showed that the minimum relative error was 0.193% and the maximum relative error was 3.670% under the condition of uneven illumination, shadow on the fruit and mutual occlusion; In the case of insufficient illumination and mutual occlusion at night, the minimum relative error was 0.831% and the maximum relative error was 4.417%; The minimum relative error was 0.176% and the maximum relative error was 4.205% when dew forms reflection on the surface of apple and the fruits block each other; When the light was weak, the shadow was small and there was occlusion, the minimum relative error was 0.168%, and the maximum relative error was 3.776%. The algorithm used in the study only needed an appropriate number of samples to meet the recognition and positioning training in different scenes, and mAP (mean Average Precision) in different scenes was 96.60%. The algorithm has strong stability, can identify the target apples in different application scenarios such as overlap, night and light refraction, and can better meet the accuracy requirements of intelligent robot in recognition and positioning.

Key words: apple intelligent picking, binocular vision, YOLOv3, depth information, algorithm

中图分类号: