Online Recognition and Counting with Deep Learning for Mango Images

doi:10.3969/j.issn.1000-2561.2020.03.002

Abstract

Abstract:

In order to realize the intelligent assessment of fruit yield, this paper carried out an research on the fruit recognition of Shengxin mango image in natural environment, and proposed an online method of recognition and counting with deep learning of mango images. Firstly, the recognition algorithm for mango image was realized using the Faster R-CNN Model, a deep learning framework. Secondly, an upload module for mango images based on Wechat applet and Web platform was developed, which can upload images to a server at all times and places. Thirdly, a server-client communication mode based on the TCP protocol and the Faster R-CNN assembly in MATLAB were adopted to construct an online analysis module, which realized the real-time and online recognition and counting for mango images. Finally, the recognition and counting results for a single picture, or all pictures in an orchard, were feedback to users through the Wechat applet program and Web page program, including the classification statistics of green mango and red mango. A total of 125 mango images in natural environment had been collected and analyzed using the method proposed in the paper. The results showed that the correct recognition and counting rate of mango fruit was 82.3%, among which the rate of missed detection and error detection was 11.7% and 8.6%, respectively, the average counting error was 4.2 and the average counting error rate was 7.9%. The experimental results demonstrated that, the method proposed in the paper was able to provide a scientific decision-making basis for the wisdom management of orchards through results quantity analysis obtained by images recognition and counting.

Key words: mango image, recognition and counting, Faster R-CNN, on-line method

CLC Number:

CEN Guanjun,HUA Junda,PAN Yiying,LIU Dahe,SU Beibei,ZHONG Zheng,ZHANG Liankuan,GAO Yan. Online Recognition and Counting with Deep Learning for Mango Images[J]. Chinese Journal of Tropical Crops, 2020, 41(3): 425-432.

Figures/Tables 7

Fig. 1 General frame of Faster R-CNN

Fig.2 Transmission and storage of mango tree image data

Fig. 3 Real-time analysis of mango tree image

Fig. 4 Data feedback interfaces of mobile phone client

Fig. 5 Data feedback interfaces of PC client

Fig. 6 Mango image detection results by Faster R-CNN A: The image of a whole mango tree with low fructification density; B: The image detection results of a whole mango tree with low fructification density; C: The image of a whole mango tree with high fructification density; D: The image detection results of a whole mango tree with high fructification density; E: The crown image of a mango tree; F: The image detection results of a mango tree crown.

Tab. 1 Mango image recognition and counting performance by Faster R-CNN

统计量Statistics	值Value	统计量Statistics	值Value
人工计数	2291	平均计数误差	4.2
检出个数	1968	平均计数误差率	7.9%
正确识别并计数个数	1619	计数识别准确率	82.3%
漏检个数	267	漏检率	11.7%
误检个数	198	误检率	8.6%

References 15

[1]	张亚静, 邓烈, 李民赞 , 等. 基于图像处理的柑橘测产方法[J]. 农业机械学报, 2009,40(S1):97-99.
[2]	马翠花, 张学平, 李育涛 , 等. 基于显著性检测与改进 Hough 变换方法识别未成熟番茄[J]. 农业工程学报, 2016,32(14):219-226.
[3]	廖崴, 郑立华, 李民赞 , 等. 基于随机森林算法的自然光照条件下绿色苹果识别[J]. 农业机械学报, 2017,48(S1):86-91.
[4]	傅隆生, 冯亚利, Elkamil Tola , 等. 基于卷积神经网络的田间多簇猕猴桃图像识别方法[J]. 农业工程学报, 2018,34(2):205-211.
[5]	Kurtulmus F L, Won S V A . Immature peach detection in colour images acquired in natural illumination conditions using statistical classifiers and neural network[J]. Precision Agriculture, 2014,15(1):57-79.
[6]	Qureshi W S, Payne A, Walsh K B , et al. Machine vision for counting fruit on mango tree canopies[J]. Precision Agriculture, 2017,18(2):224-244.
[7]	Dorj U O, Lee M, Yum S . An yield estimation in citrus orchards via fruit detection and counting using image processing[J]. Computers and Electronics in Agriculture, 2017,140:103-112.
[8]	Bargoti S, Underwood J . Deep fruit detection in orchards [C]//2017 IEEE International Conference on Robotics and Automation (ICRA), Singapore: IEEE, 2017.
[9]	Bargoti S, Underwood J P . Image segmentation for fruit detection and yield estimation in apple orchards[J]. Journal of Field Robotics, 2017,34(6):1039-1060.
[10]	Nanaa K, Mohamed, Rizon M, Rahman M N A , et al. Detecting mango fruits by using randomized hough transform and back propagation neural network [C]// International Conference on Information Visualisation, Paris: IEEE, 2014: 388-391.
[11]	Ramesh K, Avadesh M, Omkar N . MangoNet: A deep semantic segmentation architecture for a method to detect and count mangoes in an open orchard[J]. Engineering Applications of Artificial Intelligence, 2019,77:59-69.
[12]	He K M, Zhang X Y, Ren S Q , et al. Spatial pyramid pooling in deep convolutional networks for visual recognition[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2015,37(9):1904-1916.
[13]	Ren S Q, He K M, Girshick R , et al. Faster R-CNN: Towards real-time object detection with region proposal networks[J]. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017,39(6):1137-1149.
[14]	Hamuda E, Glavin M, Jones E . A survey of image processing techniques for plant extraction and segmentation in the field[J]. Computers and Electronics in Agriculture, 2016,125:184-199.
[15]	DeChant C, Wiesner-Hanks T, Chen S , et al . Automated identification of northern leaf blight-infected maize plants from field imagery using deep learning[J]. Phytopathology, 2017,107:1426-1432.