无人机松林图像早期松材线虫病害检测

doi:10.11707/j.1001-7488.LYKX20220898

摘要/Abstract

摘要：

目的: 针对无人机松林图像中早期松材线虫病害特征不明显、尺度多变导致的目标漏检、误检问题，提出一种基于深度学习的早期松材线虫病害检测方法。方法: 首先，为达到在无人机机载端的模型部署需求，提出一种降低计算量和参数量的松材线虫病害检测方法；其次，为获取早期松材线虫病害更准确的特征，采用多种方式同时提取特征并融合以增强对有效特征的学习能力；然后，为进一步提高不同尺度特征的融合能力，添加注意力机制对齐相邻两级特征；最后，以辽宁抚顺大伙房试验林场无人机拍摄的早期松材线虫病害为研究对象，利用LablImg开源软件标注拍摄高度为100~240 m的图像，构建无人机早期松材线虫病害图像（EPI）数据集。结果: 在EPI数据集上的测试结果表明，本研究方法的平均精度（AP）最高达95.2%，相比YOLOv5s、YOLOX-s和YOLOv6s的AP分别提高3.1%、4%和0.9%；该模型体积仅12. 8 M，分别是YOLOv5s、YOLOX-s和YOLOv6s模型体积的23.6%、17.8%和8.9%。结论: 本研究方法具有较高识别精度，同时模型体积较小，可为无人机在机载端识别早期松材线虫病害提供可能。

关键词: 无人机图像, 松材线虫病, 深度学习, 目标检测

Abstract:

Objecive: Aiming at the problem of missed and misdetection of targets due to the inconspicuous features and variable scales of early pine wood nematode infestation in UAV (unmanned aerial vehicle) forest images, this paper proposes a detection method for early pine wood nematode infestation. Method: Firstly, in order to facilitate the deployment at the airborne end of the UAV, this paper proposes a pine wood nematode infestation detection method that reduces the amount of computation and the number of parameters; secondly, in order to obtain more accurate features of early pine wood nematode infestation, the method uses multiple ways of simultaneous extraction of features and fusion to enhance the learning ability of effective features; then, in order to further improve the fusion ability of features at different scales, the method uses the attention mechanism to Aligning the adjacent two levels of features; finally, taking the early pine nematode infestation photographed by UAV in Dahuofang experimental forest in Fushun, Liaoning Province as the object of study, the LablImg open source software was used to annotate the images captured at the heights of 100?240 m, and to construct the early pine nematode infestation images (EPI) (early pine wood nematode infestation) dataset. Results: The test results on the EPI dataset showed that the Average Precision (AP) of this paper's method reached up to 95.2%, which was improved by 3.1%, 4% and 0.9% compared with the AP of YOLOv5s, YOLOX-s and YOLOv6s, respectively; the volume of the model was only 12.8 M, which was the volume of YOLOv5s, YOLOx-s and YOLOv6s, respectively, 23.6%, 17.8% and 8.9% of the model volume of YOLOX-s and YOLOv6s, respectively. Conclusion: In summary, the method in this paper has high recognition accuracy while keeping the model volume small, which provides the possibility for UAVs to recognize early pine wood nematode infestation at the airborne end.

Key words: unmanned aerial vehicle imagery, pine wilt disease, deep learning, object detection

中图分类号:

TP391

谢婉滢,刘文萍,王晗. 无人机松林图像早期松材线虫病害检测[J]. 林业科学, 2024, 60(9): 124-133.

Wanying Xie,Wenping Liu,Han Wang. UAV Images of Pine Forests for Early Detection of Pine Wood Nematode Infestation[J]. Scientia Silvae Sinicae, 2024, 60(9): 124-133.

图/表 15

图1

图2

图3

图4

图5

图6

图7

图8

表1

表2

表3

表4

图9

表5

图10

参考文献 0

	理永霞, 王　璇, 刘振凯, 等. 松材线虫致病机理研究进展. 中国森林病虫, 2022, 41 (3): 11- 20.
	Li Y X, Wang X, Liu Z K, et al. Research advance of pathogenic mechanism of pine wood nematode. Forest Pest and Disease, 2022, 41 (3): 11- 20.
	骆有庆. 高度重视虫传危险性森林病害——松材线虫病. 昆虫知识, 2001, 38 (2): 150.
	Luo Y Q. Insect mediated forest disease: Bursaphelenchus xylophilus. Chinese Bulletin of Entomology, 2001, 38 (2): 150.
	孙　钰, 周　焱, 袁明帅, 等. 基于深度学习的森林虫害无人机实时监测方法. 农业工程学报, 2018, 34 (21): 74- 81. doi: 10.11975/j.issn.1002-6819.2018.21.009
	Sun Y, Zhou Y, Yuan M S, et al. UAV real-time monitoring for forest pest based on deep learning. Transactions of the Chinese Society of Agricultural Engineering, 2018, 34 (21): 74- 81. doi: 10.11975/j.issn.1002-6819.2018.21.009
	陶　欢, 李存军, 谢春春, 等. 基于HSV阈值法的无人机影像变色松树识别. 南京林业大学学报(自然科学版), 2019, 43 (3): 99- 106.
	Tao H, Li C J, Xie C C, et al. Recognition of red-attack pine trees from UAV imagery based on the HSV threshold method. Journal of Nanjing Forestry University (Natural Sciences Edition), 2019, 43 (3): 99- 106.
	Bochkovskiy A, Wang C Y, Liao H Y M. 2020. YOLOv4: optimal speed and accuracy of object detection. arXiv: 2004.10934.
	Dai J F, Qi H Z, Xiong Y W, et al. 2017. Deformable convolutional networks. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 764−773.
	Ge Z, Liu S T, Wang F, et al. 2021. YOLOX: exceeding YOLO series in 2021. arXiv: 2107.08430.
	Girshick R. 2015. Fast R-CNN. Proceedings of the IEEE International Conference on Computer Vision (ICCV), 1440−1448.
	Girshick R, Donahue J, Darrell T, et al. 2014. Rich feature hierarchies for accurate object detection and semantic segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 580−587.
	Glorot X, Bordes A, Bengio Y. 2020. Deep sparse rectifier neural networks. Proceedings of the fourteenth international conference on artificial intelligence and statistics. Proceedings of the fourteenth international conference on artifcial intelligence and statistics, 315−323.
	He K M, Zhang X Y, Ren S Q, et al. 2015. Deep residual learning for image recognition. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 770−778.
	Hou Q B, Zhou D Q, Feng J S. 2021. Coordinate attention for efficient mobile network design. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 13713−13722.
	Hu J, Shen L, Sun G. 2018. Squeeze-and-excitation networks. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7132−7141.
	Huang S H, Lu Z C, Cheng R, et al. 2021. FaPN: feature-aligned pyramid network for dense image prediction. Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV), 864−873.
	Li C Y, Li L, Jiang H L, et al. 2022. YOLOv6: a single-stage object detection framework for industrial applications. arXiv: 2209.02976.
	Lin T Y, Dollár P, Girshick R, et al. 2017. Feature pyramid networks for object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 2117−2125.
	Liu S, Qi L, Qin H F, et al. 2018. Path aggregation network for instance segmentation. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 8759−8768.
	Liu W, Anguelov D, Erhan D, et al. 2016. SSD: single shot MultiBox detector//European conference on computer vision. Cham: Springer, 21−37.
	Misra D. 2019. Mish: a self regularized non-monotonic activation function. arXiv: 1908.08681.
	Osco L P, dos Santos de Arruda M, Marcato J Jr, et al. A convolutional neural network approach for counting and geolocating citrus-trees in UAV multispectral imagery. ISPRS Journal of Photogrammetry and Remote Sensing, 2020, 160, 97- 106. doi: 10.1016/j.isprsjprs.2019.12.010
	Pan X R, Ge C J, Lu R, et al. 2021. On the integration of self-attention and convolution. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 815−825.
	Redmon J, Divvala S, Girshick R, et al. 2016. You only look once: unified, real-time object detection. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 779−788.
	Redmon J, Farhadi A. 2017. YOLO9000: better, faster, stronger. Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), 7263−7271.
	Redmon J, Farhadi A. 2018. YOLOv3: an incremental improvement. arXiv: 1804.02767.
	Ren S Q, He K M, Girshick R, et al. Faster R-CNN: towards real-time object detection with region proposal networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2017, 39 (6): 1137- 1149. doi: 10.1109/TPAMI.2016.2577031
	Tan M X, Pang R M, Le Q V. 2020. EfficientDet: scalable and efficient object detection. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR), 10781−10790.
	Tian Z, Shen C H, Chen H, et al. 2019. FCOS: fully convolutional one-stage object detection. arxiv: 1904.01355.
	Wang C S, Wang Q, Wu H R, et al. Low-altitude remote sensing opium poppy image detection based on modified YOLOv3. Remote Sensing, 2021, 13 (11): 2130. doi: 10.3390/rs13112130
	Wang C Y, Liao H Y, Wu Y H, et al. 2020. CSPNet: a new backbone that can enhance learning capability of CNN. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR) Workshops, 390−391.
	Woo S, Park J, Lee J Y, et al. 2018. CBAM: convolutional block attention module//European conference on computer vision. Cham: Springer, 3−19.
	Zhang J, He L, Karkee M, et al. Branch detection for apple trees trained in fruiting wall architecture using depth features and regions-convolutional neural network (R-CNN). Computers and Electronics in Agriculture, 2018, 155, 386- 393. doi: 10.1016/j.compag.2018.10.029

方法 Method	F(%)	AP0.5(%)	AP0.5:0.95(%)
+普通卷积块 Standard convolutional block +CSP	90.2	92.1	64.4
+普通卷积块 Standard convolutional block +MFEM3	90.3	93.4	67.6
+MFEB +CSP	90.3	93.6	67.5
+MFEB +MFEM3	91.0	94.1	67.8

方法 Method	F(%)	AP0.5(%)	AP0.5:0.95(%)
+FAFM	90.0	91.9	64.1
+FAFM +CA	91.7	94.2	66.4
+FAFM +SENet	90.9	93.1	65.9
+FAFM +CBAM	72.8	74.0	43.1

层 Layer	输入图像尺寸 Input image size/pixels	模块 Module	n	s
C1	800×800×3	DS-Focus	—	2
C2	400×400×32（80）	MFEB	—	2
C2	200×200×64（160）	MFEM3	1	1
C3	200×200×64（160）	MFEB	—	2
C3	100×100×128（320）	MFEM3	3	1
C4	100×100×128（320）	MFEB	—	2
C4	50×50×256（640）	MFEM3	3	1
C5	50×50×256（640）	MFEB	—	2
C5	25×25×512（1280）	SPPF	—	1
C5	25×25×512（1280）	MFEM3	1	1

方法 Method	DS	MFEB	MFEM3	FAFM+CA	PANet引入新特征 PANet introduces new features	F(%)	AP0.5(%)	AP0.5:0.95(%)	P/M	FLOPs/G
YOLOv5s						90.2	92.1	64.4	7.1	16.3
策略1 Strategy 1	√					88.6	93.2	61.2	5.2	10.8
策略2 Strategy 2		√	√			91.0	94.1	67.8	4.6	13.4
策略3 Strategy 3				√		91.7	94.2	66.4	9.8	19.1
策略4 Strategy 4					√	90.5	92.5	64.9	7.3	17.4
策略5 Strategy5	√	√				91.3	94.5	66.9	5.5	15.3
策略6 Strategy 6	√	√	√			92.1	94.7	66.8	4.8	11.1
策略7 Strategy7	√	√	√	√		91.4	94.6	68.7	6.5	13.7
本研究方法-s Our method-s	√	√	√	√	√	93.1	95.2	68.8	6.57	13.9
YOLOv5m						91.4	92.4	68.5	21.3	50.2
策略8 Strategy 8	√					88.6	93.5	63.7	12.8	26.9
策略9 Strategy 9		√	√			91.8	94.7	70.2	13.6	36.8
策略10 Strategy10				√		91.2	94.3	69.0	26.1	55.8
策略11 Strategy11					√	90.9	92.9	68.8	15.5	52.1
策略12 Strategy12	√	√				91.6	94.4	68.1	13.5	40.2
策略13 Strategy13	√	√	√			92.9	94.9	68.4	9.9	28.4
策略14 Strategy 14	√	√	√	√		92.6	95.6	68.7	15.7	33.5
本研究方法-m Our method-m	√	√	√	√	√	93.4	96.2	69.5	15.8	33.9

方法 Method	主干网络 Backbone network	输入图像尺寸 Input image size/pixels	内存占用量 Memory usage/M	每秒处理图像的帧数 Frames per second (FPS)	AP0.5(%)
Faster R-CNN	VGG16	800×800	982.6	4.8	86.7
Faster R-CNN	ResNet101	800×800	377.5	3.6	87.9
Fcos	ResNet101	800×800	409.7	7.5	89.9
RetinaNet	ResNet101	640×640	808.2	5.8	88.7
SSD	VGG16	640×640	—	—	88.5
YOLOv5s	ResNet50	800×800	242.0	64.1	93.2
YOLOv5s	CSPDarkNet53	800×800	54.3	109.0	92.1
YOLOv5m	CSPDarkNet53	800×800	161	70.0	92.4
YOLOX-s	CSPDarkNet53	640×640	72.1	67.1	91.2
YOLOv6s	EfficientRep	800×800	144.0	85.5	94.3
本研究方法-s Our method-s	—	800×800	12.8	55.3	95.2
本研究方法-m Our method-m	—	800×800	30.6	38.0	96.2