基于深度双线性转换注意力机制网络的林业有害生物识别方法

doi:10.11707/j.1001-7488.LYKX20210787

Abstract

Abstract:

Objective: There are many forest pest species, and some of them are morphologically similar with small visual differences, and are difficult to be distinguished, which makes it difficult for forestry prevention and control technicians to quickly and accurately identify them. In regard to those problems, this paper proposes a fine-grained image identification method of forest pests based on deep bilinear transformation attention mechanism network (DBTANet). Method: The images of 60 forest pests and 14 harmful forest plants were targeted. The original images were enhanced by methods including horizontal mirror, brightness adjustment, Gaussian blur and Gaussian noise. All images were divided into training dataset, verification dataset and test dataset by a 6∶2∶2 ratio. The bilinear interpolation method was used to scale each image to a uniform size. The residual module in the ResNet network was improved. A deep bilinear conversion module and an attention mechanism module were added. DBTANet-101 model was developed to extract features and classify forest pest species. Three evaluation indexes of average accuracy, average recall rate and average F₁ score were used to evaluate the identification results of forestry pests by thd different models. Result: A total of 74 forest pest species were identified by five models of VGGNet-19, ResNet-50, ResNet-101, improved ResNet-50 and improved ResNet-101, respectively, and the average accuracy rates of the five models were 78.6%, 74.9%, 76.3%, 79.7% and 81.1%, respectively. On the basis of improved ResNet-101 of the residual module, the deep bilinear conversion module and the attention mechanism module were added, after then the average recognition rate and recall rate of 74 forestry pests increased by 10.2% and 12.1%, respectively. The average recognition rate of fine-grained images of 22 similar pests increased by 15.7%. Conclusion: The fine-grained image recognition method of forest pests based on deep bilinear transformation attention mechanism network (DBTANet) achieves the average identification rate of 74 forest pests by 91.3% and the average identification rate of 22 similar forest pest species by 85.1%. The bilinear transformation module and the attention mechanism can effectively improve the accuracy of the forest pest identification model.

Key words: forest pests, fine-grained image recognition, deep bilinear transformation, attention mechanism, ResNet

CLC Number:

Jiajie Su,Zheyu Zhang,Jiajun Xu,Bin Li,Jun Lü,Qing Yao. Forest Pest Identification Method Based on a Deep Bilinear Transformation Attention Mechanism Network[J]. Scientia Silvae Sinicae, 2023, 59(2): 121-128.

Figures/Tables 9

Fig.1

Fig.2

Fig.3

Fig.4

Fig.5

Table 1

Table 2

Fig.6

Fig.7

References 0

	陈　娟, 陈良勇, 王生生, 等基于改进残差网络的园林害虫图像识别. 农业机械学报, 2019, 50 (5): 187- 195. doi: 10.6041/j.issn.1000-1298.2019.05.022
	Chen J, Chen L Y, Wang S S, et al Pest image recognition of garden based on improved Residual Network. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50 (5): 187- 195. doi: 10.6041/j.issn.1000-1298.2019.05.022
	陈志威. 2017. 广州市林业有害生物普查. 广州 : 华南农业大学.
	Chen Z W. 2017. Investigation of forestry pest in Guangzhou city. Guangzhou: South China Agricultural University. [in Chinese]
	程　曦, 吴云志, 张友华, 等基于深度卷积神经网络的储粮害虫图像识别. 中国农学通报, 2018, 34 (1): 154- 158. doi: 10.11924/j.issn.1000-6850.casb16110146
	Cheng X, Wu Y Z, Zhang Y H, et al Image recognition of stored grain pests: based on Deep Convolutional Neural Network. Chinese Agricultural Science Bulletin, 2018, 34 (1): 154- 158. doi: 10.11924/j.issn.1000-6850.casb16110146
	高巨虎, 王桂林, 李慧杰, 等自动虫情测报灯在林业害虫测报中的应用. 中国森林病虫, 2011, 30 (3): 36- 39. doi: 10.3969/j.issn.1671-0886.2011.03.011
	Gao J H, Wang G L, Li H J, et al Application of automatic forecast light in the forecast forest pests. Forest Pest and Disease, 2011, 30 (3): 36- 39. doi: 10.3969/j.issn.1671-0886.2011.03.011
	孔建磊, 金学波, 陶　治, 等基于多流高斯概率融合网络的病虫害细粒度识别. 农业工程学报, 2020, 36 (13): 148- 157. doi: 10.11975/j.issn.1002-6819.2020.13.018
	Kong J L, Jin X B, Tao Z, et al Fine-grained recognition of diseases and pests based on multi-stream Gaussian probability fusion network. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36 (13): 148- 157. doi: 10.11975/j.issn.1002-6819.2020.13.018
	李珏闻, 许胜利, 王立宇, 等应用云计算和物联网技术促进林草有害生物监测预报智慧发展. 林业和草原机械, 2020, 1 (3): 1- 6.
	Li J W, Xu S L, Wang L Y, et al Application of cloud computing and internet of things technology promoting intelligent development of monitoring and forecasting forest and grass pests. Forestry And Grassland Machinery, 2020, 1 (3): 1- 6.
	李孝华. 2020. 基于卷积神经网络的蝴蝶识别系统设计与实现. 重庆 : 西南大学.
	Li X H. 2020. The classification of butterfly based on Convolutional Neural Network. Chongqing : Southwest University. [in Chinese]
	林　强福建省林业有害生物灾害航空监测作业设计与效果评估系统. 福建农林大学学报 (自然科学版), 2015, 44 (2): 165- 168.
	Lin Q Route design and impact assessment system for forest pest aviation monitoring in Fujian Province. Journal of Fujian Agriculture and Forestry University (Natural Science Edition), 2015, 44 (2): 165- 168.
	刘文定, 田洪宝, 谢将剑, 等基于全卷积神经网络的林区航拍图像虫害区域识别方法. 农业机械学报, 2019, 50 (3): 179- 183. doi: 10.6041/j.issn.1000-1298.2019.03.019
	Liu W D, Tian H B, Xie J J, et al Identification methods for Forest Pest areas of UAV aerial photography based on Fully Convolutional Networks. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50 (3): 179- 183. doi: 10.6041/j.issn.1000-1298.2019.03.019
	吕玫霞. 2020. 无人机林地图像传输APP及虫害检测系统. 北京 : 北京林业大学.
	Lü M X. 2020. Research and development of UAV Forest Images transmission APP and pest detection system. Beijing : Beijing Forestry University. [in Chinese]
	聂丽郦. 2019. 基于卷积神经网络的榆紫叶甲虫识别研究. 哈尔滨 : 东北林业大学.
	Nie L Y. 2019. Identification and research of the Ambrostoma quadriimpressum motschlsky based on the Convolutional Neural Network. Harbin : Northeast Forestry University. [in Chinese]
	秦　放. 2018. 基于深度学习的昆虫图像识别研究. 成都: 西南交通大学.
	Qing F. 2018. Research on insect image recognition based on Deep Learning. Chengdu : Southwest Jiaotong University. [in Chinese]
	邵亚奎, 王　蕾, 朱长明, 等 GEE云平台支持下的西天山森林遥感监测与时空变化分析. 测绘通报, 2020, 2020 (8): 13- 17.
	Shao Y K, Wang L, Zhu C M, et al Forest survey and spatio-temporal analysis in West Tianshan mountains supported by Google Earth Engine. Bulletin of Surveying and Mapping, 2020, 2020 (8): 13- 17.
	史冰莹, 李佳琦, 张　磊, 等基于CNN的农作物病虫害图像识别模型. 计算机系统应用, 2020, 29 (6): 89- 96. doi: 10.15888/j.cnki.csa.007471
	Shi B Y, Li J Q, Zhang L, et al Recognition model of crop pests and diseases images based on CNN. Computer Systems & Applications, 2020, 29 (6): 89- 96. doi: 10.15888/j.cnki.csa.007471
	孙忠秋, 吴发云, 高显连, 等基于机载大光斑激光雷达的森林冠层高度估测. 林业资源管理, 2020, (3): 111- 117.
	Sun Z Q, Wu F Y, Gao X L, et al Estimation of forest canopy height based on large-footprint airborne LiDAR data. Forest Resources Management, 2020, (3): 111- 117.
	王　佳, 张隆裕, 吕春东, 等基于地面激光雷达点云数据的树种识别方法. 农业机械学报, 2018, 49 (11): 180- 188. doi: 10.6041/j.issn.1000-1298.2018.11.021
	Wang J, Zhang R Y, Lv C D, et al Tree species identification methods based on point cloud data using ground-based LiDAR. Transactions of the Chinese Society for Agricultural Machinery, 2018, 49 (11): 180- 188. doi: 10.6041/j.issn.1000-1298.2018.11.021
	王胜梅. 2018. 油松毛虫性诱技术优化及其幼虫空间格局研究. 北京 : 北京林业大学.
	Wang S M. 2018. Study on technology optimization of sex pheromone attractant and larvae spatial distribution of Dendrolimus tabulaeformis. Beijing : Beijing Forestry University. [in Chinese]
	谢　楠. 2017. 明山万亩生态林场林业有害生物调查及主要种类发生规律. 泰安: 山东农业大学.
	Xie N. 2017. Investigation of Forest Pests and occurrence rule of main kinds in MingShan ten-thousand mu ecological forestry farm. Tai'an: Shandong Agricultural University. [in Chinese]
	应新新. 2018. 基于支持向量机的果实蝇图像的自动识别系统研究与实现. 南昌: 江西农业大学.
	Ying X X. 2018. Research and realization of image automatic identification system for Bactrocera based on SVM. Nanchang: Jiangxi Agricultural University. [in Chinese]
	岳春宇, 郑永超, 邢艳秋, 等星载激光遥感林业应用发展研究. 红外与激光工程, 2020, 49 (11): 105- 114.
	Yue C Y, Zheng Y C, Xing Y Q, et al Technical and application development study of space-borne LiDAR in forestry remote sensing. Infrared and Laser Engineering, 2020, 49 (11): 105- 114.
	张超凡, 王儒敬, 谢成军. 2017. 基于多特征字典学习的害虫图像自动分类方法. 计算机应用与软件, 34 (3) : 142−147, 180.
	Zhang C F, Wang R J, Xie C J. 2017. Automatic classification method for Pest Image based on multi-feature dictionary learning. Computer Applications and Software, 34 (3) : 142−147, 180. [in Chinese]
	张广群, 李英杰, 汪杭军基于词袋模型的林业业务图像分类. 浙江农林大学学报, 2017, 34 (5): 791- 797. doi: 10.11833/j.issn.2095-0756.2017.05.004
	Zhang G Q, Li Y J, Wang H J Classification of forestry images based on the BoW Model. Journal of Zhejiang A & F University, 2017, 34 (5): 791- 797. doi: 10.11833/j.issn.2095-0756.2017.05.004
	赵慧颖. 2019. 基于贝叶斯卷积神经网络的害虫图像识别方法. 长春: 吉林大学.
	Zhao Y H. 2019. Pest image identification method based on Bayesian Convolutional Neural Network. Changchun: Jilin University. [in Chinese]
	赵仁辉, 刘　丽, 王志宝, 等基于开源GIS的林业有害生物监测与预警系统. 计算机工程与设计, 2015, 36 (3): 802- 808. doi: 10.16208/j.issn1000-7024.2015.03.048
	Zhao R H, Liu L, Wang Z B, et al Monitoring and warning system for forest pest based on open source GIS. Computer Engineering and Design, 2015, 36 (3): 802- 808. doi: 10.16208/j.issn1000-7024.2015.03.048
	Bahdanau D, Cho K, Bengio Y. 2014. Neural machine translation by jointly learning to align and translate. arXiv preprint arXiv : 1409.0473.
	He K, Zhang X, Ren S, et al. 2016. Deep residual learning for image recognition. Proceedings of the IEEE conference on computer vision and pattern recognition, 770−778.
	Ioffe S, Szegedy C Batch normalization: Accelerating deep network training by reducing internal covariate shift. International conference on machine learning. PMLR, 2015, 37, 448- 456.
	Jaderberg M, Simonyan K, Zisserman A Spatial transformer networks. Advances in neural information processing systems, 2015, 28, 2017- 2025.
	Krizhevsky A, Sutskever I, Hinton G E Imagenet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60 (6): 84- 90. doi: 10.1145/3065386
	Qiao X, Li Y Z, Su G Y, et al MmNet: Identifying Mikania micrantha kunth in the wild via a deep Convolutional Neural Network . Journal of Integrative Agriculture, 2020, 19 (5): 1292- 1300. doi: 10.1016/S2095-3119(19)62829-7
	Simonyan K, Zisserman A. 2014. Very deep convolutional networks for large-scale image recognition. arXiv preprint arXiv : 1409. 1556.
	Srivastava N, Hinton G, Krizhevsky A, et al Dropout: a simple way to prevent neural networks from overfitting. The Journal of Machine Learning Research, 2014, 15 (1): 1929- 1958.
	Szegedy C, Ioffe S, Vanhoucke V, et al Inception-v4, inception-resnet and the impact of residual connections on learning. Proceedings of the AAAI Conference on Artificial Intelligence, 2017, 31 (1): 4278- 4284.
	Szegedy C, Liu W, Jia Y, et al. 2015. Going deeper with convolutions. Proceedings of the IEEE conference on computer vision and pattern recognition, 1−9.
	Szegedy C, Vanhoucke V, Ioffe S, et al. 2016. Rethinking the inception architecture for computer vision. Proceedings of the IEEE conference on computer vision and pattern recognition, 2818−2826.
	Wang F, Jiang M, Qian C, et al. 2017. Residual attention network for image classification. Proceedings of the IEEE conference on computer vision and pattern recognition, 3156−3164.
	Xiao T, Xu Y, Yang K, et al. 2015. The application of two-level attention models in deep convolutional neural network for fine-grained image classification. Proceedings of the IEEE conference on computer vision and pattern recognition, 842−850.
	Xin D J, Chen Y W, Li J J Fine-grained butterfly classification in ecological images using squeeze-and-excitation and spatial attention modules. Applied Sciences, 2020, 10 (5): 1681. doi: 10.3390/app10051681
	Ye H J, Han H, Zhu L N, et al Vegetable Pest image recognition method based on improved VGG convolution neural network. Journal of Physics:Conference Series, 2019, 1237 (3): 032018. doi: 10.1088/1742-6596/1237/3/032018
	Zhao B, Wu X, Feng J, et al Diversified visual attention networks for fine-grained object classification. IEEE Transactions on Multimedia, 2017, 19 (6): 1245- 1256. doi: 10.1109/TMM.2017.2648498
	Zheng H, Fu J, Zha Z J, et al Learning deep bilinear transformation for fine-grained image representation. Advances in Neural Information Processing Systems, 2019, 32, 4277- 4286.

模型 Models	平均准确率 Average Precision(%)	平均召回率 Average Recall	平均F₁值 Average F₁-measure
ResNet-50	74.9	72.7	71.5
ResNet-101	76.3	74.9	73.9
改进残差模块的ResNet-50	79.7	77.3	76.8
改进残差模块的ResNet-101	81.1	79.0	78.7

结果Results	VGGNet-19	改进残差模块的ResNet-101	改进残差模块的DBTNet-101	改进残差模块的DBTANet-101
74种平均准确率Average precision-74	78.6	81.1	88.8	91.3
22种平均准确率Average precision-22	67.5	69.4	81.1	85.1
74种平均召回率Average recall-74	74.1	79.0	89.5	91.1
22种平均召回率Average recall-22	56.4	59.3	77.6	82.5
74种平均F₁值Average F₁-74	74.5	78.7	88.5	90.8
22种平均F₁值Average F₁-22	59.6	62.8	78.0	83.0

[1]	Gui Meng,Yefei Liu,Xufeng Zhang,Chengle Zhao,Shuirong Wu. Sequential Variation Analysis of Forest Pest Disasters in China from 1998 to 2018 [J]. Scientia Silvae Sinicae, 2022, 58(7): 134-143.
[2]	Zhao Tieliang;Dong Zhenhui;Yu Zhijun;Zhao Qingshan;Jiang Fengyan. STUDY ON ESTABLISHMENT OF FOREST PEST INDEXES [J]. Scientia Silvae Sinicae, 2003, 39(3): 172-176.

Forest Pest Identification Method Based on a Deep Bilinear Transformation Attention Mechanism Network

RichHTML

PDF (PC)

Knowledge

Abstract

Cite this article

share this article

Figures/Tables 9

References 0

Related Articles 2

Recommended Articles

Metrics

Comments