基于GAN-DCNN的树叶识别

doi:10.11707/j.1001-7488.LYKX20220847

摘要/Abstract

摘要：

目的: 利用深度学习进行树叶识别时需要大量训练样本，当样本量不足、图像风格单一会导致识别准确率不稳定。研究利用少量的样本进行树叶图像增殖和风格转换，可极大减轻数据采集的负担，为提升林业调查信息化、智能化提供有效的技术手段和理论支撑。方法: 采集6种树种的树叶图像建立数据集，引入light-weight GAN对图像进行增殖和风格转换，扩充人工拍摄的树叶数据集，通过在该数据集与原数据集上分别应用AlexNet、GoogLeNet、ResNet34和ShuffleNetV2四种深度卷积神经网络进行训练，分析生成对抗网络的图像增殖技术在树叶识别中的作用。综合模型准确率和训练时间等性能指标选择最优模型，同时对模型的学习率进行调整。使用测试样本对参数优化后的模型进行验证，分析该方法在实践中的可行性和意义。结果: 基于生成对抗网络生成的样本具有高清晰度，高保真性，能够有效地辅助神经网络模型的训练工作，同时也丰富了样本类别，使之获得包含更多不同季节、形状、健康状况的树叶图像。与原始数据集相比，AlexNet、GoogLeNet、ResNet34和ShuffleNetV2四种网络在新数据集的训练上均表现出训练误差更小、验证精度更高的特点，其中学习率为0.01的ShuffleNetV2模型对该数据集的训练效果最好，训练时最高验证精度为99.7%。使用未参与训练的测试样本对该模型进行验证，模型对各树叶的识别效果较好，模型的总体识别准确率高达99.8%。与未使用GAN技术的普通深度卷积神经网络相比，本文提出的模型对树叶识别准确率明显提升。结论: 生成对抗网络可以有效地扩充图像数量，对图像进行风格转换，与深度卷积神经网络相结合，可以显著提高树叶识别准确率，适合应用于林业树叶识别领域。

关键词: 树叶识别, 生成对抗网络, 深度卷积神经网络

Abstract:

Objective: A large number of training samples are needed for leaf identification based on deep learning. Insufficient sample size and single image style can affect the identification accuracy. However, studying the use of a small number of samples for leaf image reproduction and style transformation can greatly reduce the burden of data collection, which would provide effective technical means and theoretical support for improving forestry survey informationize and intelligence. Method: The leaf images of 6 tree species were collected to establish a dataset, and the light-weight generating adversarial networks (GAN) was introduced to propagate images and style translation, and expand the manually shot leaf dataset. Four deep convolutional neural networks (DCNN), AlexNet, GoogLeNet, ResNet34 and ShuffleNetV2, were applied to train the dataset and the original dataset, respectively, by which the role of image augmentation techniques of GAN in leaf recognition was analyzed. The optimal model was selected based on performance indicators such as model accuracy and training time, and the learning rate was adjusted. Finally, the test samples were used to verify the optimized model, and the feasibility and significance of the method in practice were analyzed. Result: The samples with high definition and high fidelity were generated based on generative adversarial networks, which was able to effectively enrich the sample category, and obtain leaf images with different shapes, and health conditions at different seasons. Compared with the original dataset, AlexNet, GoogLeNet, ResNet34 and ShuffleNetV2 all showed smaller training errors and higher validating accuracy on new dataset during the model training. Among them, the ShuffleNetV2 model with a learning rate of 0.01 had the best training effect on this dataset, whose highest validating accuracy was 99.7%. The model was verified by using the test samples and a good recognition performance on each leaf was achieved, and the overall recognition accuracy of the model was up to 99.8%. Compared with the ordinary DCNN without GAN, the model proposed in this paper significantly improved the accuracy of leaf identification. Conclusion: GAN can effectively expand the number of images and transform the style of images. The combined use of GAN and DCNN can significantly improve the accuracy of leaf identification, thus it can be applied in forest leaf identification.

Key words: leaf identification, generative adversarial networks, deep convolutional neural networks

中图分类号:

TP183

徐竞怡,张志,闫飞,张雯悦. 基于GAN-DCNN的树叶识别[J]. 林业科学, 2024, 60(4): 40-51.

Jingyi Xu,Zhi Zhang,Fei Yan,Wenyue Zhang. Leaf Identification Based on GAN-DCNN[J]. Scientia Silvae Sinicae, 2024, 60(4): 40-51.

图/表 12

图1

表1

图2

图3

表2

图4

表3

图5

表4

表5

表6

表7

参考文献 0

	曹仰杰, 贾丽丽, 陈永霞, 等. 生成式对抗网络及其计算机视觉应用研究综述. 中国图象图形学报, 2018, 23 (10): 1433- 1449. doi: 10.11834/jig.180103
	Cao Y J, Jia L L, Chen Y X, et al. Review of computer vision based on generative adversarial networks. Journal of Image and Graphics, 2018, 23 (10): 1433- 1449. doi: 10.11834/jig.180103
	冯海林, 胡明越, 杨垠晖, 等. 基于树木整体图像和集成迁移学习的树种识别. 农业机械学报, 2019, 50 (8): 235- 242，279. doi: 10.6041/j.issn.1000-1298.2019.08.025
	Feng H L, Hu M Y, Yang Y H, et al. Tree species recognition based on overall tree image and ensemble of transfer learning. Transactions of the Chinese Society for Agricultural Machinery, 2019, 50 (8): 235- 242，279. doi: 10.6041/j.issn.1000-1298.2019.08.025
	高　旋, 赵亚凤, 熊　强, 等. 基于迁移学习的树种识别. 森林工程, 2019, 35 (5): 68- 75. doi: 10.3969/j.issn.1006-8023.2019.05.012
	Gao X, Zhao Y F, Xiong Q, et al. Identification of tree species based on transfer learning. Forest Engineering, 2019, 35 (5): 68- 75. doi: 10.3969/j.issn.1006-8023.2019.05.012
	李　滨, 敬启超. 改进的卷积神经网络在树种识别中的应用. 森林工程, 2021, 37 (5): 75- 81, 104. doi: 10.3969/j.issn.1006-8023.2021.05.010
	Li B, Jing Q C. Improved convolutional neural network for tree species recognition. Forest Engineering, 2021, 37 (5): 75- 81, 104. doi: 10.3969/j.issn.1006-8023.2021.05.010
	刘嘉政, 王雪峰, 王　甜. 基于多特征融合和CNN模型的树种图像识别研究. 北京林业大学学报, 2019, 41 (11): 76- 86.
	Liu J Z, Wang X F, Wang T. Image recognition of tree species based on multi feature fusion and CNN model. Journal of Beijing Forestry University, 2019, 41 (11): 76- 86.
	龙满生, 欧阳春娟, 刘　欢, 等. 基于卷积神经网络与迁移学习的油茶病害图像识别. 农业工程学报, 2018, 34 (18): 194- 201. doi: 10.11975/j.issn.1002-6819.2018.18.024
	Long M S, Ouyang C J, Liu H, et al. Image recognition of Camellia oleifera diseases based on convolutional neural network & transfer learning. Transactions of the Chinese Society of Agricultural Engineering, 2018, 34 (18): 194- 201. doi: 10.11975/j.issn.1002-6819.2018.18.024
	卢宏涛, 张秦川. 深度卷积神经网络在计算机视觉中的应用研究综述. 数据采集与处理, 2016, 31 (1): 1- 17.
	Lu H T, Zhang Q C. Applications of deep convolutional neural network in computer vision. Journal of Data Acquisition and Processing, 2016, 31 (1): 1- 17.
	吕恩辉. 2019. 基于卷积神经网络的图像分类研究. 北京: 中国矿业大学.
	Lü E H. 2019. Image classification based on convolutional neural network. Beijing: China University of Mining and Technology. ［in Chinese］
	汪美琴, 袁伟伟, 张继业. 生成对抗网络GAN的研究综述. 计算机工程与设计, 2021, 42 (12): 3389- 3395.
	Wang M Q, Yuan W W, Zhang J Y. Overview of research on generative adversarial network GAN. Computer Engineering and Design, 2021, 42 (12): 3389- 3395.
	王恩泽, 赵亚凤. 基于多任务持续学习的树种识别. 森林工程, 2022, 38 (1): 67- 75. doi: 10.3969/j.issn.1006-8023.2022.01.009
	Wang E Z, Zhao Y F. Identification of tree species based on multitask continual learning. Forest Engineering, 2022, 38 (1): 67- 75. doi: 10.3969/j.issn.1006-8023.2022.01.009
	谢家兴, 陈斌瀚, 彭家骏, 等. 基于改进ShuffleNet V2的荔枝叶片病虫害图像识别. 果树学报, 2023, 40 (5): 1024- 1035.
	Xie J X, Chen B H, Peng J J, et al. Image recognition of litchi leaf pests and diseases using improved ShuffleNetV2. Journal of Fruit Science, 2023, 40 (5): 1024- 1035.
	徐志扬, 陈　巧, 陈永富. LiDAR单木分割辅助的无人机影像CNN+EL树种识别. 农业机械学报, 2022, 53 (3): 197- 205.
	Xu Z Y, Chen Q, Chen Y F. Tree species recognition based on unmanned aerial vehicle image with LiDAR individual tree segmentation aided. Transactions of the Chinese Society for Agricultural Machinery, 2022, 53 (3): 197- 205.
	薛　勇, 王立扬, 张　瑜, 等. 基于GoogLeNet深度迁移学习的苹果缺陷检测方法. 农业机械学报, 2020, 51 (7): 30- 35.
	Xue Y, Wang L Y, Zhang Y, et al. Defect detection method of apples based on GoogLeNet deep transfer learning. Transactions of the Chinese Society for Agricultural Machinery, 2020, 51 (7): 30- 35.
	张锡英, 宋宇鹏, 陈　曦, 等. 融合STN和DenseNet的深度学习网络及其应用. 计算机工程与应用, 2019, 55 (17): 131- 136. doi: 10.3778/j.issn.1002-8331.1901-0094
	Zhang X Y, Song Y P, Chen X, et al. Deep learning network and application based on fused STN and DenseNet. Computer Engineering and Applications, 2019, 55 (17): 131- 136. doi: 10.3778/j.issn.1002-8331.1901-0094
	张　旭, 周云成, 刘忠颖, 等. 基于改进ShuffleNet V2模型的苹果叶部病害识别及应用. 沈阳农业大学学报, 2022, 53 (1): 110- 118. doi: 10.3969/j.issn.1000-1700.2022.01.014
	Zhang X, Zhou Y C, Liu Z Y, et al. Identification and application of apple leaf diseases based on improved ShuffleNet V2 model. Journal of Shenyang Agricultural University, 2022, 53 (1): 110- 118. doi: 10.3969/j.issn.1000-1700.2022.01.014
	赵立新, 侯发东, 吕正超, 等. 基于迁移学习的棉花叶部病虫害图像识别. 农业工程学报, 2020, 36 (7): 184- 191. doi: 10.11975/j.issn.1002-6819.2020.07.021
	Zhao L X, Hou F D, Lv Z C, et al. Image recognition of cotton leaf diseases and pests based on transfer learning. Transactions of the Chinese Society of Agricultural Engineering, 2020, 36 (7): 184- 191. doi: 10.11975/j.issn.1002-6819.2020.07.021
	赵世达, 王树才, 白　宇, 等. 基于生成对抗网络与ICNet的羊骨架图像实时语义分割. 农业机械学报, 2021, 52 (2): 329- 339，380. doi: 10.6041/j.issn.1000-1298.2021.02.032
	Zhao S D, Wang S C, Bai Y, et al. Real-time semantic segmentation of sheep skeleton image based on generative adversarial network and ICNet. Transactions of the Chinese Society for Agricultural Machinery, 2021, 52 (2): 329- 339，380. doi: 10.6041/j.issn.1000-1298.2021.02.032
	赵增顺, 高寒旭, 孙　骞, 等. 生成对抗网络理论框架、衍生模型与应用最新进展. 小型微型计算机系统, 2018, 39 (12): 2602- 2606. doi: 10.3969/j.issn.1000-1220.2018.12.008
	Zhao Z S, Gao H X, Sun Q, et al. Latest development of the theory framework, derivative model and application of generative adversarial nets. Journal of Chinese Computer Systems, 2018, 39 (12): 2602- 2606. doi: 10.3969/j.issn.1000-1220.2018.12.008
	郑一力, 张　露. 基于迁移学习的卷积神经网络植物叶片图像识别方法. 农业机械学报, 2018, 49 (S1): 354- 359. doi: 10.6041/j.issn.1000-1298.2018.S0.047
	Zheng Y L, Zhang L. Plant leaf image recognition method based on transfer learning with convolutional neural networks. Transactions of the Chinese Society for Agricultural Machinery, 2018, 49 (S1): 354- 359. doi: 10.6041/j.issn.1000-1298.2018.S0.047
	Brock A, Donahue J, Simonyan K. 2018. Large Scale GAN Training for High Fidelity Natural Image Synthesis. arXiv: 1809.11096.
	Goodfellow I, Pouget-Abadie J, Mirza M, et al. Generative adversarial networks. Communications of the ACM, 2020, 63 (11): 139- 144.
	He K M, Zhang X Y, Ren S Q, et al. 2016. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 770−778.
	Karras T, Aittala M, Hellsten J, et al. 2020a. Training generative adversarial networks with limited data. arXiv: 2006.06676.
	Karras T, Aittala M, Laine S, et al. 2021. Alias-free generative adversarial networks. arXiv: 2106.12423.
	Karras T, Laine S, Aila T. 2019. A style-based generator architecture for generative adversarial networks. arXiv: 1812.04948.
	Karras T, Laine S, Aittala M, et al. 2020b. Analyzing and improving the image quality of styleGAN. arXiv: 1912.04958.
	Kingma D P, Welling M. 2013. Auto-encoding variational bayes. arXiv: 1312.6114.
	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60 (6): 84- 90.
	Liu B C, Song K P, Zhu Y Z, et al. 2020a. Time: text and image mutual-translation adversarial networks. arXiv: 2005.13192.
	Liu B, Tan C, Li S Q, et al. A data augmentation method based on generative adversarial networks for grape leaf disease identification. IEEE Access, 2020b, 8, 102188- 102198.
	Liu B C, Zhu Y Z, Song K P, et al. 2021. Towards faster and stabilized GAN training for high-fidelity few-shot image synthesis. arXiv: 2101.04775.
	Liu J Z, Wang X F, Wang T. Classification of tree species and stock volume estimation in ground forest images using deep learning. Computers and Electronics in Agriculture, 2019, 166, 105012.
	Ma N, Zhang X, Zheng H T, et al. 2018. ShuffleNet V2: practical guidelines for efficient CNN architecture design. arXiv: 1807.11164.
	Simonyan K, Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition. arXiv: 1409.1556.
	Szegedy C, Liu W, Jia Y Q, et al. 2014. Going deeper with convolutions. arXiv: 1409.4842.
	Tran N T, Tran V H, Nguyen N B, et al. 2021. On data augmentation for GAN training. IEEE Transactions on Image Processing. 30, 1882–1897.
	Xiong Y H, Liang L F, Wang L, et al. 2020. Identification of cash crop diseases using automatic image segmentation algorithm and deep learning with expanded dataset. Computers and Electronics in Agriculture. 177, 105712.
	Yang H W, Hsu H C, Yang C K, et al. Differentiating between morphologically similar species in genus Cinnamomum (Lauraceae) using deep convolutional neural networks. Computers and Electronics in Agriculture, 2019, 162, 739- 748.

层名称 Layer name	输出尺寸 Output size	卷积核大小 Kernel size	步长 Stride	重复次数/次 Repeat times
输入图像 Image	224×224×3
卷积层1 Conv1	112×112×24	3×3	2	1
最大池化层MaxPool	56×56×24	3×3	2	1
残差单元堆叠层2 Stage2	28×28×116		2	1
残差单元堆叠层2 Stage2	28×28×116		1	3
残差单元堆叠层3 Stage3	14×14×232		2	1
残差单元堆叠层3 Stage3	14×14×232		1	7
残差单元堆叠层4 Stage4	7×7×464		2	1
残差单元堆叠层4 Stage4	7×7×464		1	3
卷积层5 Conv5	7×7×1024	1×1	1	1
全局池化层GlobalPool	1×1	7×7
全连接层 FC	channels=6
归一化指数函数 Softmax

树种 Tree species	原始图像 Original images	生成图像 Generated images
菜豆树 Radermachera sinica
鹅掌楸 Liriodendron chinense
灰莉 Fagraea ceilanica
肉桂 Cinnamomum cassia
五角槭 Acer pictum subsp. mono
鸭脚木 Heptapleurum heptaphyllum

	AlexNet	GoogLeNet	ResNet34	ShufffeNetV2
训练时长 Training time	32 min 4 s	93 min 59 s	133 min 37 s	43 min 27 s
训练最小损失 Minimum training loss	0.119	0.084	0.046	0.055
验证最高精度 Maximum validating accuracy	99%	99%	99.3%	99.4%
收敛速度 Rate of convergence	较快 Fast	快 Faster	较快 Fast	快 Faster
收敛后稳定性 Convergence stability	波动 Fluctuant	小波动 Small fluctuant	稳定 Stable	非常稳定 Very stable

学习率 Learning rate	0.001	0.01	0.1
训练时长 Training time	43 min 27 s	45 min	49 min 8 s
训练最小损失 Minimum training loss	0.055	0.02	0.02
验证最高精度 Maximum validating accuracy	99.4%	99.7%	99.5%
收敛速度 Rate of convergence	较快 Fast	最快 Fastest	快 Faster
收敛后稳定性 Convergence stability	稳定 More stable	最稳定 Most stable	较稳定 Stable

树种 Tree species	五角槭 Acer pictum subsp. mono	肉桂 Cinnamomum cassia	灰莉 Fagraea ceilanica	鹅掌楸 Liriodendron chinense	菜豆树 Radermachera sinica	鸭脚木 Heptapleurum heptaphyllum	总和Total
五角槭 Acer pictum subsp. mono	111	0	0	0	0	0	111
肉桂 Cinnamomum cassia	0	169	0	0	0	0	169
灰莉 Fagraea ceilanica	0	0	167	0	0	0	167
鹅掌楸 Liriodendron chinense	1	0	0	139	0	0	140
菜豆树 Radermachera sinica	0	0	0	0	152	0	152
鸭脚木 Heptapleurum heptaphyllum	0	0	1	0	0	137	138
总和Total	112	169	168	139	152	137	877
准确率Accuracy rate	99.1%	100%	99.4%	100%	100%	100%	100%
总体准确率 Total accuracy rate	99.8%