基于卷积神经网络的近红外光谱与数字图像特征信息融合木材树种识别

doi:10.11707/j.1001-7488.LYKX20230357

摘要/Abstract

摘要：

目的: 基于卷积神经网络自动提取，探究融合木材近红外光谱与数字图像特征信息准确识别木材树种的可行性。方法: 以樟科10种木材标本为例，使用手持式近红外光谱仪和便携式扫描仪采集木材标本横切面近红外光谱和图像。创新引入递归图方法，将手持式近红外光谱仪采集的一维短波长近红外光谱转换为二维图像，促进卷积神经网络从近红外光谱数据中提取判别性更强的特征，实现近红外光谱与图像在二维尺度上的融合。构建结构简单的双分支卷积神经网络模型，自动提取、融合近红外光谱与图像特征识别木材树种。结果: 与直接使用一维近红外光谱的建模方法相比，近红外光谱递归图结合卷积神经网络模型的识别性能提升1.79%~14%；与使用近红外光谱或图像单一特征识别相比，双分支卷积神经网络模型自动提取、融合近红外光谱与图像特征，对10种木材的识别性能至少提高3%，模型准确率、精度和召回率均大于99%。结论: 一维短波长近红外光谱递归图转换能够促进卷积神经网络从近红外光谱数据中提取判别性更强的特征，提高模型识别性能。双分支卷积神经网络能够充分提取并有效融合木材近红外光谱与图像特征，一定程度上可克服使用单一特征识别木材树种的不足，提高木材树种识别效果。

关键词: 木材树种识别, 卷积神经网络, 近红外光谱, 图像, 特征提取与融合

Abstract:

Objective: The feasibility of accurately identifying wood species by integrating near infrared spectroscopy (NIR) and digital image feature information, automatically extracted based on convolutional neural networks (CNN), is investigated. Method: 10 wood specimens from the Lauraceae family were used as examples. NIR spectra and digital images from the transverse surfaces of specimens were collected using a handheld NIR spectrometer and portable scanner. The recurrence plot (RP) method was innovatively employed to encode one-dimensional (1D) NIR spectra as two-dimensional (2D) images. This not only enabled the CNN model to extract more robust discriminative features from the handheld NIR spectrometer’s shorter-wavelength NIR spectra, but also facilitated the fusion of NIR spectra and images. A simple two branch CNN (TB-CNN) model was developed to automatically extract and fuse NIR spectral and digital image features for wood species identification. Result: Compared to different modeling methods that utilize 1D NIR spectra directly, integrating 2D RP of NIR spectra with CNN resulted in a performance improvement ranging from 1.79% to 14%. Furthermore, when compared with the results obtained using a single feature from either NIR spectra or digital image, the TB-CNN method showed a significant increase in identification rates of at least 3%. Notably, the accuracy, precision and recall values all surpassed 99%. Conclusion: The conversion of 1D short-wavelength NIR spectra into RP enhances the ability of CNN model to extract more discriminative features from the NIR spectral data, improving model performance in species identification. The TB-CNN model effectively extracts and integrates wood NIR spectra and digital image features, addressing the limitations of using a single feature type for wood species identification and improving overall identification accuracy.

Key words: wood species identification, convolutional neural network (CNN), near-infrared spectroscopy, image, feature extraction and fusion

中图分类号:

S781.1

潘玺,李康,杨忠. 基于卷积神经网络的近红外光谱与数字图像特征信息融合木材树种识别[J]. 林业科学, 2024, 60(12): 136-145.

Xi Pan,Kang Li,Zhong Yang. Wood Species Identification through Fusion of NIR Spectroscopy and Digital Image Features Information Using Convolutional Neural Networks[J]. Scientia Silvae Sinicae, 2024, 60(12): 136-145.

图/表 11

表1

图1

图2

图3

表2

表3

表4

表5

表6

图4

图5

参考文献 0

	戴天虹, 翟　冰. 基于改进EfficientNet的木材识别研究. 森林工程, 2023, 39 (4): 93- 100.
	Dai T H, Zhai B. Wood recognition research based on improved EfficientNet. Forest Engineering, 2023, 39 (4): 93- 100.
	付立岩, 冯国红, 刘旭铭. 基于多元经验模态分解的可见/近红外光谱识别木材研究. 森林工程, 2023, 39 (4): 101- 109.
	Fu L Y, Feng G H, Liu X M. Wood recognition by visible/near infrared spectroscopy based on multivariate empirical mode decomposition. Forest Engineering, 2023, 39 (4): 101- 109.
	高振忠, 孙　瑾, 吴　鸿, 等. 樟科10属14种木材解剖学特征的比较研究. 林业科学研究, 2009, 22 (3): 413- 417. doi: 10.3321/j.issn:1001-1498.2009.03.018
	Gao Z Z, Sun J, Wu H, et al. Comparative study on wood anatomical structure of 14 species belonging to 10 genera of Lauraceae. Forest Research, 2009, 22 (3): 413- 417. doi: 10.3321/j.issn:1001-1498.2009.03.018
	何　拓, 刘守佳, 陆　杨, 等. 木材树种计算机视觉识别技术发展与应用. 林业工程学报, 2021a, 6 (3): 18- 27.
	He T, Liu S J, Lu Y, et al. Advances and prospects of wood identification technology coupled with computer vision. Journal of Forestry Engineering, 2021a, 6 (3): 18- 27.
	何　拓, 刘守佳, 陆　杨, 等. iWood: 基于卷积神经网络的濒危珍贵树种木材自动识别系统. 林业科学, 2021b, 57 (9): 152- 159.
	He T, Liu S J, Lu Y, et al. iWood: an automated wood identification system for endangered and precious tree species using convolutional neural networks. Scientia Silvae Sinicae, 2021b, 57 (9): 152- 159.
	刘守佳, 汪嘉君, 陆　杨, 等. 机器学习方法在木材树种识别领域的应用. 世界林业研究, 2023, 36 (5): 70- 75.
	Liu S J, Wang J J, Lu Y, et al. Application of machine learning approaches to wood species identification. World Forestry Research, 2023, 36 (5): 70- 75.
	宿恒硕, 吕　军, 丁志平, 等. 基于改进残差神经网络的木材识别算法. 林业科学, 2021, 57 (12): 147- 154.
	Su H S, Lü J, Ding Z P, et al. Wood identification algorithm based on improved residual neural network. Scientia Silvae Sinicae, 2021, 57 (12): 147- 154.
	孙　瑾, 王晓静, 王　飞, 等. 滇西南地区樟科17种木材解剖特征研究. 华南农业大学学报, 2014, 35 (5): 102- 107. doi: 10.7671/j.issn.1001-411X.2014.05.018
	Sun J, Wang X J, Wang F, et al. Wood anatomical characteristics of 17 Lauraceae species from southwest of Yunnan. Journal of South China Agricultural University, 2014, 35 (5): 102- 107. doi: 10.7671/j.issn.1001-411X.2014.05.018
	王登举. 全球林产品贸易现状与特点. 国际木业, 2019, 49 (3): 49- 53. doi: 10.3969/j.issn.1671-4911.2019.03.020
	Wang D J. Status quo and characteristics of global trade of forest products. International Wood Industry, 2019, 49 (3): 49- 53. doi: 10.3969/j.issn.1671-4911.2019.03.020
	赵子宇, 杨霄霞, 郭　慧, 等. 基于卷积神经网络模型的木材宏、微观辨识方法. 林业科学, 2021, 57 (6): 134- 143. doi: 10.11707/j.1001-7488.20210615
	Zhao Z Y, Yang X X, Guo H, et al. Recognition method of wood macro-and micro-structure based on convolution neural network. Scientia Silvae Sinicae, 2021, 57 (6): 134- 143. doi: 10.11707/j.1001-7488.20210615
	Bai R X, Meng Z, Xu Q S, et al. Fractional Fourier and time domain recurrence plot fusion combining convolutional neural network for bearing fault diagnosis under variable working conditions. Reliability Engineering & System Safety, 2023, 232, 109076.
	Barmpoutis P, Dimitropoulos K, Barboutis I, et al. Wood species recognition through multidimensional texture analysis. Computers and Electronics in Agriculture, 2018, 144, 241- 248. doi: 10.1016/j.compag.2017.12.011
	Carmona R J, Wiemann M C, Baas P, et al. Forensic identification of CITES Appendix I. Cupressaceae using anatomy and mass spectrometry. IAWA Journal, 2020, 41 (4): 720- 739. doi: 10.1163/22941932-bja10002
	Chollet F. 2017. Xception: deep learning with depthwise separable convolutions. 2017 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Honolulu: HI,USA,1800−1807.
	Eckmann J P, Kamphorst S O, Ruelle D. Recurrence plots of dynamical systems. Europhysics Letters (EPL), 1987, 4 (9): 973- 977. doi: 10.1209/0295-5075/4/9/004
	Fathurahman T, Gunawan P H, Prakasa E, et al. Wood classification of Japanese Fagaceae using partial sample area and convolutional neural networks. Journal of the Korean Wood Science and Technology, 2021, 49 (5): 491- 503. doi: 10.5658/WOOD.2021.49.5.491
	Han J C, Zhao P, Wang C K. Wood species recognition through FGLAM textural and spectral feature fusion. Wood Science and Technology, 2021, 55 (2): 535- 552. doi: 10.1007/s00226-021-01262-0
	He K M, Zhang X Y, Ren S Q, et al. 2016. Deep residual learning for image recognition. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 770−778.
	He T, Lu Y, Jiao L C, et al. Developing deep learning models to Automate rosewood tree species identification for CITES designation and implementation. Holzforschung, 2020, 74 (12): 1123- 1133. doi: 10.1515/hf-2020-0006
	He T, Mu S B, Zhou H K, et al. Wood species identification based on an ensemble of deep convolution neural networks. Wood Research, 2021, 66 (1): 1- 14.
	Hinton G, Osindero S, Welling M, et al. Unsupervised discovery of nonlinear structure using contrastive backpropagation. Cognitive Science, 2006, 30 (4): 725- 731. doi: 10.1207/s15516709cog0000_76
	Hwang S W, Kobayashi K, Zhai S C, et al. Automated identification of Lauraceae by scale-invariant feature transform. Journal of Wood Science, 2018, 64 (2): 69- 77. doi: 10.1007/s10086-017-1680-x
	Hwang S W, Sugiyama J. Computer vision-based wood identification and its expansion and contribution potentials in wood science: a review. Plant Methods, 2021, 17 (1): 47. doi: 10.1186/s13007-021-00746-1
	Jiao L C, Yu M, Wiedenhoeft A C, et al. DNA barcode authentication and library development for the wood of six commercial Pterocarpus species: the critical role of xylarium specimens. Scientific Reports, 2018, 8 (1): 1945. doi: 10.1038/s41598-018-20381-6
	Krizhevsky A, Sutskever I, Hinton G E. ImageNet classification with deep convolutional neural networks. Communications of the ACM, 2017, 60 (6): 84- 90. doi: 10.1145/3065386
	Lens F, Liang C, Guo Y H, et al. Computer-assisted timber identification based on features extracted from microscopic wood sections. IAWA Journal, 2020, 41 (4): 660- 680. doi: 10.1163/22941932-bja10029
	Liu Z F, Ci X Q, Li L, et al. DNA barcoding evaluation and implications for phylogenetic relationships in Lauraceae from China. PLoS ONE, 2017, 12 (4): e0175788. doi: 10.1371/journal.pone.0175788
	Liu Z F, Ma H, Zhang X Y, et al. 2022. Do taxon-specific DNA barcodes improve species discrimination relative to universal barcodes in Lauraceae? Botanical Journal of the Linnean Society, 199(4): 741−753.
	Pan X, Li K, Chen Z J, et al. Identifying wood based on near-infrared spectra and four gray-level co-occurrence matrix texture features. Forests, 2021, 12 (11): 1527. doi: 10.3390/f12111527
	Rocha H S, Braga J W B, Kunze D C G C, et al. Identification of mahogany sliced veneer using handheld near-infrared spectroscopy device and multivariate data analysis. IAWA Journal, 2021, 42 (3): 336- 347. doi: 10.1163/22941932-bja10054
	Simonyan K, Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition. 3rd International Conference on Learning Representations (ICLR 2015), arXiv: 1409.1556.
	Sohi A, Avramidis S, Mansfield S. Near-infrared spectroscopic separation of green chain sub-alpine fir lumber from a spruce-pine-fir mix. BioResources, 2017, 12 (2): 3720- 3727.
	Sun J, Wu J Y, Wang X J, et al. Comparative wood anatomy of 56 species of Lauraceae from Yunnan, China. Brazilian Journal of Botany, 2015, 38 (3): 645- 656. doi: 10.1007/s40415-015-0172-8
	Szegedy C, Vanhoucke V, Ioffe S, et al. 2016. Rethinking the Inception architecture for computer vision. 2016 IEEE Conference on Computer Vision and Pattern Recognition (CVPR), Las Vegas, NV, USA, 2818−2826.
	Wang C K, Zhao P, Li Z Y, et al. Comparison of VIS/NIR spectral curves plus RGB images with hyperspectral images for the identification of Pterocarpus species. Holzforschung, 2022, 76 (7): 579- 591. doi: 10.1515/hf-2021-0194
	Wang C K, Zhao P. Classification of wood species using spectral and texture features of transverse section. European Journal of Wood and Wood Products, 2021, 79 (5): 1283- 1296. doi: 10.1007/s00107-021-01728-9
	Yang S Y, Kwon O, Park Y, et al. Application of neural networks for classifying softwood species using near infrared spectroscopy. Journal of Near Infrared Spectroscopy, 2020, 28 (5/6): 298- 307.

标签 Label	属 Genus	木材树种 Wood species
0	琼楠属 Beilschmiedia	李榄琼楠 B. linocieroides
1	琼楠属 Beilschmiedia	滇琼楠 B. yunnanensis
2	樟属 Cinnamomum	云南樟 C. glanduliferum
3		长柄樟 C. longipetiolatum
4		黄樟 C. porrectum
5		细毛樟 C. tenuipilum
6	润楠属 Machilus	绿叶润楠 M. viridis
7	润楠属 Machilus	滇润楠 M. yunnanensis
8	楠属 Phoebe	披针叶楠 P. lanceolata
9	楠属 Phoebe	普文楠 P. puwenensis

模型 Model	数据类型 Data type	准确率 Accuracy	精度 Precision	召回率 Recall
NIR-B-CNN	NIR-RP	95.54	95.82	95.51
NIR-B-CNN-1D	1D-NIR	89.46	87.52	87.83
CNN-1D (Yang et al., 2020)		81.54	82.56	81.52
PLS-DA		93.13	93.29	93.05
SVM		93.75	94.09	93.75
RF		83.54	85.47	83.54

预处理 Pre-processing	NIR-B-CNN-1D			NIR-B-CNN
预处理 Pre-processing	准确率 Accuracy	精度 Precision	召回率 Recall	准确率 Accuracy	精度 Precision	召回率 Recall
原始近红外光谱 Raw near infrared spectroscopy	89.46	87.52	87.83	95.54	95.82	95.51
去趋势Detrend	93.88	94.13	93.87	96.00	96.12	95.99
一阶导数First derivative (1^st-D)	95.13	94.82	95.15	96.58	96.75	96.58
均值中心化Mean center (MC)	92.21	92.95	92.07	94.50	95.08	94.43
标准变量变换 Standard normal variate (SNV)	90.38	91.05	90.37	95.88	96.12	95.86
多元散射校正 Multiplicative scatter correction (MSC)	91.08	91.77	91.13	95.33	95.66	95.28

模型 Model	图像尺寸 Image size	准确率 Accuracy	精度 Precision	召回率 Recall
Image-B-CNN	125 pix×125 pix	95.67	96.09	95.69
VGG19		93.79	94.02	93.73
ResNe-t50		96.42	96.66	96.32
Inception V3		95.04	95.34	94.91
Xception		95.96	96.18	95.89
Image-B-CNN	250 pix×250 pix	96.75	96.87	96.75
VGG19		95.04	95.29	94.95
ResNet-50		96.54	96.84	96.50
Inception V3		96.50	96.70	96.46
Xception		96.46	96.71	96.42

模型 Model	输入数据（尺寸） Input date (size)	准确率 Accuracy	精度 Precision	召回率 Recall
NIR-B-CNN	NIR RP	95.54	95.82	95.51
Image-B-CNN	Image (125 pix×125 pix)	95.67	96.09	95.69
Image-B-CNN	Image (250 pix×250 pix)	96.75	96.87	96.75
TB-CNN	NIR RP + Image (125 pix×125 pix)	99.92	99.92	99.92
TB-CNN	NIR RP + Image (250 pix×250 pix)	99.96	99.96	99.96