基于Wasserstein距离和相关对齐迁移学习的野生动物图像识别方法

doi:10.11707/j.1001-7488.LYKX20230399

摘要/Abstract

摘要：

目的: 为解决光照、背景和拍摄尺度等复杂因素对野生动物图像识别准确性的影响。方法: 以野外红外触发相机采集的野生动物图像为对象：1）利用ENA24和NACTI两个公开的野生动物数据集构建包含不相交领域数据集S1和S2，共11个动物类别，包含25 591幅图像；2）针对领域偏移问题，采用ResNet50网络作为特征提取模块构建领域对抗网络，有效减轻了领域偏移；3）引入Wasserstein距离和相关对齐的表征学习网络，建立了基于Wasserstein距离和相关对齐的迁移学习网络，用于特征提取和识别，挖掘迁移性的特征。结果: 采用平均准确率作为评价指标，ResNet50、DDC、DCORAL、DAN、DANN、CDAN、HAN和JTN这8个模型在11种野生动物上的识别表现分别为48.4%、51.6%、49.6%、52.6%、45.2%、50.9%、54.6%和53.5%；在改进残差模块的ResNet50基础上，并引入Wasserstein距离和相关对齐的表征学习网络后，与现有最佳方法相比，11种（类）野生动物的平均准确率提高了2.7%。结论: 基于Wasserstein距离和相关对齐的迁移学习方法在野生动物识别方面的平均准确率达到57.3%；引入Wasserstein距离和相关对齐的表征学习可有效提高野生动物识别模型的准确性。

关键词: 野生动物, Wasserstein距离, 相关对齐, 迁移学习, 图像识别

Abstract:

Objective: This study aims to address the influence of complex factors such as lighting, background, and shooting scale on the accuracy of wildlife image recognition. Method: In this study, the wild animal images captured by infrared triggered cameras in the wild were used as the object: 1) Two publicly available wildlife datasets, ENA24 and NACTI, were used to construct disjoint datasets S1 and S2, comprising a total of 11 animal categories and 25 591 images. 2) To tackle domain shift issues, a ResNet50 network was utilized as a feature extraction module to build a domain adversarial network, effectively alleviating domain bias. 3) A representation learning network incorporating Wasserstein distance and correlation alignment was proposed to establish a transfer learning network for feature extraction and recognition, so as to further exploit transferable features. Result: The performance of different models in wildlife recognition was evaluated using the average accuracy metric. Results indicated that the average accuracy on 11 wildlife categories for eight models, namely ResNet50, DDC, DCORAL, DAN, DANN, CDAN, HAN, and JTN, was 48.4%, 51.6%, 49.6%, 52.6%, 45.2%, 50.9%, 54.6%, and 53.5%, respectively. Upon enhancing the ResNet50 base model with improved residual modules and introducing a representation learning network incorporating Wasserstein distance and correlation alignment, the average accuracy for 11 wildlife categories was improved by 2.7% compared to the existing best result with the comparative methods. Conclusion: The transfer learning method based on Wasserstein distance and correlation alignment has achieved an average accuracy of 57.3% in wildlife recognition. The introduction of representation learning based on Wasserstein distance and correlation alignment can effectively improve the accuracy of the wildlife recognition model.

Key words: wild animal, Wasserstein distance, correlation alignment, transfer learning, image recognition

中图分类号:

TP391.4

张长春,李大方,张军国. 基于Wasserstein距离和相关对齐迁移学习的野生动物图像识别方法[J]. 林业科学, 2024, 60(8): 25-32.

Changchun Zhang,Dafang Li,Junguo Zhang. Wildlife Images Recognition Method Based on Wasserstein Distance and Correlation Alignment Transfer Learning[J]. Scientia Silvae Sinicae, 2024, 60(8): 25-32.

图/表 8

图1

表1

图2

表2

表3

图3

表4

表5

参考文献 0

	程浙安. 2019. 基于深度卷积神经网络的内蒙古地区陆生野生动物自动识别. 北京: 北京林业大学.
	Cheng Z A.2019. Automatic recognition of terrestrial wildlife in inner mongolia based on deep convolution neural network.Beijing:Beijing Foresty University. ［in Chinese］
	李安琪. 2020. 基于卷积神经网络的野生动物监测图像自动识别方法研究. 北京: 北京林业大学.
	Li A Q. 2020.Research on automatic recognition method of wildlife monitoring images based on convolutional neural network.Beijing: Beijing Forestry University.［in Chinese］
	齐建东, 马鐘添, 张德怀, 等. 基于BS-ResNeXt-50的密云地区野生动物图像识别. 林业科学, 2023, 59 (8): 112- 122. doi: 10.11707/j.1001-7488.LYKX20220378
	Qi J D, Ma Z T, Zhang D H, et al. Wildlife image recognition in Miyun District based on BS-ResNeXt-50. Scientia Silvae Sinicae, 2023, 59 (8): 112- 122. doi: 10.11707/j.1001-7488.LYKX20220378
	谢　斌, 汪　宁, 范有伟. 相关对齐的总变分风格迁移新模型. 中国图象图形学报, 2020, 25 (2): 241- 254.
	Xie B, Wang N, Fan Y W. Correlation alignment total variation model and algorithm for style transfer. Journal of Image and Graphics, 2020, 25 (2): 241- 254.
	Chen P, Zhao R, He T, et al. Unsupervised domain adaptation of bearing fault diagnosis based on join sliced Wasserstein distance. ISA transactions, 2022, 129, 504- 519. doi: 10.1016/j.isatra.2021.12.037
	Ganin Y, Ustinova E, Ajakan H, et al. Domain-adversarial training of neural networks. Journal of Machine Learning Research, 2016, 17 (1): 2096- 2030.
	He K, Zhang X, Ren S, et al. 2016. Deep residual learning for image recognition. IEEE Conference on Computer Vision and Pattern Recognition, 770−778.
	Long M, Cao Y, Cao Z, et al. Transferable representation learning with deep adaptation networks. IEEE Transactions on Pattern Analysis and Machine Intelligence, 2019, 41 (12): 3071- 3085. doi: 10.1109/TPAMI.2018.2868685
	Long M, Cao Z, Wang J, et al. 2018. Conditional adversarial domain adaptation. Advances in Neural Information Processing Systems, 31.
	Miao Z, Liu Z, Gaynor K M, et al. Iterative human and automated identification of wildlife images. Nature Machine Intelligence, 2021, 3 (10): 885- 895. doi: 10.1038/s42256-021-00393-0
	Norouzzadeh M S, Nguyen A, Kosmala M., et al. Automatically identifying, counting, and describing wild animals in camera-trap images with deep learning. Proceedings of the National Academy of Sciences, 2018, 115 (25): E5716- E5725.
	Oza P, Sindagi V A, Sharmini V V, et al. 2023. Unsupervised domain adaptation of object detectors: a survey. IEEE Transactions on Pattern Analysis and Machine Intelligence, 1−24.
	Peng S, Zeng R, Cao L, et al. Multi-source domain adaptation method for textual emotion classification using deep and broad learning. Knowledge-Based Systems, 2023, 260, 110173. doi: 10.1016/j.knosys.2022.110173
	Petso T, Jamisola R S, Mpoeleng D. Review on methods used for wildlife species and individual identification. European Journal of Wildlife Research, 2022, 68, 1- 18. doi: 10.1007/s10344-021-01555-6
	Roy A M, Bhaduri J, Kumar T, et al. WilDect-YOLO: an efficient and robust computer vision-based accurate object localization model for automated endangered wildlife detection. Ecological Informatics, 2023, 75, 101919. doi: 10.1016/j.ecoinf.2022.101919
	Sun B, Saenko K. 2016. Deep coral: Correlation alignment for deep domain adaptation. European Conference on Computer Vision, 443-450.
	Simonyan K, Zisserman A. 2015. Very deep convolutional networks for large-scale image recognition. International Conference on Learning Representations.
	Tuia D, Kellenberger B, Beery S. et al. Perspectives in machine learning for wildlife conservation. Nature Communication, 2022, 13 (1): 1- 15. doi: 10.1038/s41467-021-27699-2
	Tzeng E, Hoffman J, Zhang N, et al. 2014. Deep domain confusion: Maximizing for domain invariance. arXiv preprint arXiv: 1412.3474.
	Tabak M A, Norouzzadeh M S, Wolfson D W, et al. Machine learning to classify animal species in camera trap images: Applications in ecology. Methods in Ecology and Evolution, 2019, 10, 585- 590. doi: 10.1111/2041-210X.13120
	Thangaraj R, Rajendar S, Sanjith M, et al. 2023. Automated recognition of wild animal species in camera trap images using deep learning models. International Conference on Advances in Electrical, Computing, Communication and Sustainable Technologies, 1−5.
	Wang X, Li P, Zhu C, 2020. Classification of wildlife based on transfer learning. International Conference on Video and Image Processing, 236−240.
	Wang S, Wang B, Zhang Z, et al. Class-aware sample reweighting optimal transport for multi-source domain adaptation. Neurocomputing, 2023, 523, 213- 223. doi: 10.1016/j.neucom.2022.12.048
	Yang D Q, Ren G P, Tan K, et al. An adaptive automatic approach to filtering empty images from camera traps using a deep learning model. Wildlife Society Bulletin, 2021, 45, 230- 236. doi: 10.1002/wsb.1176
	Yin Y, Yang Z, Hu H, et al. Universal multi-Source domain adaptation for image classification. Pattern Recognition, 2022, 121, 108238. doi: 10.1016/j.patcog.2021.108238
	Yousif H, Kays R, He Z. 2019. Dynamic programming selection of object proposals for sequence-level animal species classification in the wild. IEEE Transactions on Circuits and Systems for Video Technology, 20.
	Zhang C, Zhao Q, Wu H. Deep domain adaptation via joint transfer networks. Neurocomputing, 2022, 489, 441- 448. doi: 10.1016/j.neucom.2022.03.028
	Zhang C, Zhao Q, Wang Y. Hybrid adversarial network for unsupervised domain adaptation. Information Sciences, 2020, 514, 44- 55. doi: 10.1016/j.ins.2019.12.005
	Zhang C, Zhang J. DJAN: Deep joint adaptation network for wildlife image recognition. Animals, 2023, 13, 3333. doi: 10.3390/ani13213333

数据集 Datasets	熊 Bear	短尾猫 Bobcat	丛林狼 Coyote	鹿 Deer	狐狸 Fox	浣熊 Raccoon	松鼠 Squirrel	臭鼬 Striped skunk	负鼠 Virginia opossum	野猪 Wild boar	火鸡 Wild turkey
S1	2870	455	1814	4336	1866	1376	1386	1323	110	1893	815
S2	867	328	181	2357	427	142	927	295	715	819	289

模型 Models	迁移任务S1→S2 Transfer task S1→S2	迁移任务S2→S1 Transfer task S2→S1	平均准确率 Average accuracy(% )
ResNet50	41.4	55.4	48.4
DDC	44.0	59.2	51.6
DCORAL	44.1	55.1	49.6
DANN	40.8	49.6	45.2
CDAN	45.5	56.2	50.9
DAN	44.7	60.5	52.6
HAN	47.2	62.0	54.6
JTN	44.4	62.5	53.5
WDCA-TLNet	48.4	66.2	57.3

模型 Models	迁移任务S1→S2 Transfer task S1→S2	迁移任务S2→S1 Transfer task S2→S1	平均准确率 Average accuracy
VGG16	42.0	55.0	48.5
DDC	33.8	51.7	42.8
DCORAL	40.8	53.3	47.1
CDAN	43.8	56.5	50.2
JTN	43.5	57.3	50.4
WDCA-TLNet	44.9	58.3	51.6

模型 Models	迁移任务S1→S2 Transfer task S1→S2	迁移任务 S2→S1 Transfer ask S2→S1	平均准确率 Average accuracy（%）
ResNet50	41.4	55.4	48.4
WDCA-TLNet w/o CAL	42.4	58.5	50.5
WDCA-TLNet w/o DAL	44.0	58.4	51.2
WDCA-TLNet w/o WD	46.2	61.0	53.6
WDCA-TLNet	48.4	66.2	57.3

模型 Models	平均准确率 Average accuracy（%）	精确率 Precision	召回率 Recall	F1值
DCORAL	44.1	0.38	0.29	0.24
JTN	44.4	0.38	0.37	0.32
WDCA-TLNet	48.4	0.43	0.40	0.35