关键词: 缺失数据, 数据填补, 时间序列, LSTM神经网络, 茎体水分

Abstract:

Objective: With the advent of the era of big data, ecological data is emerging in large numbers, but there are data missing phenomena in the process of collection and transmission, resulting in incomplete data, which brings difficulties for subsequent analysis and application. In order to improve the integrity and accuracy of data, it is important to find a suitable data filling method. In this study, the stem moisture data of a plant was used as the object. For the missing data on the same data segment, the different data filling methods were compared to verify the validity and accuracy of the LSTM model to fill the stem moisture data. Method: The integral data of stems water of Lagerstroemia indica planted in Haidian District of Beijing in June 2017 were selected as experimental data, and some data were manually deleted as missing data. The missing parts were filled by interpolation method, RNN neural network and LSTM neural network respectively. The results were compared with the original data and analyzed. Based on the error distribution of neural network predictive value error which increases with the delay of the prediction time, this paper proposed a method of adding late data processing on the basis of neural network prediction value:Prediction was implemented from the forward and reverse two directions of missing data, and the predicted values were multiplied by a set of weight values decreasing according to the prediction time, and then added together. In combination of the advantages of the two prediction directions, the prediction accuracy could be further improved. Result: Among the three methods, the RNN and LSTM neural network methods had obvious advantages compared with the traditional interpolation methods. The accuracy of the interpolation method decreased rapidly when the missing value increases, while the neural network method decreased slowly. When the error between filled value and real value was set within 2% as the accurate, the filling accuracy of the interpolation method was less than 50%, the RNN method was between 50% to 60%, and the LSTM method reached 80% or more; When the error between filled value and real value was set within 4% as the accurate, the filling accuracy of the interpolation method was 60%, the highest accuracy of the RNN method reached to 90%, and the accuracy of the LSTM method was more than 95%. When the weight processing was added on this basis, ,the accuracy of the LSTM prediction result was 97% within 2%, and 100% within 3%. Conclusion: This paper innovatively adopts the bidirectional comprehensive prediction method based on LSTM model, which significantly reduces the influence of cumulative error in long-term prediction on prediction results and improves the accuracy of prediction data. Compared with the other two kinds of data filling methods, the data filling method based on LSTM Neural network has a greater advantage in the long-term missing time series data filling.

Key words: missing data, data filling, time series, LSTM neural network, stem moisture