滨海湿地物联网观测数据预处理方法

连续在线滨海湿地生态物联网观测系统，因传感器技术局限及环境干扰会产生异常观测数据，影响数据使用，有效的数据预处理极为重要。以上海崇明东滩国际重要湿地生态观测数据为研究对象，将异常数据分为数值异常、波动异常与异常事件3种类型，基于回归残差概率分布异常检测算法，使用查找表和多指标时间序列模型，综合多环境要素相互关系，构建针对滨海湿地生态观测的数据预处理方法。相比传统方法，该方法在保证异常数据检测精度的同时，更好地区分了异常事件与传感器异常，减少误判。通过分析9个指标5万余条数据，以10^-8~10^-20的阈值分别检测出0.18%~8.12%的数值异常和波动异常，以及2次异常事件。分析数据预处理结果，传感器的观测原理、观测季节等因素会影响传感器的稳定性，人类活动是造成观测区异常事件发生的主要因素。

Data Preprocessing Method of IoT Observation System in Coastal Wetland

Effective data preprocessing is essential to an online coastal wetland ecological internet of things (IoT) observation system. Outliers always occur due to the limitations of measuring methods and harsh environmental conditions, which challenge data applications. Based on the ecological observation data of Chongming Dongtan wetland in Shanghai, the outliers were divided into three types:abnormal values, abnormal fluctuation,and abnormal events. Integrating the interactions between indicators of coastal wetlands, we proposed a preprocessing method for the outliers of the coastal wetland ecological IoT system based on the residual probabilistic outlier detection algorithm, look-up table, and multi-indicator time series model. Compared with the traditional methods, this method can not only ensure the accuracy of outlier detection, but also better distinguish abnormal events from sensor problems to reduce false positives. Through the analysis of more than 50 000 data records of nine indicators, two abnormal events and 0.18%-8.12% abnormal values and abnormal fluctuations were detected with the threshold of 10^-8-10^-20. Through the analysis of the preprocessed data, we find that the observation principle and observation season will affect the stability of sensors, and the human activities in the observation area are the main factors causing abnormal events.