Modeling and prediction of environmental data in space and time using Kalman filtering

 The Kalman filter is used in this paper as a framework for space time data analysis. Using Kalman filtering it is possible to include physically based simulation models into the data analysis procedure. Attention is concentrated on the development of fast filter algorithms to make Kalman filtering feasible for high dimensional space time models. The ensemble Kalman filter and the reduced rank square root filter algorithm are briefly summarized. A new algorithm, the partially orthogonal ensemble Kalman filter is introduced too. We will illustrate the performance of the Kalman filter algorithms with a real life air pollution problem. Here ozone concentrations in a part of North West Europe are estimated and predicted.     

一种新的基于卡尔曼滤波的地震记录同相轴跟踪方法及性能分析

在地震勘探领域中,卡尔曼滤波常用于地震信号的反褶积以提高地震勘探资料的分辨率和信噪比. 不同于此,本文建立一个新的卡尔曼滤波系统模型并利用卡尔曼滤波跟踪地震记录同相轴. 同相轴信息在对地下介质性质、界面的深度、界面的产状以及油气定性判别等方面具有极其重要的作用. 目前多数拾取地震同相轴的方法与地震波的运动规律结合较少.本文依据地震反射波运动规律构建了用于跟踪地震反射同相轴的卡尔曼滤波系统的状态方程和量测方程,并将炮检距、地震子波到达时和层速度等重要物理量融入所建方程,给出滤波模型和初始化方法,分析不同因素对该系统滤波性能的影响. 仿真实验表明,所提出的跟踪滤波系统能较好地拾取地震反射同相轴信息,为拾取地震同相轴提供了一条新途径.     

A COMPARISON BETWEEN WIENER FILTERING,KALMAN FILTERING,AND DETERMINISTIC LEAST SQUARES ESTIMATION*

The least squares estimation procedures used in different disciplines can be classified in four categories:

• a. Wiener filtering,
• b. b. Autoregressive estimation,
• c. c. Kalman filtering,
• d. d. Recursive least squares estimation.

The recursive least squares estimator is the time average form of the Kalman filter. Likewise, the autoregressive estimator is the time average form of the Wiener filter. Both the Kalman and the Wiener filters use ensemble averages and can basically be constructed without having a particular measurement realisation available. It follows that seismic deconvolution should be based either on autoregression theory or on recursive least squares estimation theory rather than on the normally used Wiener or Kalman theory. A consequence of this change is the need to apply significance tests on the filter coefficients. The recursive least squares estimation theory is particularly suitable for solving the time variant deconvolution problem.     

Identification of non-linear and non-stationary soil properties during the 1995 Hyogoken-nanbu earthquake

An extended Kalman filter algorithm with local iteration is presented for the identification of non-linear and non-stationary soil properties. Borehole-array strong motions were recorded at a liquefied site during the 1995 Hyogoken-nanbu earthquake. In this study, a modified Kalman filtering method in which the extended Kalman filter is iteratively used at every local time-step to track rapid parameter changes is proposed. The method is then applied to the instrumented soil layer, which is modeled by an equivalent linear model. An identification of non-linear and non-stationary soil properties was conducted successfully; and non-linear restoring force–displacement relationships including progression with time were obtained.     

Application of the distributed parameter filter to predict simulated tidal induced shallow water flow

Distributed parameter filtering theory is employed for estimating the state variables and associated error covariances of a dynamical distributed system under highly random tidal and meteorological influences. The stochastic-deterministic mathematical model of the physical system under study consists of the shallow water equations described by the momentum and continuity equations in which the external forces such as Coriolis force, wind friction, and atmospheric pressure are considered. White Gaussian noises in the system and measurement equations are used to account for the inherent stochasticity of the system. By using an optimal distributed parameter filter, the information provided by the stochastic dynamical model and the noisy measurements taken from the actual system are combined to obtain an optimal estimate of the state of the system, which in turn is used as the initial condition for the prediction procedure. The approach followed here has numerical approximation carried out at the end, which means that the numerical discretization is performed in the filtering equations, and not in the equations modelling the system. Therefore, the continuous distributed nature of the original system is maintained as long as possible and the propagation of modelling errors in the problem is minimized. The appropriateness of the distributed parameter filter is demonstrated in an application involving the prediction of storm surges in the North Sea. The results confirm excellent filter performance with considerable improvement with respect to the deterministic prediction.     

Application of the distributed parameter filter to predict simulated tidal induced shallow water flow

Distributed parameter filtering theory is employed for estimating the state variables and associated error covariances of a dynamical distributed system under highly random tidal and meteorological influences. The stochastic-deterministic mathematical model of the physical system under study consists of the shallow water equations described by the momentum and continuity equations in which the external forces such as Coriolis force, wind friction, and atmospheric pressure are considered. White Gaussian noises in the system and measurement equations are used to account for the inherent stochasticity of the system. By using an optimal distributed parameter filter, the information provided by the stochastic dynamical model and the noisy measurements taken from the actual system are combined to obtain an optimal estimate of the state of the system, which in turn is used as the initial condition for the prediction procedure. The approach followed here has numerical approximation carried out at the end, which means that the numerical discretization is performed in the filtering equations, and not in the equations modelling the system. Therefore, the continuous distributed nature of the original system is maintained as long as possible and the propagation of modelling errors in the problem is minimized. The appropriateness of the distributed parameter filter is demonstrated in an application involving the prediction of storm surges in the North Sea. The results confirm excellent filter performance with considerable improvement with respect to the deterministic prediction.     

基于改进广义卡尔曼滤波的结构损伤识别方法

针对传统的广义卡尔曼滤波算法不能有效地追踪结构刚度的变化情况,本文以传统卡尔曼滤波理论为基础,得到了基于衰减记忆的广义卡尔曼滤波算法公式,利用该算法对所得到的地震响应信号进行分析,提取结构的特性,辨识结构的参数,并从中判断结构损伤发生的时刻、位置及程度,改善了广义卡尔曼滤波的效果。衰减记忆的广义卡尔曼滤波算法只能够判断出结构参数变化的时间并且容易出现振荡,因此采用了一种新的自适应追踪技术,用一个自适应因子矩阵代替了原有的遗忘因子,这种技术可以有效追踪结构参数变化的时间、位置和大小,从而能够在线识别出结构的损伤。     

Kalman滤波在沉降监测数据处理中的应用

将卡尔曼滤波应用于建筑物变形监测数据分析,给出了离散卡尔曼滤波模型的建立思路及相应的精度评定公式,结合西安市某建筑物沉降观测数据,建立了相应的离散卡尔曼滤波数学模型,通过Matlab编程对观测数据进行处理,成果图像显示滤波值曲线与原始观测数据曲线的变化趋势基本一致。该模型较好地模拟了建筑物沉降的变化规律,对于改善沉降监测数据处理的精度效果非常理想。     

Identification of wind stress on shallow water surfaces by optimal smoothing

Using the state space approach, an on-line filter procedure for combined wind stress identification and tidal flow forecasting is developed. The stochastic dynamic approach is based on the linear twodimensional shallow water equations. Using a finite difference scheme, a system representation of the model is obtained. To account for uncertainties, the system is embedded into a stochastic environment. By employing a Kalman filter, the on-line measurements of the water-level available can be used to identify and predict the shallow water flow. Because it takes a certain time before a fluctuation in the wind stress can be noticed in the water-level measurements, an optimal fixed-lag smoother is used to identify the stress.     

根据数字滤波器的频率响应优选数字处理方法

本文根据数字滤波器压制干扰的原理,指出利用数字滤波器的频率响应、优选数字处理方法和确定最佳滤波参数的具体原则。对于那些未给出频率响应函数或周期选择性函数的数字处理方法,可以直接根据计算方法的权系数算出它们的频率响应。计算递归滤波器的频率响应式子,也可以用来计算其它非递归滤波器的频率响应。本文利用重力固体潮汐分析中计算中心点的零点飘移值若干方法的系数和它们的周期选择性函数,分别计算了它们的频率响应,从两者的数值上比较,结果是一致的。本文还计算了目前大家常用的一些数字处理方法,如五日均值、一阶差分、二阶差分、多点数字平滑等方法的频率响应值。并分析了它们的处理效果和方法的局限性。还根据最佳数字滤波器和DAI数字滤波器的频率响应曲线,说明选择数字滤波器和确定最佳滤波参数的具体方法。     

Comparison of deterministic ensemble Kalman filters for assimilating hydrogeological data

Groundwater models are critical decision support tools for water resources management and environmental remediation. However, limitations in site characterization data and conceptual models can adversely affect the reliability of groundwater models. Therefore, there is a strong need for continuous model uncertainty reduction. Ensemble filters have recently emerged as promising high-dimensional data assimilation techniques. Two general categories of ensemble filters exist in the literature: perturbation-based and deterministic. Deterministic ensemble filters have been extensively studied for their better performance and robustness in assimilating oceanographic and atmospheric data. In hydrogeology, while a number of previous studies demonstrated the usefulness of the perturbation-based ensemble Kalman filter (EnKF) for joint parameter and state estimation, there have been few systematic studies investigating the performance of deterministic ensemble filters. This paper presents a comparative study of four commonly used deterministic ensemble filters for sequentially estimating the hydraulic conductivity parameter in low- and moderately high-dimensional groundwater models. The performance of the filters is assessed on the basis of twin experiments in which the true hydraulic conductivity field is assumed known. The test results indicate that the deterministic ensemble Kalman filter (DEnKF) is the most robust filter and achieves the best performance at relatively small ensemble sizes. Deterministic ensemble filters often make use of covariance inflation and localization to stabilize filter performance. Sensitivity studies demonstrate the effects of covariance inflation, localization, observation density, and conditioning on filter performance.     

位移和加速度融合的自适应多速率Kalman滤波方法

多速率Kalman滤波方法可用于低采样率的位移和高采样率的加速度数据融合,而未知的噪声协方差信息则显著制约着多速率Kalman滤波精度.本文通过将多速率Kalman滤波转换为传统的单速率Kalman滤波,建立了Kalman滤波增益的自协方差矢量与未知的加速度谱密度和观测噪声参数间的线性函数模型,并采用最小二乘估计方法对未知的噪声协方差参数进行估计,进而有效地提高了多速率Kalman滤波精度.数值仿真和震动台实验结果验证了本文方法的正确性和有效性.     

An approach to handling non-Gaussianity of parameters and state variables in ensemble Kalman filtering

The ensemble Kalman filter (EnKF) is a commonly used real-time data assimilation algorithm in various disciplines. Here, the EnKF is applied, in a hydrogeological context, to condition log-conductivity realizations on log-conductivity and transient piezometric head data. In this case, the state vector is made up of log-conductivities and piezometric heads over a discretized aquifer domain, the forecast model is a groundwater flow numerical model, and the transient piezometric head data are sequentially assimilated to update the state vector. It is well known that all Kalman filters perform optimally for linear forecast models and a multiGaussian-distributed state vector. Of the different Kalman filters, the EnKF provides a robust solution to address non-linearities; however, it does not handle well non-Gaussian state-vector distributions. In the standard EnKF, as time passes and more state observations are assimilated, the distributions become closer to Gaussian, even if the initial ones are clearly non-Gaussian. A new method is proposed that transforms the original state vector into a new vector that is univariate Gaussian at all times. Back transforming the vector after the filtering ensures that the initial non-Gaussian univariate distributions of the state-vector components are preserved throughout. The proposed method is based in normal-score transforming each variable for all locations and all time steps. This new method, termed the normal-score ensemble Kalman filter (NS-EnKF), is demonstrated in a synthetic bimodal aquifer resembling a fluvial deposit, and it is compared to the standard EnKF. The proposed method performs better than the standard EnKF in all aspects analyzed (log-conductivity characterization and flow and transport predictions).     

基于无迹卡尔曼滤波的新安江模型实时校正方法

通过利用实时水文观测数据对洪水预报模型进行校正,可增加流域洪水预报的实时性和精确度.本文讨论了水文模型状态变量选取对滤波效果的影响,并给出了状态变量选取原则.在集总式新安江模型的基础上,结合状态变量选取原则,应用无迹卡尔曼滤波技术构建了新安江模型的实时校正方法.方法应用于闽江邵武流域洪水预报的计算结果表明,采用无迹卡尔曼滤波方法后,不仅能够直接校正模型状态,同时也能有效地提高模型预报精度,适合应用于实际流域洪水预报作业中.     

自适应卡尔曼滤波在航空重力异常解算的应用研究

依据航空重力测量基本原理,构建了航空重力异常解算的卡尔曼滤波模型,将新息自适应卡尔曼滤波器(IAE,Innovation based Adaptive Estimation)应用于量测噪声未知的航空重力异常解算.针对IAE滤波器滑动窗口宽度难以准确确定的问题,通过对多个不同滑动窗口新息协方差估计的加权平均,获得改进的IAE滤波器,该IAE滤波器不仅具有量测噪声自适应估计能力,还能实现滑动采样窗口的优化选取.试验结果表明,IAE滤波器可以降低因量测噪声统计信息不明引起的解算误差,改进IAE解算的重力异常误差约为1mGal.     

Selection of noise parameters for Kalman filter

The Bayesian probabilistic approach is proposed to estimate the process noise and measurement noise parameters for a Kalman filter. With state vectors and covariance matrices estimated by the Kalman filter, the likehood of the measurements can be constructed as a function of the process noise and measurement noise parameters. By maximizing the likelihood function with respect to these noise parameters, the optimal values can be obtained. Furthermore, the Bayesian probabilistic approach allows the associated uncertainty to be quantified. Examples using a single-degree-of-freedom system and a ten-story building illustrate the proposed method. The effect on the performance of the Kalman filter due to the selection of the process noise and measurement noise parameters was demonstrated. The optimal values of the noise parameters were found to be close to the actual values in the sense that the actual parameters were in the region with significant probability density. Through these examples, the Bayesian approach was shown to have the capability to provide accurate estimates of the noise parameters of the Kalman filter, and hence for state estimation.     

THE KALMAN FILTER AS A PREDICTION ERROR FILTER*

In mathematical statistical filtering the deconvolution problem can be solved by two different methods:

• 1 by inverse filtering
• 2 by calculating the prediction error.

Both methods are well known in the theory of Wiener filters. If, however, the generating process of the signal is known and can be described by a set of linear first order differential equations, then the Kalman filter can also be used to solve the deconvolution problem. In the case of the inverse filtering method this was shown by Bayless and Brigham (1970). But, while their method can only be used if the original signal is a colored random process, this paper shows that in the case of a white process the prediction error filtering method is a more appropriate approach. The method is extremely efficient and simple. This can be demonstrated by an example which maybe of special interest for seismic exploration.     

Median filter behaviour with seismic data1

Median filters may be used with seismic data to attenuate coherent wavefields. An example is the attenuation of the downgoing wavefield in VSP data processing. The filter is applied across the traces in the ‘direction’ of the wavefield. The final result is given by subtracting the filtered version of the record from the original record. This method of median filtering may be called ‘median filtering operated in subtraction’. The method may be extended by automatically estimating the slowness of coherent wavefields on a record. The filter is then applied in a time- and-space varying manner across the record on the basis of the slowness values at each point on the record. Median filters are non-linear and hence their behaviour is more difficult to determine than linear filters. However, there are a number of methods that may be used to analyse median filter behaviour: (1) pseudo-transfer functions to specific time series; (2) the response of median filters to simple seismic models; and (3) the response of median filters to steps that simulate terminating wavefields, such as faults on stacked data. These simple methods provide an intuitive insight into the behaviour of these filters, as well as providing a semiquantitative measurement of performance. The performance degradation of median filters in the presence of trace-to-trace variations in amplitude is shown to be similar to that of linear filters. The performance of median filters (in terms of signal distortion) applied obliquely across a record may be improved by low-pass filtering (in the t-dimension). The response of median filters to steps is shown to be affected by background noise levels. The distortion of steps introduced by median filters approaches the distortion of steps introduced by the corresponding linear filter for high levels of noise.     

The attenuation of tidal and subtidal oscillations in the Patos Lagoon estuary

The single, long and narrow channel that usually connects choked coastal lagoons to the ocean can serve as a natural hydraulic low-pass filter that reduces or eliminates tidal and subtidal effects inside the lagoon. This study proposes an alternative method of estimating the attenuation of the tidal and subtidal oscillations throughout the Patos Lagoon estuary. The attenuation is estimated for conditions of contrasting river runoff and weather (summer and winter). A high-pass/low-pass filter (fast fourier transformation technique – FFT) is applied to time series of sea-surface elevation (SSE) measured at the mouth of the Patos Lagoon. The resulting high-frequency (tidal) and low-frequency (subtidal) signals are used in independent simulations to force the TELEMAC-2D model. Attenuation of the tidal and subtidal signals throughout the estuary is estimated by applying cross-spectral analysis between the model-generated SSE time series at different locations throughout the estuary and the filtered SSE time series measured at the mouth. Results from the proposed method suggest that: (1) the low-frequency (subtidal) oscillations are less attenuated and propagate further than the high-frequency (tidal) oscillations in the Patos Lagoon estuary; (2) the filtering capability of the Patos Lagoon estuary is expected to follow a seasonal pattern, although further investigations on an interannual time scale are recommended in order to confirm this hypothesis; (3) the influence of the oceanic boundary processes on the SSE dynamics of the lagoon is restricted to the lower estuary. Further inland, the local forcing generated by the wind and freshwater input is likely to be the main forcing effect controlling the dynamics of the system. The proposed method proved to be an efficient and alternative way of estimating the attenuation of energy in the tidal and subtidal bands throughout the access channel of a choked coastal lagoon located in an area of reduced tidal influence.Responsible Editor: Iris Grabemann