An approach to handling non-Gaussianity of parameters and state variables in ensemble Kalman filtering

The ensemble Kalman filter (EnKF) is a commonly used real-time data assimilation algorithm in various disciplines. Here, the EnKF is applied, in a hydrogeological context, to condition log-conductivity realizations on log-conductivity and transient piezometric head data. In this case, the state vector is made up of log-conductivities and piezometric heads over a discretized aquifer domain, the forecast model is a groundwater flow numerical model, and the transient piezometric head data are sequentially assimilated to update the state vector. It is well known that all Kalman filters perform optimally for linear forecast models and a multiGaussian-distributed state vector. Of the different Kalman filters, the EnKF provides a robust solution to address non-linearities; however, it does not handle well non-Gaussian state-vector distributions. In the standard EnKF, as time passes and more state observations are assimilated, the distributions become closer to Gaussian, even if the initial ones are clearly non-Gaussian. A new method is proposed that transforms the original state vector into a new vector that is univariate Gaussian at all times. Back transforming the vector after the filtering ensures that the initial non-Gaussian univariate distributions of the state-vector components are preserved throughout. The proposed method is based in normal-score transforming each variable for all locations and all time steps. This new method, termed the normal-score ensemble Kalman filter (NS-EnKF), is demonstrated in a synthetic bimodal aquifer resembling a fluvial deposit, and it is compared to the standard EnKF. The proposed method performs better than the standard EnKF in all aspects analyzed (log-conductivity characterization and flow and transport predictions).     

Parameter estimation of subsurface flow models using iterative regularized ensemble Kalman filter

A new parameter estimation algorithm based on ensemble Kalman filter (EnKF) is developed. The developed algorithm combined with the proposed problem parametrization offers an efficient parameter estimation method that converges using very small ensembles. The inverse problem is formulated as a sequential data integration problem. Gaussian process regression is used to integrate the prior knowledge (static data). The search space is further parameterized using Karhunen–Loève expansion to build a set of basis functions that spans the search space. Optimal weights of the reduced basis functions are estimated by an iterative regularized EnKF algorithm. The filter is converted to an optimization algorithm by using a pseudo time-stepping technique such that the model output matches the time dependent data. The EnKF Kalman gain matrix is regularized using truncated SVD to filter out noisy correlations. Numerical results show that the proposed algorithm is a promising approach for parameter estimation of subsurface flow models.     

Data assimilation with the ensemble Kalman filter in a numerical model of the North Sea

Coastal management and maritime safety strongly rely on accurate representations of the sea state. Both dynamical models and observations provide abundant pieces of information. However, none of them provides the complete picture. The assimilation of observations into models is one way to improve our knowledge of the ocean state. Its application in coastal models remains challenging because of the wide range of temporal and spatial variabilities of the processes involved. This study investigates the assimilation of temperature profiles with the ensemble Kalman filter in 3-D North Sea simulations. The model error is represented by the standard deviation of an ensemble of model states. Parameters’ values for the ensemble generation are first computed from the misfit between the data and the model results without assimilation. Then, two square root algorithms are applied to assimilate the data. The impact of data assimilation on the simulated temperature is assessed. Results show that the ensemble Kalman filter is adequate for improving temperature forecasts in coastal areas, under adequate model error specification.     

Calibration of a Land Subsidence Model Using InSAR Data via the Ensemble Kalman Filter

The application of interferometric synthetic aperture radar (InSAR) has been increasingly used to improve capabilities to model land subsidence in hydrogeologic studies. A number of investigations over the last decade show how spatially detailed time‐lapse images of ground displacements could be utilized to advance our understanding for better predictions. In this work, we use simulated land subsidences as observed measurements, mimicking InSAR data to inversely infer inelastic specific storage in a stochastic framework. The inelastic specific storage is assumed as a random variable and modeled using a geostatistical method such that the detailed variations in space could be represented and also that the uncertainties of both characterization of specific storage and prediction of land subsidence can be assessed. The ensemble Kalman filter (EnKF), a real‐time data assimilation algorithm, is used to inversely calibrate a land subsidence model by matching simulated subsidences with InSAR data. The performance of the EnKF is demonstrated in a synthetic example in which simulated surface deformations using a reference field are assumed as InSAR data for inverse modeling. The results indicate: (1) the EnKF can be used successfully to calibrate a land subsidence model with InSAR data; the estimation of inelastic specific storage is improved, and uncertainty of prediction is reduced, when all the data are accounted for; and (2) if the same ensemble is used to estimate Kalman gain, the analysis errors could cause filter divergence; thus, it is essential to include localization in the EnKF for InSAR data assimilation.     

The Local Ensemble Transform Kalman Filter with the Weather Research and Forecasting Model: Experiments with Real Observations

The local ensemble transform Kalman filter (LETKF) is implemented with the Weather Research and Forecasting (WRF) model, and real observations are assimilated to assess the newly-developed WRF-LETKF system. The WRF model is a widely-used mesoscale numerical weather prediction model, and the LETKF is an ensemble Kalman filter (EnKF) algorithm particularly efficient in parallel computer architecture. This study aims to provide the basis of future research on mesoscale data assimilation using the WRF-LETKF system, an additional testbed to the existing EnKF systems with the WRF model used in the previous studies. The particular LETKF system adopted in this study is based on the system initially developed in 2004 and has been continuously improved through theoretical studies and wide applications to many kinds of dynamical models including realistic geophysical models. Most recent and important improvements include an adaptive covariance inflation scheme which considers the spatial and temporal inhomogeneity of inflation parameters. Experiments show that the LETKF successfully assimilates real observations and that adaptive inflation is advantageous. Additional experiments with various ensemble sizes show that using more ensemble members improves the analyses consistently.     

Comparison of deterministic ensemble Kalman filters for assimilating hydrogeological data

Groundwater models are critical decision support tools for water resources management and environmental remediation. However, limitations in site characterization data and conceptual models can adversely affect the reliability of groundwater models. Therefore, there is a strong need for continuous model uncertainty reduction. Ensemble filters have recently emerged as promising high-dimensional data assimilation techniques. Two general categories of ensemble filters exist in the literature: perturbation-based and deterministic. Deterministic ensemble filters have been extensively studied for their better performance and robustness in assimilating oceanographic and atmospheric data. In hydrogeology, while a number of previous studies demonstrated the usefulness of the perturbation-based ensemble Kalman filter (EnKF) for joint parameter and state estimation, there have been few systematic studies investigating the performance of deterministic ensemble filters. This paper presents a comparative study of four commonly used deterministic ensemble filters for sequentially estimating the hydraulic conductivity parameter in low- and moderately high-dimensional groundwater models. The performance of the filters is assessed on the basis of twin experiments in which the true hydraulic conductivity field is assumed known. The test results indicate that the deterministic ensemble Kalman filter (DEnKF) is the most robust filter and achieves the best performance at relatively small ensemble sizes. Deterministic ensemble filters often make use of covariance inflation and localization to stabilize filter performance. Sensitivity studies demonstrate the effects of covariance inflation, localization, observation density, and conditioning on filter performance.     

A COMPARISON BETWEEN WIENER FILTERING,KALMAN FILTERING,AND DETERMINISTIC LEAST SQUARES ESTIMATION*

The least squares estimation procedures used in different disciplines can be classified in four categories:

• a. Wiener filtering,
• b. b. Autoregressive estimation,
• c. c. Kalman filtering,
• d. d. Recursive least squares estimation.

The recursive least squares estimator is the time average form of the Kalman filter. Likewise, the autoregressive estimator is the time average form of the Wiener filter. Both the Kalman and the Wiener filters use ensemble averages and can basically be constructed without having a particular measurement realisation available. It follows that seismic deconvolution should be based either on autoregression theory or on recursive least squares estimation theory rather than on the normally used Wiener or Kalman theory. A consequence of this change is the need to apply significance tests on the filter coefficients. The recursive least squares estimation theory is particularly suitable for solving the time variant deconvolution problem.     

A novel general methodology for studying and remedying finite precision error with application in Kalman filtering

 Least squares (LS) techniques, like Kalman filtering, are widely used in environmental science and engineering. In this paper, a new general approach is introduced for the study of the generation, propagation and accumulation of the quantization error in any algorithm. This methodology employs a number of fundamental propositions demonstrating the way the four operations addition, multiplication, division and subtraction, influence quantization error generation and transmission. Using these, one can obtain knowledge of the exact number of erroneous digits with which all quantities of any algorithm are computed at each step of it. This methodology offers understanding of the actual cause of the generation and propagation of finite precision error in any computational scheme. Application of this approach to all Kalman type LS algorithms shows that not all their formulas are equivalent concerning the quantization error effects. More specifically, few generate the greater amount of quantization error. Finally, a stabilization procedure, applicable to all Kalman type algorithms, is introduced that renders all these algorithms very robust.     

The Ensemble Kalman Filter for Groundwater Plume Characterization: A Case Study

The Kalman filter is an efficient data assimilation tool to refine an estimate of a state variable using measured data and the variable's correlations in space and/or time. The ensemble Kalman filter (EnKF) (Evensen 2004, 2009) is a Kalman filter variant that employs Monte Carlo analysis to define the correlations that help to refine the updated state. While use of EnKF in hydrology is somewhat limited, it has been successfully applied in other fields of engineering (e.g., oil reservoir modeling, weather forecasting). Here, EnKF is used to refine a simulated groundwater tetrachloroethylene (TCE) plume that underlies the Tooele Army Depot‐North (TEAD‐N) in Utah, based on observations of TCE in the aquifer. The resulting EnKF‐based assimilated plume is simulated forward in time to predict future plume migration. The correlations that underpin EnKF updating implicitly contain information about how the plume developed over time under the influence of complex site hydrology and variable source history, as they are predicated on multiple realizations of a well‐calibrated numerical groundwater flow and transport model. The EnKF methodology is compared to an ordinary kriging‐based assimilation method with respect to the accurate representation of plume concentrations in order to determine the relative efficacy of EnKF for water quality data assimilation.     

Real-time assimilation of streamflow observations into a hydrological routing model: effects of model structures and updating methods

This paper comparatively assesses the performance of five data assimilation techniques for three-parameter Muskingum routing with a spatially lumped or distributed model structure. The assimilation techniques used include direct insertion (DI), nudging scheme (NS), Kalman filter (KF), ensemble Kalman filter (EnKF) and asynchronous ensemble Kalman filter (AEnKF), which are applied to river reaches in Texas and Louisiana, USA. For both lumped and distributed routing, results from KF, EnKF and AEnKF are sensitive to the error specification. As expected, DI outperformed the other models in the case of lumped modelling, while in distributed routing, KF approaches, particularly AEnKF and EnKF, performed better than DI or nudging, reflecting the benefit of updating distributed states through error covariance modelling in KF approaches. The results of this work would be useful in setting up data assimilation systems that employ increasingly abundant real-time observations using distributed hydrological routing models.     

Calibration framework for a Kalman filter applied to a groundwater model

The paper presents a novel approach to the setup of a Kalman filter by using an automatic calibration framework for estimation of the covariance matrices. The calibration consists of two sequential steps: (1) Automatic calibration of a set of covariance parameters to optimize the performance of the system and (2) adjustment of the model and observation variance to provide an uncertainty analysis relying on the data instead of ad-hoc covariance values. The method is applied to a twin-test experiment with a groundwater model and a colored noise Kalman filter. The filter is implemented in an ensemble framework. It is demonstrated that lattice sampling is preferable to the usual Monte Carlo simulation because its ability to preserve the theoretical mean reduces the size of the ensemble needed. The resulting Kalman filter proves to be efficient in correcting dynamic error and bias over the whole domain studied. The uncertainty analysis provides a reliable estimate of the error in the neighborhood of assimilation points but the simplicity of the covariance models leads to underestimation of the errors far from assimilation points.     

An Application of the Multi-Physics Ensemble Kalman Filter to Typhoon Forecast

This study examines the roles of the multi-physics approach in accounting for model errors for typhoon forecasts with the local ensemble transform Kalman filter (LETKF). Experiments with forecasts of Typhoon Conson (2010) using the weather research and forecasting (WRF) model show that use of the WRF’s multiple physical parameterization schemes to represent the model uncertainties can help the LETKF provide better forecasts of Typhoon Conson in terms of the forecast errors, the ensemble spread, the root mean square errors, the cross-correlation between mass and wind field as well as the coherent structure of the ensemble spread along the storm center. Sensitivity experiments with the WRF model show that the optimum number of the multi-physics ensemble is roughly equal to the number of combinations of different physics schemes assigned in the multi-physics ensemble. Additional idealized experiments with the Lorenz 40-variable model to isolate the dual roles of the multi-physics ensemble in correcting model errors and expanding the local ensemble space show that the multi-physics approach appears to be more essential in augmenting the local rank representation of the LETKF algorithm rather than directly accounting for model errors during the early cycles. The results in this study suggest that the multi-physics approach is a good option for short-range forecast applications with full physics models in which the spinup of the ensemble Kalman filter may take too long for the ensemble spread to capture efficiently model errors and cross-correlations among model variables.     

A land surface data assimilation framework using the land information system: Description and applications

The Land Information System (LIS) is an established land surface modeling framework that integrates various community land surface models, ground measurements, satellite-based observations, high performance computing and data management tools. The use of advanced software engineering principles in LIS allows interoperability of individual system components and thus enables assessment and prediction of hydrologic conditions at various spatial and temporal scales. In this work, we describe a sequential data assimilation extension of LIS that incorporates multiple observational sources, land surface models and assimilation algorithms. These capabilities are demonstrated here in a suite of experiments that use the ensemble Kalman filter (EnKF) and assimilation through direct insertion. In a soil moisture experiment, we discuss the impact of differences in modeling approaches on assimilation performance. Provided careful choice of model error parameters, we find that two entirely different hydrological modeling approaches offer comparable assimilation results. In a snow assimilation experiment, we investigate the relative merits of assimilating different types of observations (snow cover area and snow water equivalent). The experiments show that data assimilation enhancements in LIS are uniquely suited to compare the assimilation of various data types into different land surface models within a single framework. The high performance infrastructure provides adequate support for efficient data assimilation integrations of high computational granularity.     

基于改进广义卡尔曼滤波的结构损伤识别方法

针对传统的广义卡尔曼滤波算法不能有效地追踪结构刚度的变化情况,本文以传统卡尔曼滤波理论为基础,得到了基于衰减记忆的广义卡尔曼滤波算法公式,利用该算法对所得到的地震响应信号进行分析,提取结构的特性,辨识结构的参数,并从中判断结构损伤发生的时刻、位置及程度,改善了广义卡尔曼滤波的效果。衰减记忆的广义卡尔曼滤波算法只能够判断出结构参数变化的时间并且容易出现振荡,因此采用了一种新的自适应追踪技术,用一个自适应因子矩阵代替了原有的遗忘因子,这种技术可以有效追踪结构参数变化的时间、位置和大小,从而能够在线识别出结构的损伤。     

Identification of non-linear and non-stationary soil properties during the 1995 Hyogoken-nanbu earthquake

An extended Kalman filter algorithm with local iteration is presented for the identification of non-linear and non-stationary soil properties. Borehole-array strong motions were recorded at a liquefied site during the 1995 Hyogoken-nanbu earthquake. In this study, a modified Kalman filtering method in which the extended Kalman filter is iteratively used at every local time-step to track rapid parameter changes is proposed. The method is then applied to the instrumented soil layer, which is modeled by an equivalent linear model. An identification of non-linear and non-stationary soil properties was conducted successfully; and non-linear restoring force–displacement relationships including progression with time were obtained.     

Data assimilation for unsaturated flow models with restart adaptive probabilistic collocation based Kalman filter

The ensemble Kalman filter (EnKF) has gained popularity in hydrological data assimilation problems. As a Monte Carlo based method, a sufficiently large ensemble size is usually required to guarantee the accuracy. As an alternative approach, the probabilistic collocation based Kalman filter (PCKF) employs the polynomial chaos expansion (PCE) to represent and propagate the uncertainties in parameters and states. However, PCKF suffers from the so-called “curse of dimensionality”. Its computational cost increases drastically with the increasing number of parameters and system nonlinearity. Furthermore, PCKF may fail to provide accurate estimations due to the joint updating scheme for strongly nonlinear models. Motivated by recent developments in uncertainty quantification and EnKF, we propose a restart adaptive probabilistic collocation based Kalman filter (RAPCKF) for data assimilation in unsaturated flow problems. During the implementation of RAPCKF, the important parameters are identified and active PCE basis functions are adaptively selected at each assimilation step; the “restart” scheme is utilized to eliminate the inconsistency between updated model parameters and states variables. The performance of RAPCKF is systematically tested with numerical cases of unsaturated flow models. It is shown that the adaptive approach and restart scheme can significantly improve the performance of PCKF. Moreover, RAPCKF has been demonstrated to be more efficient than EnKF with the same computational cost.     

Practical aspects of stochastic dynamic tidal modelling

Kalman filtering for stochastic dynamic tidal models, is a hyperbolic filtering problem. The questions of observability and stability of the filter as well as the effects of the finite difference approximation on the filter performance are studied. The degradation of the performance of the filter, in case an erroneous filter model is used, is investigated. In this paper we discuss these various practical aspects of the application of Kalman filtering for tidal flow identification problems. Filters are derived on the basis of the linear shallow water equations. Analytical methods are used to study the performance of the filters under a variety of circumstances.     

Ensemble-Based Observation Targeting for Improving Ozone Prediction in Houston and the Surrounding Area

This study examines the effectiveness of targeted meteorological observations for improving ozone prediction in Houston and the surrounding area based on perfect-model simulation experiments. Supplementary observations are targeted for the location that has the highest impact factor (maximum Kalman gain) estimated from an ensemble and is expected to minimize ozone forecast uncertainty at the verification time. It is found that the observational impact factor field varies with time and is sensitive to ensemble resolutions and physics parameterizations. The efficiency of observation targeting is further examined through assimilating observations in areas with different impact factors using an ensemble Kalman filter. It is found that the ensemble sensitivity analysis is capable of locating supplementary observations that may reduce meteorological and ozone forecast error, but not as effectively as expected.