An approach to handling non-Gaussianity of parameters and state variables in ensemble Kalman filtering

The ensemble Kalman filter (EnKF) is a commonly used real-time data assimilation algorithm in various disciplines. Here, the EnKF is applied, in a hydrogeological context, to condition log-conductivity realizations on log-conductivity and transient piezometric head data. In this case, the state vector is made up of log-conductivities and piezometric heads over a discretized aquifer domain, the forecast model is a groundwater flow numerical model, and the transient piezometric head data are sequentially assimilated to update the state vector. It is well known that all Kalman filters perform optimally for linear forecast models and a multiGaussian-distributed state vector. Of the different Kalman filters, the EnKF provides a robust solution to address non-linearities; however, it does not handle well non-Gaussian state-vector distributions. In the standard EnKF, as time passes and more state observations are assimilated, the distributions become closer to Gaussian, even if the initial ones are clearly non-Gaussian. A new method is proposed that transforms the original state vector into a new vector that is univariate Gaussian at all times. Back transforming the vector after the filtering ensures that the initial non-Gaussian univariate distributions of the state-vector components are preserved throughout. The proposed method is based in normal-score transforming each variable for all locations and all time steps. This new method, termed the normal-score ensemble Kalman filter (NS-EnKF), is demonstrated in a synthetic bimodal aquifer resembling a fluvial deposit, and it is compared to the standard EnKF. The proposed method performs better than the standard EnKF in all aspects analyzed (log-conductivity characterization and flow and transport predictions).     

An adaptive hybrid EnKF-OI scheme for efficient state-parameter estimation of reactive contaminant transport models

Reactive contaminant transport models are used by hydrologists to simulate and study the migration and fate of industrial waste in subsurface aquifers. Accurate transport modeling of such waste requires clear understanding of the system’s parameters, such as sorption and biodegradation. In this study, we present an efficient sequential data assimilation scheme that computes accurate estimates of aquifer contamination and spatially variable sorption coefficients. This assimilation scheme is based on a hybrid formulation of the ensemble Kalman filter (EnKF) and optimal interpolation (OI) in which solute concentration measurements are assimilated via a recursive dual estimation of sorption coefficients and contaminant state variables. This hybrid EnKF-OI scheme is used to mitigate background covariance limitations due to ensemble under-sampling and neglected model errors. Numerical experiments are conducted with a two-dimensional synthetic aquifer in which cobalt-60, a radioactive contaminant, is leached in a saturated heterogeneous clayey sandstone zone. Assimilation experiments are investigated under different settings and sources of model and observational errors. Simulation results demonstrate that the proposed hybrid EnKF-OI scheme successfully recovers both the contaminant and the sorption rate and reduces their uncertainties. Sensitivity analyses also suggest that the adaptive hybrid scheme remains effective with small ensembles, allowing to reduce the ensemble size by up to 80% with respect to the standard EnKF scheme.     

Data assimilation methods for estimating a heterogeneous conductivity field by assimilating transient solute transport data via ensemble Kalman filter

An ensemble Kalman filter (EnKF) is developed to identify a hydraulic conductivity distribution in a heterogeneous medium by assimilating solute concentration measurements of solute transport in the field with a steady‐state flow. A synthetic case with the mixed Neumann/Dirichlet boundary conditions is designed to investigate the capacity of the data assimilation methods to identify a conductivity distribution. The developed method is demonstrated in 2‐D transient solute transport with two different initial instant solute injection areas. The influences of the observation error and model error on the updated results are considered in this study. The study results indicate that the EnKF method will significantly improve the estimation of the hydraulic conductivity field by assimilating solute concentration measurements. The larger area of the initial distribution and the more observed data obtained, the better the calculation results. When the standard deviation of the observation error varies from 1% to 30% of the solute concentration measurements, the simulated results by the data assimilation method do not change much, which indicates that assimilation results are not very sensitive to the standard deviation of the observation error in this study. When the inflation factor is more than 1.0 to enlarge the model error by increasing the forecast error covariance matrix, the updated results of the hydraulic conductivity by the data assimilation method are not good at all. Copyright © 2012 John Wiley & Sons, Ltd.     

The Local Ensemble Transform Kalman Filter with the Weather Research and Forecasting Model: Experiments with Real Observations

The local ensemble transform Kalman filter (LETKF) is implemented with the Weather Research and Forecasting (WRF) model, and real observations are assimilated to assess the newly-developed WRF-LETKF system. The WRF model is a widely-used mesoscale numerical weather prediction model, and the LETKF is an ensemble Kalman filter (EnKF) algorithm particularly efficient in parallel computer architecture. This study aims to provide the basis of future research on mesoscale data assimilation using the WRF-LETKF system, an additional testbed to the existing EnKF systems with the WRF model used in the previous studies. The particular LETKF system adopted in this study is based on the system initially developed in 2004 and has been continuously improved through theoretical studies and wide applications to many kinds of dynamical models including realistic geophysical models. Most recent and important improvements include an adaptive covariance inflation scheme which considers the spatial and temporal inhomogeneity of inflation parameters. Experiments show that the LETKF successfully assimilates real observations and that adaptive inflation is advantageous. Additional experiments with various ensemble sizes show that using more ensemble members improves the analyses consistently.     

Evaluating the utility of the ensemble transform Kalman filter for adaptive sampling when updating a hydrodynamic model

This paper compares two Monte Carlo sequential data assimilation methods based on the Kalman filter, for estimating the effect of measurements on simulations of state error variance made by a one-dimensional hydrodynamic model. The first method used an ensemble Kalman filter (EnKF) to update state estimates, which were then used as initial conditions for further simulations. The second method used an ensemble transform Kalman filter (ETKF) to quickly estimate the effect of measurement error covariance on forecast error covariance without the need to re-run the simulation model. The ETKF gave an unbiased estimate of EnKF analysed error variance, although differences in the treatment of measurement errors meant the results were not identical. Estimates of forecast error variance could also be made, but their accuracy deteriorated as the time from measurements increased due in part to model non-linearity and the decreasing signal variance. The motivation behind the study was to assess the ability of the ETKF to target possible measurements, as part of an adaptive sampling framework, before they are assimilated by an EnKF-based forecasting model on the River Crouch, Essex, UK. The ETKF was found to be a useful tool for quickly estimating the error covariance expected after assimilating measurements into the hydrodynamic model. It, thus, provided a means of quantifying the ‘usefulness’ (in terms of error variance) of possible sampling schemes.     

Real-time assimilation of streamflow observations into a hydrological routing model: effects of model structures and updating methods

This paper comparatively assesses the performance of five data assimilation techniques for three-parameter Muskingum routing with a spatially lumped or distributed model structure. The assimilation techniques used include direct insertion (DI), nudging scheme (NS), Kalman filter (KF), ensemble Kalman filter (EnKF) and asynchronous ensemble Kalman filter (AEnKF), which are applied to river reaches in Texas and Louisiana, USA. For both lumped and distributed routing, results from KF, EnKF and AEnKF are sensitive to the error specification. As expected, DI outperformed the other models in the case of lumped modelling, while in distributed routing, KF approaches, particularly AEnKF and EnKF, performed better than DI or nudging, reflecting the benefit of updating distributed states through error covariance modelling in KF approaches. The results of this work would be useful in setting up data assimilation systems that employ increasingly abundant real-time observations using distributed hydrological routing models.     

Prediction and predictability of a catastrophic local extreme precipitation event through cloud-resolving ensemble analysis and forecasting with Doppler radar observations

Local extreme rain usually resulted in disasters such as flash floods and landslides. Upon today, it is still one of the most difficult tasks for operational weather forecast centers to predict those events accurately. In this paper, we simulate an extreme precipitation event with ensemble Kalman filter (EnKF) assimilation of Doppler radial-velocity observations, and analyze the uncertainties of the assimilation. The results demonstrate that, without assimilation radar data, neither a single initialization of deterministic forecast nor an ensemble forecast with adding perturbations or multiple physical parameterizations can predict the location of strong precipitation. However, forecast was significantly improved with assimilation of radar data, especially the location of the precipitation. The direct cause of the improvement is the buildup of a deep mesoscale convection system with EnKF assimilation of radar data. Under a large scale background favorable for mesoscale convection, efficient perturbations of upstream mid-low level meridional wind and moisture are key factors for the assimilation and forecast. Uncertainty still exists for the forecast of this case due to its limited predictability. Both the difference of large scale initial fields and the difference of analysis obtained from EnKF assimilation due to small amplitude of initial perturbations could have critical influences to the event's prediction. Forecast could be improved through more cycles of EnKF assimilation. Sensitivity tests also support that more accurate forecasts are expected through improving numerical models and observations.     

Comparison of deterministic ensemble Kalman filters for assimilating hydrogeological data

Groundwater models are critical decision support tools for water resources management and environmental remediation. However, limitations in site characterization data and conceptual models can adversely affect the reliability of groundwater models. Therefore, there is a strong need for continuous model uncertainty reduction. Ensemble filters have recently emerged as promising high-dimensional data assimilation techniques. Two general categories of ensemble filters exist in the literature: perturbation-based and deterministic. Deterministic ensemble filters have been extensively studied for their better performance and robustness in assimilating oceanographic and atmospheric data. In hydrogeology, while a number of previous studies demonstrated the usefulness of the perturbation-based ensemble Kalman filter (EnKF) for joint parameter and state estimation, there have been few systematic studies investigating the performance of deterministic ensemble filters. This paper presents a comparative study of four commonly used deterministic ensemble filters for sequentially estimating the hydraulic conductivity parameter in low- and moderately high-dimensional groundwater models. The performance of the filters is assessed on the basis of twin experiments in which the true hydraulic conductivity field is assumed known. The test results indicate that the deterministic ensemble Kalman filter (DEnKF) is the most robust filter and achieves the best performance at relatively small ensemble sizes. Deterministic ensemble filters often make use of covariance inflation and localization to stabilize filter performance. Sensitivity studies demonstrate the effects of covariance inflation, localization, observation density, and conditioning on filter performance.     

Data assimilation for unsaturated flow models with restart adaptive probabilistic collocation based Kalman filter

The ensemble Kalman filter (EnKF) has gained popularity in hydrological data assimilation problems. As a Monte Carlo based method, a sufficiently large ensemble size is usually required to guarantee the accuracy. As an alternative approach, the probabilistic collocation based Kalman filter (PCKF) employs the polynomial chaos expansion (PCE) to represent and propagate the uncertainties in parameters and states. However, PCKF suffers from the so-called “curse of dimensionality”. Its computational cost increases drastically with the increasing number of parameters and system nonlinearity. Furthermore, PCKF may fail to provide accurate estimations due to the joint updating scheme for strongly nonlinear models. Motivated by recent developments in uncertainty quantification and EnKF, we propose a restart adaptive probabilistic collocation based Kalman filter (RAPCKF) for data assimilation in unsaturated flow problems. During the implementation of RAPCKF, the important parameters are identified and active PCE basis functions are adaptively selected at each assimilation step; the “restart” scheme is utilized to eliminate the inconsistency between updated model parameters and states variables. The performance of RAPCKF is systematically tested with numerical cases of unsaturated flow models. It is shown that the adaptive approach and restart scheme can significantly improve the performance of PCKF. Moreover, RAPCKF has been demonstrated to be more efficient than EnKF with the same computational cost.     

Calibration of a Land Subsidence Model Using InSAR Data via the Ensemble Kalman Filter

The application of interferometric synthetic aperture radar (InSAR) has been increasingly used to improve capabilities to model land subsidence in hydrogeologic studies. A number of investigations over the last decade show how spatially detailed time‐lapse images of ground displacements could be utilized to advance our understanding for better predictions. In this work, we use simulated land subsidences as observed measurements, mimicking InSAR data to inversely infer inelastic specific storage in a stochastic framework. The inelastic specific storage is assumed as a random variable and modeled using a geostatistical method such that the detailed variations in space could be represented and also that the uncertainties of both characterization of specific storage and prediction of land subsidence can be assessed. The ensemble Kalman filter (EnKF), a real‐time data assimilation algorithm, is used to inversely calibrate a land subsidence model by matching simulated subsidences with InSAR data. The performance of the EnKF is demonstrated in a synthetic example in which simulated surface deformations using a reference field are assumed as InSAR data for inverse modeling. The results indicate: (1) the EnKF can be used successfully to calibrate a land subsidence model with InSAR data; the estimation of inelastic specific storage is improved, and uncertainty of prediction is reduced, when all the data are accounted for; and (2) if the same ensemble is used to estimate Kalman gain, the analysis errors could cause filter divergence; thus, it is essential to include localization in the EnKF for InSAR data assimilation.     

Application of data assimilation with the Root Zone Water Quality Model for soil moisture profile estimation in the upper Cedar Creek,Indiana

Data assimilation techniques have been proven as an effective tool to improve model forecasts by combining information about observed variables in many areas. This article examines the potential of assimilating surface soil moisture observations into a field‐scale hydrological model, the Root Zone Water Quality Model, to improve soil moisture estimation. The Ensemble Kalman Filter (EnKF), a popular data assimilation technique for nonlinear systems, was applied and compared with a simple direct insertion method. In situ soil moisture data at four different depths (5, 20, 40, and 60 cm) from two agricultural fields (AS1 and AS2) in northeastern Indiana were used for assimilation and validation purposes. Through daily update, the EnKF improved soil moisture estimation compared with the direct insertion method and model results without assimilation, having more distinct improvement at the 5 and 20 cm depths than for deeper layers (40 and 60 cm). Local vertical soil property heterogeneity in AS1 deteriorated soil moisture estimates with the EnKF. Removal of systematic bias in the forecast model was found to be critical for more successful soil moisture data assimilation studies. This study also demonstrates that a more frequent update generally contributes in enhancing the open loop simulation; however, large forecasting error can prevent more frequent update from providing better results. In addition, results indicate that various ensemble sizes make little difference in the assimilation results. An ensemble of 100 members produced results that were comparable with results obtained from larger ensembles. Copyright © 2011 John Wiley & Sons, Ltd.     

Parameter estimation of subsurface flow models using iterative regularized ensemble Kalman filter

A new parameter estimation algorithm based on ensemble Kalman filter (EnKF) is developed. The developed algorithm combined with the proposed problem parametrization offers an efficient parameter estimation method that converges using very small ensembles. The inverse problem is formulated as a sequential data integration problem. Gaussian process regression is used to integrate the prior knowledge (static data). The search space is further parameterized using Karhunen–Loève expansion to build a set of basis functions that spans the search space. Optimal weights of the reduced basis functions are estimated by an iterative regularized EnKF algorithm. The filter is converted to an optimization algorithm by using a pseudo time-stepping technique such that the model output matches the time dependent data. The EnKF Kalman gain matrix is regularized using truncated SVD to filter out noisy correlations. Numerical results show that the proposed algorithm is a promising approach for parameter estimation of subsurface flow models.     

Data assimilation in groundwater modelling: ensemble Kalman filter versus ensemble smoothers

Groundwater modelling calls for an effective and robust data integrating method to fill the gap between the model and observation data. The ensemble Kalman filter (EnKF), a real‐time data assimilation method, has been increasingly applied in multiple disciplines such as petroleum engineering and hydrogeology. In this approach, a groundwater model is updated sequentially with measured data such as hydraulic head and concentration. As an alternative to the EnKF, the ensemble smoother (ES) has been proposed for updating groundwater models using all the data together, with much less computational cost. To further improve the performance of the ES, an iterative ES has been proposed for continuously updating the model by assimilating measurements together. In this work, we compare the performance of the EnKF, the ES, and the iterative ES using a synthetic example in groundwater modelling. Hydraulic head data modelled on the basis of the reference conductivity field are used to inversely estimate conductivities at unsampled locations. Results are evaluated in terms of the characterization of conductivity and groundwater flow predictions. It is concluded that (a) the iterative ES works better than the standard ES because of its continuous updating and (b) the iterative ES could achieve results comparable with those of the EnKF, with less computational cost. These findings show that the iterative ES should be paid much more attention for data assimilation in groundwater modelling.     

An application of Ensemble Kalman Filter in integral-balance subsurface modeling

Data assimilation method provides a framework to decrease the uncertainty of hydrological modeling by sequentially incorporating observations into numerical model. Such a process involves estimating statistical moments of different order based on the evolution of conditional probability distribution function. Because of the nonlinearity of many hydrological dynamics, explicit and analytical solutions for moments of state distribution are often impossible. Evensen [J Geophys Res 99(c5): 10143–10162 (1994)] introduced Ensemble Kalman Filtering (EnKF) method to address such problems. We test and evaluate the performance of EnKF in fusing model predictions and observations for a saturated–unsaturated integral-balance subsurface model. We find EnKF improve the model predictions, and also we conclude a good estimate of state variance is essential for the success of EnKF.     

Assimilating transient groundwater flow data via a localized ensemble Kalman filter to calibrate a heterogeneous conductivity field

A localized ensemble Kalman filter (EnKF) method is developed to assimilate transient flow data to calibrate a heterogeneous conductivity field. To update conductivity value at a point in a study domain, instead of assimilating all the measurements in the study domain, only limited measurement data in an area around the point are used for the conductivity updating in the localized EnKF method. The localized EnKF is proposed to solve the problems of the filter divergence usually existing in a data assimilation method without localization. The developed method is applied, in a synthetical two dimensional case, to calibrate a heterogeneous conductivity field by assimilating transient hydraulic head data. The simulations by the data assimilation with and without localized EnKF are compared. The study results indicate that the hydraulic conductivity field can be updated efficiently by the localized EnKF, while it cannot be by the EnKF. The covariance inflation and localization are found to solve the problem of the filter divergence efficiently. In comparison with the EnKF method without localization, the localized EnKF method needs smaller ensemble size to achieve stabilized results. The simulation results by the localized EnKF method are much more sensitive to conductivity correlation length than to the localization radius. The developed localized EnKF method provides an approach to improve EnKF method in conductivity calibration.     

Data assimilation for distributed hydrological catchment modeling via ensemble Kalman filter

Catchment scale hydrological models are critical decision support tools for water resources management and environment remediation. However, the reliability of hydrological models is inevitably affected by limited measurements and imperfect models. Data assimilation techniques combine complementary information from measurements and models to enhance the model reliability and reduce predictive uncertainties. As a sequential data assimilation technique, the ensemble Kalman filter (EnKF) has been extensively studied in the earth sciences for assimilating in-situ measurements and remote sensing data. Although the EnKF has been demonstrated in land surface data assimilations, there are no systematic studies to investigate its performance in distributed modeling with high dimensional states and parameters. In this paper, we present an assessment on the EnKF with state augmentation for combined state-parameter estimation on the basis of a physical-based hydrological model, Soil and Water Assessment Tool (SWAT). Through synthetic simulation experiments, the capability of the EnKF is demonstrated by assimilating the runoff and other measurements, and its sensitivities are analyzed with respect to the error specification, the initial realization and the ensemble size. It is found that the EnKF provides an efficient approach for obtaining a set of acceptable model parameters and satisfactory runoff, soil water content and evapotranspiration estimations. The EnKF performance could be improved after augmenting with other complementary data, such as soil water content and evapotranspiration from remote sensing retrieval. Sensitivity studies demonstrate the importance of consistent error specification and the potential with small ensemble size in the data assimilation system.     

Comparison of Ensemble Kalman Filter groundwater-data assimilation methods based on stochastic moment equations and Monte Carlo simulation

Traditional Ensemble Kalman Filter (EnKF) data assimilation requires computationally intensive Monte Carlo (MC) sampling, which suffers from filter inbreeding unless the number of simulations is large. Recently we proposed an alternative EnKF groundwater-data assimilation method that obviates the need for sampling and is free of inbreeding issues. In our new approach, theoretical ensemble moments are approximated directly by solving a system of corresponding stochastic groundwater flow equations. Like MC-based EnKF, our moment equations (ME) approach allows Bayesian updating of system states and parameters in real-time as new data become available. Here we compare the performances and accuracies of the two approaches on two-dimensional transient groundwater flow toward a well pumping water in a synthetic, randomly heterogeneous confined aquifer subject to prescribed head and flux boundary conditions.     

Flow relevant covariance localization during dynamic data assimilation using EnKF

Multiphase dynamic data integration into high resolution subsurface models is an integral aspect of reservoir and groundwater management strategies and uncertainty assessment. Over the past two decades, advances in computing and the development and implementation of robust algorithms for automatic history matching have considerably reduced the time and effort associated with subsurface characterization and reduced the subjectivity associated with manual model calibration. However, reliable and accurate subsurface characterization continues to be challenging due to the large number of model unknowns to be estimated using a relatively smaller set of measurements. For ensemble-based methods in particular, the difficulties are compounded by the need for a large number of model replicates to estimate sample-based statistical measures, specifically the covariances and cross-covariances that directly impact the spread of information from the measurement locations to the model parameters. Statistical noise resulting from modest ensemble sizes can overwhelm and degrade the model updates leading to geologically inconsistent subsurface models. In this work we propose to address the difficulties in the implementation of the ensemble Kalman filter (EnKF) for operational data integration problems. The methods described here use streamline-derived information to identify regions within the reservoir that will have a maximum impact on the dynamic response. This is achieved through spatial localization of the sample-based cross-covariance estimates between the measurements and the model unknowns using streamline trajectories. We illustrate the approach with a synthetic example and a large field-study that demonstrate the difficulties with the traditional EnKF implementation. In both the numerical experiments, it is shown that these challenges are addressed using flow relevant conditioning of the cross-covariance matrix. By mitigating sampling error in the cross-covariance estimates, the proposed approach provides significant computational savings through the use of modest ensemble sizes, and consequently offers the opportunity for use with large field-scale groundwater and reservoir characterization studies.     

Application of data assimilation for improving forecast of water levels and residual currents in Singapore regional waters

Hydrodynamic models are commonly used for predicting water levels and currents in the deep ocean, ocean margins and shelf seas. Their accuracy is typically limited by factors, such as the complexity of the coastal geometry and bathymetry, plus the uncertainty in the flow forcing (deep ocean tide, winds and pressure). In Southeast Asian waters with its strongly hydrodynamic characteristics, the lack of detailed marine observations (bathymetry and tides) for model validation is an additional factor limiting flow representation. This paper deals with the application of ensemble Kalman filter (EnKF)-based data assimilation with the purpose of improving the deterministic model forecast. The efficacy of the EnKF is analysed via a twin experiment conducted with the 2D barotropic Singapore regional model. The results show that the applied data assimilation can improve the forecasts significantly in this complex flow regime.