Hydrological data assimilation with the ensemble Kalman filter: Use of streamflow observations to update states in a distributed hydrological model

This paper describes an application of the ensemble Kalman filter (EnKF) in which streamflow observations are used to update states in a distributed hydrological model. We demonstrate that the standard implementation of the EnKF is inappropriate because of non-linear relationships between model states and observations. Transforming streamflow into log space before computing error covariances improves filter performance. We also demonstrate that model simulations improve when we use a variant of the EnKF that does not require perturbed observations. Our attempt to propagate information to neighbouring basins was unsuccessful, largely due to inadequacies in modelling the spatial variability of hydrological processes. New methods are needed to produce ensemble simulations that both reflect total model error and adequately simulate the spatial variability of hydrological states and fluxes.     

Sampling strategies and square root analysis schemes for the EnKF

The purpose of this paper is to examine how different sampling strategies and implementations of the analysis scheme influence the quality of the results in the EnKF. It is shown that by selecting the initial ensemble, the model noise and the measurement perturbations wisely, it is possible to achieve a significant improvement in the EnKF results, using the same number of members in the ensemble. The results are also compared with a square root implementation of the EnKF analysis scheme where the analyzed ensemble is computed without the perturbation of measurements. It is shown that the measurement perturbations introduce sampling errors which can be reduced using improved sampling schemes in the standard EnKF or fully eliminated when the square root analysis algorithm is used. Further, a new computationally efficient square root algorithm is proposed which allows for the use of a low-rank representation of the measurement error covariance matrix. It is shown that this algorithm in fact solves the full problem at a low cost without introducing any new approximations.     

Data assimilation with the EnKF in a 1-D numerical model of a North Sea station

A series of numerical experiments for data assimilation with the Ensemble Kalman Filter (EnKF) in a shallow water model are reported. Temperature profiles measured at a North Sea location, 55°30ˊ North and 0°55ˊ East (referred to as the CS station of the NERC North Sea project), are assimilated in 1-D simulations. Comparison of simulations without assimilation to model results obtained when assimilating data with the EnKF allows us to assess the filter performance in reproducing features of the observations not accounted for by the model. The quality of the model error sampling is tested as well as the validity of the Gaussian hypothesis underlying the analysis scheme of the EnKF. The influence of the model error parameters and the frequency of the data assimilation are investigated and discussed. From these experiments, a set of optimal parameters for the model error sampling are deduced and used to test the behavior of the EnKF when propagating surface information into the water column.     

The Ensemble Kalman Filter for Groundwater Plume Characterization: A Case Study

The Kalman filter is an efficient data assimilation tool to refine an estimate of a state variable using measured data and the variable's correlations in space and/or time. The ensemble Kalman filter (EnKF) (Evensen 2004, 2009) is a Kalman filter variant that employs Monte Carlo analysis to define the correlations that help to refine the updated state. While use of EnKF in hydrology is somewhat limited, it has been successfully applied in other fields of engineering (e.g., oil reservoir modeling, weather forecasting). Here, EnKF is used to refine a simulated groundwater tetrachloroethylene (TCE) plume that underlies the Tooele Army Depot‐North (TEAD‐N) in Utah, based on observations of TCE in the aquifer. The resulting EnKF‐based assimilated plume is simulated forward in time to predict future plume migration. The correlations that underpin EnKF updating implicitly contain information about how the plume developed over time under the influence of complex site hydrology and variable source history, as they are predicated on multiple realizations of a well‐calibrated numerical groundwater flow and transport model. The EnKF methodology is compared to an ordinary kriging‐based assimilation method with respect to the accurate representation of plume concentrations in order to determine the relative efficacy of EnKF for water quality data assimilation.     

Dual state–parameter estimation of hydrological models using ensemble Kalman filter

Hydrologic models are twofold: models for understanding physical processes and models for prediction. This study addresses the latter, which modelers use to predict, for example, streamflow at some future time given knowledge of the current state of the system and model parameters. In this respect, good estimates of the parameters and state variables are needed to enable the model to generate accurate forecasts. In this paper, a dual state–parameter estimation approach is presented based on the Ensemble Kalman Filter (EnKF) for sequential estimation of both parameters and state variables of a hydrologic model. A systematic approach for identification of the perturbation factors used for ensemble generation and for selection of ensemble size is discussed. The dual EnKF methodology introduces a number of novel features: (1) both model states and parameters can be estimated simultaneously; (2) the algorithm is recursive and therefore does not require storage of all past information, as is the case in the batch calibration procedures; and (3) the various sources of uncertainties can be properly addressed, including input, output, and parameter uncertainties. The applicability and usefulness of the dual EnKF approach for ensemble streamflow forecasting is demonstrated using a conceptual rainfall-runoff model.     

Impacts of localisation in the EnKF and EnOI: experiments with a small model

The performance of an inexpensive, ensemble-based optimal interpolation (EnOI) scheme that uses a stationary ensemble of model anomalies to approximate forecast error covariances, is compared with that of an ensemble Kalman filter (EnKF). The model to which the methods are applied is a pair of “perfect”, one-dimensional, linear advection equations for two related variables. While EnOI is sub-optimal, it can give results that are comparable to those of the EnKF. The computational cost of EnOI is typically about times less than that of EnKF, where is the ensemble size. We suggest that EnOI may provide a practical and cost-effective alternative to the EnKF for some applications where computational cost is a limiting factor. We demonstrate that when the ensemble size is smaller than the dimension of the model’s sub-space, both the EnKF and EnOI may require localisation around each observation to eliminate effects of sampling error and to increase the effective number of independent ensemble members used to construct an analysis. However, localisation can degrade an analysis if the length-scales of the localising function are too short. We demonstrate that, as the length-scale of the localising function is decreased, localisation can significantly compromise the model’s dynamical balances. We also find that localisation artificially amplifies high frequencies for applications of the EnKF. Based on our experiments, for applications where localisation is necessary, the length-scales of the localisation should be larger than the decorrelation length-scales of the variables being updated.     

Calibration of a Land Subsidence Model Using InSAR Data via the Ensemble Kalman Filter

The application of interferometric synthetic aperture radar (InSAR) has been increasingly used to improve capabilities to model land subsidence in hydrogeologic studies. A number of investigations over the last decade show how spatially detailed time‐lapse images of ground displacements could be utilized to advance our understanding for better predictions. In this work, we use simulated land subsidences as observed measurements, mimicking InSAR data to inversely infer inelastic specific storage in a stochastic framework. The inelastic specific storage is assumed as a random variable and modeled using a geostatistical method such that the detailed variations in space could be represented and also that the uncertainties of both characterization of specific storage and prediction of land subsidence can be assessed. The ensemble Kalman filter (EnKF), a real‐time data assimilation algorithm, is used to inversely calibrate a land subsidence model by matching simulated subsidences with InSAR data. The performance of the EnKF is demonstrated in a synthetic example in which simulated surface deformations using a reference field are assumed as InSAR data for inverse modeling. The results indicate: (1) the EnKF can be used successfully to calibrate a land subsidence model with InSAR data; the estimation of inelastic specific storage is improved, and uncertainty of prediction is reduced, when all the data are accounted for; and (2) if the same ensemble is used to estimate Kalman gain, the analysis errors could cause filter divergence; thus, it is essential to include localization in the EnKF for InSAR data assimilation.     

Flow relevant covariance localization during dynamic data assimilation using EnKF

Multiphase dynamic data integration into high resolution subsurface models is an integral aspect of reservoir and groundwater management strategies and uncertainty assessment. Over the past two decades, advances in computing and the development and implementation of robust algorithms for automatic history matching have considerably reduced the time and effort associated with subsurface characterization and reduced the subjectivity associated with manual model calibration. However, reliable and accurate subsurface characterization continues to be challenging due to the large number of model unknowns to be estimated using a relatively smaller set of measurements. For ensemble-based methods in particular, the difficulties are compounded by the need for a large number of model replicates to estimate sample-based statistical measures, specifically the covariances and cross-covariances that directly impact the spread of information from the measurement locations to the model parameters. Statistical noise resulting from modest ensemble sizes can overwhelm and degrade the model updates leading to geologically inconsistent subsurface models. In this work we propose to address the difficulties in the implementation of the ensemble Kalman filter (EnKF) for operational data integration problems. The methods described here use streamline-derived information to identify regions within the reservoir that will have a maximum impact on the dynamic response. This is achieved through spatial localization of the sample-based cross-covariance estimates between the measurements and the model unknowns using streamline trajectories. We illustrate the approach with a synthetic example and a large field-study that demonstrate the difficulties with the traditional EnKF implementation. In both the numerical experiments, it is shown that these challenges are addressed using flow relevant conditioning of the cross-covariance matrix. By mitigating sampling error in the cross-covariance estimates, the proposed approach provides significant computational savings through the use of modest ensemble sizes, and consequently offers the opportunity for use with large field-scale groundwater and reservoir characterization studies.     

On ensemble representation of the observation-error covariance in the Ensemble Kalman Filter

Evensen (2003) presents a modification of the Ensemble Kalman Filter (EnKF), in which the observation-error and background-error covariance matrices are both represented by ensembles, in contrast to the usual practice, where only the background error is so represented. It is shown that this modification can cause the ensemble to collapse to a single member, in the common situation where the number of observations is more than twice the number of ensemble members, and to be rank-deficient when the number of observations is greater than or equal to the ensemble size. It is also shown that some further modifications to the scheme, presented by Evensen as offering numerical efficiencies, can prevent this collapse. However, these latter modifications are shown in some simple numerical examples to require tuning to produce acceptable results, which are nevertheless inferior to those of the standard EnKF.Acknowledgements The author acknowledges useful discussions with Peter Steinle, and other participants at the EnKF workshop held in BMRC in November, 2003.     

Data assimilation for unsaturated flow models with restart adaptive probabilistic collocation based Kalman filter

The ensemble Kalman filter (EnKF) has gained popularity in hydrological data assimilation problems. As a Monte Carlo based method, a sufficiently large ensemble size is usually required to guarantee the accuracy. As an alternative approach, the probabilistic collocation based Kalman filter (PCKF) employs the polynomial chaos expansion (PCE) to represent and propagate the uncertainties in parameters and states. However, PCKF suffers from the so-called “curse of dimensionality”. Its computational cost increases drastically with the increasing number of parameters and system nonlinearity. Furthermore, PCKF may fail to provide accurate estimations due to the joint updating scheme for strongly nonlinear models. Motivated by recent developments in uncertainty quantification and EnKF, we propose a restart adaptive probabilistic collocation based Kalman filter (RAPCKF) for data assimilation in unsaturated flow problems. During the implementation of RAPCKF, the important parameters are identified and active PCE basis functions are adaptively selected at each assimilation step; the “restart” scheme is utilized to eliminate the inconsistency between updated model parameters and states variables. The performance of RAPCKF is systematically tested with numerical cases of unsaturated flow models. It is shown that the adaptive approach and restart scheme can significantly improve the performance of PCKF. Moreover, RAPCKF has been demonstrated to be more efficient than EnKF with the same computational cost.     

An approach to handling non-Gaussianity of parameters and state variables in ensemble Kalman filtering

The ensemble Kalman filter (EnKF) is a commonly used real-time data assimilation algorithm in various disciplines. Here, the EnKF is applied, in a hydrogeological context, to condition log-conductivity realizations on log-conductivity and transient piezometric head data. In this case, the state vector is made up of log-conductivities and piezometric heads over a discretized aquifer domain, the forecast model is a groundwater flow numerical model, and the transient piezometric head data are sequentially assimilated to update the state vector. It is well known that all Kalman filters perform optimally for linear forecast models and a multiGaussian-distributed state vector. Of the different Kalman filters, the EnKF provides a robust solution to address non-linearities; however, it does not handle well non-Gaussian state-vector distributions. In the standard EnKF, as time passes and more state observations are assimilated, the distributions become closer to Gaussian, even if the initial ones are clearly non-Gaussian. A new method is proposed that transforms the original state vector into a new vector that is univariate Gaussian at all times. Back transforming the vector after the filtering ensures that the initial non-Gaussian univariate distributions of the state-vector components are preserved throughout. The proposed method is based in normal-score transforming each variable for all locations and all time steps. This new method, termed the normal-score ensemble Kalman filter (NS-EnKF), is demonstrated in a synthetic bimodal aquifer resembling a fluvial deposit, and it is compared to the standard EnKF. The proposed method performs better than the standard EnKF in all aspects analyzed (log-conductivity characterization and flow and transport predictions).     

Comparison of Ensemble Kalman Filter groundwater-data assimilation methods based on stochastic moment equations and Monte Carlo simulation

Traditional Ensemble Kalman Filter (EnKF) data assimilation requires computationally intensive Monte Carlo (MC) sampling, which suffers from filter inbreeding unless the number of simulations is large. Recently we proposed an alternative EnKF groundwater-data assimilation method that obviates the need for sampling and is free of inbreeding issues. In our new approach, theoretical ensemble moments are approximated directly by solving a system of corresponding stochastic groundwater flow equations. Like MC-based EnKF, our moment equations (ME) approach allows Bayesian updating of system states and parameters in real-time as new data become available. Here we compare the performances and accuracies of the two approaches on two-dimensional transient groundwater flow toward a well pumping water in a synthetic, randomly heterogeneous confined aquifer subject to prescribed head and flux boundary conditions.     

Reduction of Used Memory Ensemble Kalman Filtering (RumEnKF): A data assimilation scheme for memory intensive,high performance computing

Reduction of Used Memory Ensemble Kalman Filtering (RumEnKF) is introduced as a variant on the Ensemble Kalman Filter (EnKF). RumEnKF differs from EnKF in that it does not store the entire ensemble, but rather only saves the first two moments of the ensemble distribution. In this way, the number of ensemble members that can be calculated is less dependent on available memory, and mainly on available computing power (CPU). RumEnKF is developed to make optimal use of current generation super computer architecture, where the number of available floating point operations (flops) increases more rapidly than the available memory and where inter-node communication can quickly become a bottleneck. RumEnKF reduces the used memory compared to the EnKF when the number of ensemble members is greater than half the number of state variables. In this paper, three simple models are used (auto-regressive, low dimensional Lorenz and high dimensional Lorenz) to show that RumEnKF performs similarly to the EnKF. Furthermore, it is also shown that increasing the ensemble size has a similar impact on the estimation error from the three algorithms.     

An application of Ensemble Kalman Filter in integral-balance subsurface modeling

Data assimilation method provides a framework to decrease the uncertainty of hydrological modeling by sequentially incorporating observations into numerical model. Such a process involves estimating statistical moments of different order based on the evolution of conditional probability distribution function. Because of the nonlinearity of many hydrological dynamics, explicit and analytical solutions for moments of state distribution are often impossible. Evensen [J Geophys Res 99(c5): 10143–10162 (1994)] introduced Ensemble Kalman Filtering (EnKF) method to address such problems. We test and evaluate the performance of EnKF in fusing model predictions and observations for a saturated–unsaturated integral-balance subsurface model. We find EnKF improve the model predictions, and also we conclude a good estimate of state variance is essential for the success of EnKF.     

Assimilation of SLA and SST data into an OGCM for the Indian Ocean

 Remotely sensed observations of sea-level anomaly and sea-surface temperature have been assimilated into an implementation of the Miami Isopycnic Coordinate Ocean Model (MICOM) for the Indian Ocean using the Ensemble Kalman Filter (EnKF). The system has been applied in a hindcast validation experiment to examine the properties of the assimilation scheme when used with a full ocean general circulation model and real observations. This work is considered as a first step towards an operational ocean monitoring and forecasting system for the Indian Ocean. The assimilation of real data has demonstrated that the sequential EnKF can efficiently control the model evolution in time. The use of data assimilation requires a significant amount of additional processing and computational resources. However, we have tried to justify the cost of using a sophisticated assimilation scheme by demonstrating strong regional and temporal dependencies of the covariance statistics, which include highly anisotropic and flow-dependent correlation functions. In particular, we observed a marked difference between error statistics in the equatorial region and at off-equatorial latitudes. We have also demonstrated how the assimilation of SLA and SST improves the model fields with respect to real observations. Independent in situ temperature profiles have been used to examine the impact of assimilating the remotely sensed observations. These intercomparisons have shown that the model temperature and salinity fields better resemble in situ observations in the assimilation experiment than in a model free-run case. On the other hand, it is also expected that assimilation of in situ profiles is needed to properly control the deep ocean circulation. Received: 8 January 2002 / Accepted: 8 April 2002     

The Local Ensemble Transform Kalman Filter with the Weather Research and Forecasting Model: Experiments with Real Observations

The local ensemble transform Kalman filter (LETKF) is implemented with the Weather Research and Forecasting (WRF) model, and real observations are assimilated to assess the newly-developed WRF-LETKF system. The WRF model is a widely-used mesoscale numerical weather prediction model, and the LETKF is an ensemble Kalman filter (EnKF) algorithm particularly efficient in parallel computer architecture. This study aims to provide the basis of future research on mesoscale data assimilation using the WRF-LETKF system, an additional testbed to the existing EnKF systems with the WRF model used in the previous studies. The particular LETKF system adopted in this study is based on the system initially developed in 2004 and has been continuously improved through theoretical studies and wide applications to many kinds of dynamical models including realistic geophysical models. Most recent and important improvements include an adaptive covariance inflation scheme which considers the spatial and temporal inhomogeneity of inflation parameters. Experiments show that the LETKF successfully assimilates real observations and that adaptive inflation is advantageous. Additional experiments with various ensemble sizes show that using more ensemble members improves the analyses consistently.     

An effective method based on dynamic sampling for data assimilation in a global wave model

The ensemble Kalman filter (EnKF) performs well because that the covariance of background error is varying along time. It provides a dynamic estimate of background error and represents the reasonable statistic characters of background error. However, high computational cost due to model ensemble in EnKF is employed. In this study, two methods referred as static and dynamic sampling methods are proposed to obtain a good performance and reduce the computation cost. Ensemble adjustment Kalman filter (EAKF) method is used in a global surface wave model to examine the performance of EnKF. The 24-h interval difference of simulated significant wave height (SWH) within 1 year is used to compose the static samples for ensemble errors, and these errors are used to construct the ensemble states at each time the observations are available. And then, the same method of updating the model states in the EAKF is applied for the ensemble states constructed by a static sampling method. The dynamic sampling method employs a similar method to construct the ensemble states, but the period of the simulated SWH is changing with time. Here, 7 days before and after the observation time is used as this period. To examine the performance of three schemes, EAKF, static, or dynamic sampling method, observations from satellite Jason-2 in 2014 are assimilated into a global wave model, and observations from satellite Saral are used for validation. The results indicate that the EAKF performs best, while the static sampling method is relatively worse. The dynamic sampling method improves an assimilation effect dramatically compared to the static sampling method, and its overall performance is closed to the EAKF. In low latitudes, the dynamic sampling method has a slight advantage over the EAKF. In the dynamic or static sampling methods, only one wave model is required to run and their computational cost is reduced sharply. According to the performance of these three methods, the dynamic sampling method can treated as an effective alternative of EnKF, which could reduce the computational cost and provide a good performance of data assimilation.     

The role of prior model calibration on predictions with ensemble Kalman filter

This paper, based on a real world case study (Limmat aquifer, Switzerland), compares inverse groundwater flow models calibrated with specified numbers of monitoring head locations. These models are updated in real time with the ensemble Kalman filter (EnKF) and the prediction improvement is assessed in relation to the amount of monitoring locations used for calibration and updating. The prediction errors of the models calibrated in transient state are smaller if the amount of monitoring locations used for the calibration is larger. For highly dynamic groundwater flow systems a transient calibration is recommended as a model calibrated in steady state can lead to worse results than a noncalibrated model with a well-chosen uniform conductivity. The model predictions can be improved further with the assimilation of new measurement data from on-line sensors with the EnKF. Within all the studied models the reduction of 1-day hydraulic head prediction error (in terms of mean absolute error [MAE]) with EnKF lies between 31% (assimilation of head data from 5 locations) and 72% (assimilation of head data from 85 locations). The largest prediction improvements are expected for models that were calibrated with only a limited amount of historical information. It is worthwhile to update the model even with few monitoring locations as it seems that the error reduction with EnKF decreases exponentially with the amount of monitoring locations used. These results prove the feasibility of data assimilation with EnKF also for a real world case and show that improved predictions of groundwater levels can be obtained.     

Dual states estimation of a subsurface flow-transport coupled model using ensemble Kalman filtering

Modeling the spread of subsurface contaminants requires coupling a groundwater flow model with a contaminant transport model. Such coupling may provide accurate estimates of future subsurface hydrologic states if essential flow and contaminant data are assimilated in the model. Assuming perfect flow, an ensemble Kalman filter (EnKF) can be used for direct data assimilation into the transport model. This is, however, a crude assumption as flow models can be subject to many sources of uncertainty. If the flow is not accurately simulated, contaminant predictions will likely be inaccurate even after successive Kalman updates of the contaminant model with the data. The problem is better handled when both flow and contaminant states are concurrently estimated using the traditional joint state augmentation approach. In this paper, we introduce a dual estimation strategy for data assimilation into a one-way coupled system by treating the flow and the contaminant models separately while intertwining a pair of distinct EnKFs, one for each model. The presented strategy only deals with the estimation of state variables but it can also be used for state and parameter estimation problems. This EnKF-based dual state-state estimation procedure presents a number of novel features: (i) it allows for simultaneous estimation of both flow and contaminant states in parallel; (ii) it provides a time consistent sequential updating scheme between the two models (first flow, then transport); (iii) it simplifies the implementation of the filtering system; and (iv) it yields more stable and accurate solutions than does the standard joint approach. We conducted synthetic numerical experiments based on various time stepping and observation strategies to evaluate the dual EnKF approach and compare its performance with the joint state augmentation approach. Experimental results show that on average, the dual strategy could reduce the estimation error of the coupled states by 15% compared with the joint approach. Furthermore, the dual estimation is proven to be very effective computationally, recovering accurate estimates at a reasonable cost.