The Local Ensemble Transform Kalman Filter with the Weather Research and Forecasting Model: Experiments with Real Observations

The local ensemble transform Kalman filter (LETKF) is implemented with the Weather Research and Forecasting (WRF) model, and real observations are assimilated to assess the newly-developed WRF-LETKF system. The WRF model is a widely-used mesoscale numerical weather prediction model, and the LETKF is an ensemble Kalman filter (EnKF) algorithm particularly efficient in parallel computer architecture. This study aims to provide the basis of future research on mesoscale data assimilation using the WRF-LETKF system, an additional testbed to the existing EnKF systems with the WRF model used in the previous studies. The particular LETKF system adopted in this study is based on the system initially developed in 2004 and has been continuously improved through theoretical studies and wide applications to many kinds of dynamical models including realistic geophysical models. Most recent and important improvements include an adaptive covariance inflation scheme which considers the spatial and temporal inhomogeneity of inflation parameters. Experiments show that the LETKF successfully assimilates real observations and that adaptive inflation is advantageous. Additional experiments with various ensemble sizes show that using more ensemble members improves the analyses consistently.     

Real-time assimilation of streamflow observations into a hydrological routing model: effects of model structures and updating methods

This paper comparatively assesses the performance of five data assimilation techniques for three-parameter Muskingum routing with a spatially lumped or distributed model structure. The assimilation techniques used include direct insertion (DI), nudging scheme (NS), Kalman filter (KF), ensemble Kalman filter (EnKF) and asynchronous ensemble Kalman filter (AEnKF), which are applied to river reaches in Texas and Louisiana, USA. For both lumped and distributed routing, results from KF, EnKF and AEnKF are sensitive to the error specification. As expected, DI outperformed the other models in the case of lumped modelling, while in distributed routing, KF approaches, particularly AEnKF and EnKF, performed better than DI or nudging, reflecting the benefit of updating distributed states through error covariance modelling in KF approaches. The results of this work would be useful in setting up data assimilation systems that employ increasingly abundant real-time observations using distributed hydrological routing models.     

An effective method based on dynamic sampling for data assimilation in a global wave model

The ensemble Kalman filter (EnKF) performs well because that the covariance of background error is varying along time. It provides a dynamic estimate of background error and represents the reasonable statistic characters of background error. However, high computational cost due to model ensemble in EnKF is employed. In this study, two methods referred as static and dynamic sampling methods are proposed to obtain a good performance and reduce the computation cost. Ensemble adjustment Kalman filter (EAKF) method is used in a global surface wave model to examine the performance of EnKF. The 24-h interval difference of simulated significant wave height (SWH) within 1 year is used to compose the static samples for ensemble errors, and these errors are used to construct the ensemble states at each time the observations are available. And then, the same method of updating the model states in the EAKF is applied for the ensemble states constructed by a static sampling method. The dynamic sampling method employs a similar method to construct the ensemble states, but the period of the simulated SWH is changing with time. Here, 7 days before and after the observation time is used as this period. To examine the performance of three schemes, EAKF, static, or dynamic sampling method, observations from satellite Jason-2 in 2014 are assimilated into a global wave model, and observations from satellite Saral are used for validation. The results indicate that the EAKF performs best, while the static sampling method is relatively worse. The dynamic sampling method improves an assimilation effect dramatically compared to the static sampling method, and its overall performance is closed to the EAKF. In low latitudes, the dynamic sampling method has a slight advantage over the EAKF. In the dynamic or static sampling methods, only one wave model is required to run and their computational cost is reduced sharply. According to the performance of these three methods, the dynamic sampling method can treated as an effective alternative of EnKF, which could reduce the computational cost and provide a good performance of data assimilation.     

Subsurface characterization with localized ensemble Kalman filter employing adaptive thresholding

Ensemble Kalman filter, EnKF, as a Monte Carlo sequential data assimilation method has emerged promisingly for subsurface media characterization during past decade. Due to high computational cost of large ensemble size, EnKF is limited to small ensemble set in practice. This results in appearance of spurious correlation in covariance structure leading to incorrect or probable divergence of updated realizations. In this paper, a universal/adaptive thresholding method is presented to remove and/or mitigate spurious correlation problem in the forecast covariance matrix. This method is, then, extended to regularize Kalman gain directly. Four different thresholding functions have been considered to threshold forecast covariance and gain matrices. These include hard, soft, lasso and Smoothly Clipped Absolute Deviation (SCAD) functions. Three benchmarks are used to evaluate the performances of these methods. These benchmarks include a small 1D linear model and two 2D water flooding (in petroleum reservoirs) cases whose levels of heterogeneity/nonlinearity are different. It should be noted that beside the adaptive thresholding, the standard distance dependant localization and bootstrap Kalman gain are also implemented for comparison purposes. We assessed each setup with different ensemble sets to investigate the sensitivity of each method on ensemble size. The results indicate that thresholding of forecast covariance yields more reliable performance than Kalman gain. Among thresholding function, SCAD is more robust for both covariance and gain estimation. Our analyses emphasize that not all assimilation cycles do require thresholding and it should be performed wisely during the early assimilation cycles. The proposed scheme of adaptive thresholding outperforms other methods for subsurface characterization of underlying benchmarks.     

An adaptive hybrid EnKF-OI scheme for efficient state-parameter estimation of reactive contaminant transport models

Reactive contaminant transport models are used by hydrologists to simulate and study the migration and fate of industrial waste in subsurface aquifers. Accurate transport modeling of such waste requires clear understanding of the system’s parameters, such as sorption and biodegradation. In this study, we present an efficient sequential data assimilation scheme that computes accurate estimates of aquifer contamination and spatially variable sorption coefficients. This assimilation scheme is based on a hybrid formulation of the ensemble Kalman filter (EnKF) and optimal interpolation (OI) in which solute concentration measurements are assimilated via a recursive dual estimation of sorption coefficients and contaminant state variables. This hybrid EnKF-OI scheme is used to mitigate background covariance limitations due to ensemble under-sampling and neglected model errors. Numerical experiments are conducted with a two-dimensional synthetic aquifer in which cobalt-60, a radioactive contaminant, is leached in a saturated heterogeneous clayey sandstone zone. Assimilation experiments are investigated under different settings and sources of model and observational errors. Simulation results demonstrate that the proposed hybrid EnKF-OI scheme successfully recovers both the contaminant and the sorption rate and reduces their uncertainties. Sensitivity analyses also suggest that the adaptive hybrid scheme remains effective with small ensembles, allowing to reduce the ensemble size by up to 80% with respect to the standard EnKF scheme.     

Snow water equivalent prediction using Bayesian data assimilation methods

Using the U.S. National Weather Service’s SNOW-17 model, this study compares common sequential data assimilation methods, the ensemble Kalman filter (EnKF), the ensemble square root filter (EnSRF), and four variants of the particle filter (PF), to predict seasonal snow water equivalent (SWE) within a small watershed near Lake Tahoe, California. In addition to SWE estimation, the various data assimilation methods are used to estimate five of the most sensitive parameters of SNOW-17 by allowing them to evolve with the dynamical system. Unlike Kalman filters, particle filters do not require Gaussian assumptions for the posterior distribution of the state variables. However, the likelihood function used to scale particle weights is often assumed to be Gaussian. This study evaluates the use of an empirical cumulative distribution function (ECDF) based on the Kaplan–Meier survival probability method to compute particle weights. These weights are then used in different particle filter resampling schemes. Detailed analyses are conducted for synthetic and real data assimilation and an assessment of the procedures is made. The results suggest that the particle filter, especially the empirical likelihood variant, is superior to the ensemble Kalman filter based methods for predicting model states, as well as model parameters.     

Comparison of Ensemble Kalman Filter groundwater-data assimilation methods based on stochastic moment equations and Monte Carlo simulation

Traditional Ensemble Kalman Filter (EnKF) data assimilation requires computationally intensive Monte Carlo (MC) sampling, which suffers from filter inbreeding unless the number of simulations is large. Recently we proposed an alternative EnKF groundwater-data assimilation method that obviates the need for sampling and is free of inbreeding issues. In our new approach, theoretical ensemble moments are approximated directly by solving a system of corresponding stochastic groundwater flow equations. Like MC-based EnKF, our moment equations (ME) approach allows Bayesian updating of system states and parameters in real-time as new data become available. Here we compare the performances and accuracies of the two approaches on two-dimensional transient groundwater flow toward a well pumping water in a synthetic, randomly heterogeneous confined aquifer subject to prescribed head and flux boundary conditions.     

The Ensemble Kalman Filter for Groundwater Plume Characterization: A Case Study

The Kalman filter is an efficient data assimilation tool to refine an estimate of a state variable using measured data and the variable's correlations in space and/or time. The ensemble Kalman filter (EnKF) (Evensen 2004, 2009) is a Kalman filter variant that employs Monte Carlo analysis to define the correlations that help to refine the updated state. While use of EnKF in hydrology is somewhat limited, it has been successfully applied in other fields of engineering (e.g., oil reservoir modeling, weather forecasting). Here, EnKF is used to refine a simulated groundwater tetrachloroethylene (TCE) plume that underlies the Tooele Army Depot‐North (TEAD‐N) in Utah, based on observations of TCE in the aquifer. The resulting EnKF‐based assimilated plume is simulated forward in time to predict future plume migration. The correlations that underpin EnKF updating implicitly contain information about how the plume developed over time under the influence of complex site hydrology and variable source history, as they are predicated on multiple realizations of a well‐calibrated numerical groundwater flow and transport model. The EnKF methodology is compared to an ordinary kriging‐based assimilation method with respect to the accurate representation of plume concentrations in order to determine the relative efficacy of EnKF for water quality data assimilation.     

Data assimilation for distributed hydrological catchment modeling via ensemble Kalman filter

Catchment scale hydrological models are critical decision support tools for water resources management and environment remediation. However, the reliability of hydrological models is inevitably affected by limited measurements and imperfect models. Data assimilation techniques combine complementary information from measurements and models to enhance the model reliability and reduce predictive uncertainties. As a sequential data assimilation technique, the ensemble Kalman filter (EnKF) has been extensively studied in the earth sciences for assimilating in-situ measurements and remote sensing data. Although the EnKF has been demonstrated in land surface data assimilations, there are no systematic studies to investigate its performance in distributed modeling with high dimensional states and parameters. In this paper, we present an assessment on the EnKF with state augmentation for combined state-parameter estimation on the basis of a physical-based hydrological model, Soil and Water Assessment Tool (SWAT). Through synthetic simulation experiments, the capability of the EnKF is demonstrated by assimilating the runoff and other measurements, and its sensitivities are analyzed with respect to the error specification, the initial realization and the ensemble size. It is found that the EnKF provides an efficient approach for obtaining a set of acceptable model parameters and satisfactory runoff, soil water content and evapotranspiration estimations. The EnKF performance could be improved after augmenting with other complementary data, such as soil water content and evapotranspiration from remote sensing retrieval. Sensitivity studies demonstrate the importance of consistent error specification and the potential with small ensemble size in the data assimilation system.     

Adaptive multi-fidelity probabilistic collocation-based Kalman filter for subsurface flow data assimilation: numerical modeling and real-world experiment

Stochastic Environmental Research and Risk Assessment - The ensemble Kalman filter (EnKF) has received substantial attention in hydrologic data assimilation due to its ease of implementation. In...     

Application of a data assimilation method via an ensemble Kalman filter to reactive urea hydrolysis transport modeling

With growing importance of water resources in the world, remediations of anthropogenic contaminations due to reactive solute transport become even more important. A good understanding of reactive rate parameters such as kinetic parameters is the key to accurately predicting reactive solute transport processes and designing corresponding remediation schemes. For modeling reactive solute transport, it is very difficult to estimate chemical reaction rate parameters due to complex processes of chemical reactions and limited available data. To find a method to get the reactive rate parameters for the reactive urea hydrolysis transport modeling and obtain more accurate prediction for the chemical concentrations, we developed a data assimilation method based on an ensemble Kalman filter (EnKF) method to calibrate reactive rate parameters for modeling urea hydrolysis transport in a synthetic one-dimensional column at laboratory scale and to update modeling prediction. We applied a constrained EnKF method to pose constraints to the updated reactive rate parameters and the predicted solute concentrations based on their physical meanings after the data assimilation calibration. From the study results we concluded that we could efficiently improve the chemical reactive rate parameters with the data assimilation method via the EnKF, and at the same time we could improve solute concentration prediction. The more data we assimilated, the more accurate the reactive rate parameters and concentration prediction. The filter divergence problem was also solved in this study.     

Evaluating the utility of the ensemble transform Kalman filter for adaptive sampling when updating a hydrodynamic model

This paper compares two Monte Carlo sequential data assimilation methods based on the Kalman filter, for estimating the effect of measurements on simulations of state error variance made by a one-dimensional hydrodynamic model. The first method used an ensemble Kalman filter (EnKF) to update state estimates, which were then used as initial conditions for further simulations. The second method used an ensemble transform Kalman filter (ETKF) to quickly estimate the effect of measurement error covariance on forecast error covariance without the need to re-run the simulation model. The ETKF gave an unbiased estimate of EnKF analysed error variance, although differences in the treatment of measurement errors meant the results were not identical. Estimates of forecast error variance could also be made, but their accuracy deteriorated as the time from measurements increased due in part to model non-linearity and the decreasing signal variance. The motivation behind the study was to assess the ability of the ETKF to target possible measurements, as part of an adaptive sampling framework, before they are assimilated by an EnKF-based forecasting model on the River Crouch, Essex, UK. The ETKF was found to be a useful tool for quickly estimating the error covariance expected after assimilating measurements into the hydrodynamic model. It, thus, provided a means of quantifying the ‘usefulness’ (in terms of error variance) of possible sampling schemes.     

Assimilating transient groundwater flow data via a localized ensemble Kalman filter to calibrate a heterogeneous conductivity field

A localized ensemble Kalman filter (EnKF) method is developed to assimilate transient flow data to calibrate a heterogeneous conductivity field. To update conductivity value at a point in a study domain, instead of assimilating all the measurements in the study domain, only limited measurement data in an area around the point are used for the conductivity updating in the localized EnKF method. The localized EnKF is proposed to solve the problems of the filter divergence usually existing in a data assimilation method without localization. The developed method is applied, in a synthetical two dimensional case, to calibrate a heterogeneous conductivity field by assimilating transient hydraulic head data. The simulations by the data assimilation with and without localized EnKF are compared. The study results indicate that the hydraulic conductivity field can be updated efficiently by the localized EnKF, while it cannot be by the EnKF. The covariance inflation and localization are found to solve the problem of the filter divergence efficiently. In comparison with the EnKF method without localization, the localized EnKF method needs smaller ensemble size to achieve stabilized results. The simulation results by the localized EnKF method are much more sensitive to conductivity correlation length than to the localization radius. The developed localized EnKF method provides an approach to improve EnKF method in conductivity calibration.     

A dual-pass data assimilation scheme for estimating surface fluxes with FY3A-VIRR land surface temperature

In this work, a dual-pass data assimilation scheme is developed to improve predictions of surface flux. Pass 1 of the dual-pass data assimilation scheme optimizes the model vegetation parameters at the weekly temporal scale, and Pass 2 optimizes the soil moisture at the daily temporal scale. Based on ensemble Kalman filter(EnKF), the land surface temperature(LST) data derived from the new generation of Chinese meteorology satellite(FY3A-VIRR) are assimilated into common land model(CoLM) for the first time. Six sites, Daman, Guantao, Arou, BJ, Miyun and Jiyuan, are selected for the data assimilation experiments and include different climatological conditions. The results are compared with those from a dataset generated by a multi-scale surface flux observation system that includes an automatic weather station(AWS), eddy covariance(EC) and large aperture scintillometer(LAS). The results indicate that the dual-pass data assimilation scheme is able to reduce model uncertainties and improve predictions of surface flux with the assimilation of FY3A-VIRR LST data.     

Data assimilation in groundwater modelling: ensemble Kalman filter versus ensemble smoothers

Groundwater modelling calls for an effective and robust data integrating method to fill the gap between the model and observation data. The ensemble Kalman filter (EnKF), a real‐time data assimilation method, has been increasingly applied in multiple disciplines such as petroleum engineering and hydrogeology. In this approach, a groundwater model is updated sequentially with measured data such as hydraulic head and concentration. As an alternative to the EnKF, the ensemble smoother (ES) has been proposed for updating groundwater models using all the data together, with much less computational cost. To further improve the performance of the ES, an iterative ES has been proposed for continuously updating the model by assimilating measurements together. In this work, we compare the performance of the EnKF, the ES, and the iterative ES using a synthetic example in groundwater modelling. Hydraulic head data modelled on the basis of the reference conductivity field are used to inversely estimate conductivities at unsampled locations. Results are evaluated in terms of the characterization of conductivity and groundwater flow predictions. It is concluded that (a) the iterative ES works better than the standard ES because of its continuous updating and (b) the iterative ES could achieve results comparable with those of the EnKF, with less computational cost. These findings show that the iterative ES should be paid much more attention for data assimilation in groundwater modelling.     

Hydrological data assimilation with the ensemble Kalman filter: Use of streamflow observations to update states in a distributed hydrological model

This paper describes an application of the ensemble Kalman filter (EnKF) in which streamflow observations are used to update states in a distributed hydrological model. We demonstrate that the standard implementation of the EnKF is inappropriate because of non-linear relationships between model states and observations. Transforming streamflow into log space before computing error covariances improves filter performance. We also demonstrate that model simulations improve when we use a variant of the EnKF that does not require perturbed observations. Our attempt to propagate information to neighbouring basins was unsuccessful, largely due to inadequacies in modelling the spatial variability of hydrological processes. New methods are needed to produce ensemble simulations that both reflect total model error and adequately simulate the spatial variability of hydrological states and fluxes.     

An approach to handling non-Gaussianity of parameters and state variables in ensemble Kalman filtering

The ensemble Kalman filter (EnKF) is a commonly used real-time data assimilation algorithm in various disciplines. Here, the EnKF is applied, in a hydrogeological context, to condition log-conductivity realizations on log-conductivity and transient piezometric head data. In this case, the state vector is made up of log-conductivities and piezometric heads over a discretized aquifer domain, the forecast model is a groundwater flow numerical model, and the transient piezometric head data are sequentially assimilated to update the state vector. It is well known that all Kalman filters perform optimally for linear forecast models and a multiGaussian-distributed state vector. Of the different Kalman filters, the EnKF provides a robust solution to address non-linearities; however, it does not handle well non-Gaussian state-vector distributions. In the standard EnKF, as time passes and more state observations are assimilated, the distributions become closer to Gaussian, even if the initial ones are clearly non-Gaussian. A new method is proposed that transforms the original state vector into a new vector that is univariate Gaussian at all times. Back transforming the vector after the filtering ensures that the initial non-Gaussian univariate distributions of the state-vector components are preserved throughout. The proposed method is based in normal-score transforming each variable for all locations and all time steps. This new method, termed the normal-score ensemble Kalman filter (NS-EnKF), is demonstrated in a synthetic bimodal aquifer resembling a fluvial deposit, and it is compared to the standard EnKF. The proposed method performs better than the standard EnKF in all aspects analyzed (log-conductivity characterization and flow and transport predictions).     

Using data assimilation method to calibrate a heterogeneous conductivity field conditioning on transient flow test data

A data assimilation method is developed to calibrate a heterogeneous hydraulic conductivity field conditioning on transient pumping test data. The ensemble Kalman filter (EnKF) approach is used to update model parameters such as hydraulic conductivity and model variables such as hydraulic head using available data. A synthetical two-dimensional flow case is used to assess the capability of the EnKF method to calibrate a heterogeneous conductivity field by assimilating transient flow data from observation wells under different hydraulic boundary conditions. The study results indicate that the EnKF method will significantly improve the estimation of the hydraulic conductivity field by assimilating continuous hydraulic head measurements and the hydraulic boundary condition will significantly affect the simulation results. For our cases, after a few data assimilation steps, the assimilated conductivity field with four Neumann boundaries matches the real field well while the assimilated conductivity field with mixed Dirichlet and Neumann boundaries does not. We found in our cases that the ensemble size should be 300 or larger for the numerical simulation. The number and the locations of the observation wells will significantly affect the hydraulic conductivity field calibration.     

Dual states estimation of a subsurface flow-transport coupled model using ensemble Kalman filtering

Modeling the spread of subsurface contaminants requires coupling a groundwater flow model with a contaminant transport model. Such coupling may provide accurate estimates of future subsurface hydrologic states if essential flow and contaminant data are assimilated in the model. Assuming perfect flow, an ensemble Kalman filter (EnKF) can be used for direct data assimilation into the transport model. This is, however, a crude assumption as flow models can be subject to many sources of uncertainty. If the flow is not accurately simulated, contaminant predictions will likely be inaccurate even after successive Kalman updates of the contaminant model with the data. The problem is better handled when both flow and contaminant states are concurrently estimated using the traditional joint state augmentation approach. In this paper, we introduce a dual estimation strategy for data assimilation into a one-way coupled system by treating the flow and the contaminant models separately while intertwining a pair of distinct EnKFs, one for each model. The presented strategy only deals with the estimation of state variables but it can also be used for state and parameter estimation problems. This EnKF-based dual state-state estimation procedure presents a number of novel features: (i) it allows for simultaneous estimation of both flow and contaminant states in parallel; (ii) it provides a time consistent sequential updating scheme between the two models (first flow, then transport); (iii) it simplifies the implementation of the filtering system; and (iv) it yields more stable and accurate solutions than does the standard joint approach. We conducted synthetic numerical experiments based on various time stepping and observation strategies to evaluate the dual EnKF approach and compare its performance with the joint state augmentation approach. Experimental results show that on average, the dual strategy could reduce the estimation error of the coupled states by 15% compared with the joint approach. Furthermore, the dual estimation is proven to be very effective computationally, recovering accurate estimates at a reasonable cost.     

基于热层电离层耦合数据同化的热层参量估计

本文采用高效集合卡尔曼滤波(EnKF)算法和背景场热层电离层理论模式NCAR-TIEGCM,开发了热层电离层数据同化系统.基于全球空地基GNSS电离层斜TEC观测、CHAMP和TIMED/GUVI热层参量观测构型设计了系列观测系统模拟实验,对热层参量进行估计.实验结果表明,(1)通过集合卡尔曼滤波算法同化电离层TEC观测能够较好地优化热层参量.(2)中性质量密度优化效果在整个同化阶段均有提升,提升百分比能达到40%.(3)积分氧氮比在同化阶段也能得到较好的优化,但在电子密度水平梯度变化剧烈区域效果较差.最后本文对中性质量密度进行了预报评估,结果表明,由于中性成分优化,在地磁平静条件下其预报时间尺度可长达24h.