Particle Filter-based assimilation algorithms for improved estimation of root-zone soil moisture under dynamic vegetation conditions

In this study, we implement Particle Filter (PF)-based assimilation algorithms to improve root-zone soil moisture (RZSM) estimates from a coupled SVAT-vegetation model during a growing season of sweet corn in North Central Florida. The results from four different PF algorithms were compared with those from the Ensemble Kalman Filter (EnKF) when near-surface soil moisture was assimilated every 3 days using both synthetic and field observations. In the synthetic case, the PF algorithm with the best performance used residual resampling of the states and obtained resampled parameters from a uniform distribution and provided reductions of 76% in root mean square error (RMSE) over the openloop estimates. The EnKF provided the RZSM and parameter estimates that were closer to the truth than the PF with an 84% reduction in RMSE. When field observations were assimilated, the PF algorithm that maintained maximum parameter diversity offered the largest reduction of 16% in root mean square difference (RMSD) over the openloop estimates. Minimal differences were observed in the overall performance of the EnKF and PF using field observations since errors in model physics affected both the filters in a similar manner, with maximum reductions in RMSD compared to the openloop during the mid and reproductive stages.     

Application of data assimilation with the Root Zone Water Quality Model for soil moisture profile estimation in the upper Cedar Creek,Indiana

Data assimilation techniques have been proven as an effective tool to improve model forecasts by combining information about observed variables in many areas. This article examines the potential of assimilating surface soil moisture observations into a field‐scale hydrological model, the Root Zone Water Quality Model, to improve soil moisture estimation. The Ensemble Kalman Filter (EnKF), a popular data assimilation technique for nonlinear systems, was applied and compared with a simple direct insertion method. In situ soil moisture data at four different depths (5, 20, 40, and 60 cm) from two agricultural fields (AS1 and AS2) in northeastern Indiana were used for assimilation and validation purposes. Through daily update, the EnKF improved soil moisture estimation compared with the direct insertion method and model results without assimilation, having more distinct improvement at the 5 and 20 cm depths than for deeper layers (40 and 60 cm). Local vertical soil property heterogeneity in AS1 deteriorated soil moisture estimates with the EnKF. Removal of systematic bias in the forecast model was found to be critical for more successful soil moisture data assimilation studies. This study also demonstrates that a more frequent update generally contributes in enhancing the open loop simulation; however, large forecasting error can prevent more frequent update from providing better results. In addition, results indicate that various ensemble sizes make little difference in the assimilation results. An ensemble of 100 members produced results that were comparable with results obtained from larger ensembles. Copyright © 2011 John Wiley & Sons, Ltd.     

A land surface soil moisture data assimilation framework in consideration of the model subgrid-scale heterogeneity and soil water thawing and freezing

The Ensemble Kalman Filter (EnKF) is well known and widely used in land data assimilation for its high precision and simple operation. The land surface models used as the forecast operator in a land data assimilation system are usually designed to consider the model subgrid-heterogeneity and soil water thawing and freezing. To neglect their effects could lead to some errors in soil moisture assimilation. The dual EnKF method is employed in soil moisture data assimilation to build a soil moisture data as- similation framework based on the NCAR Community Land Model version 2.0 (CLM 2.0) in considera- tion of the effects of the model subgrid-heterogeneity and soil water thawing and freezing: Liquid volumetric soil moisture content in a given fraction is assimilated through the state filter process, while solid volumetric soil moisture content in the same fraction and solid/liquid volumetric soil moisture in the other fractions are optimized by the parameter filter. Preliminary experiments show that this dual EnKF-based assimilation framework can assimilate soil moisture more effectively and precisely than the usual EnKF-based assimilation framework without considering the model subgrid-scale heteroge- neity and soil water thawing and freezing. With the improvement of soil moisture simulation, the soil temperature-simulated precision can be also improved to some extent.     

Real-time assimilation of streamflow observations into a hydrological routing model: effects of model structures and updating methods

This paper comparatively assesses the performance of five data assimilation techniques for three-parameter Muskingum routing with a spatially lumped or distributed model structure. The assimilation techniques used include direct insertion (DI), nudging scheme (NS), Kalman filter (KF), ensemble Kalman filter (EnKF) and asynchronous ensemble Kalman filter (AEnKF), which are applied to river reaches in Texas and Louisiana, USA. For both lumped and distributed routing, results from KF, EnKF and AEnKF are sensitive to the error specification. As expected, DI outperformed the other models in the case of lumped modelling, while in distributed routing, KF approaches, particularly AEnKF and EnKF, performed better than DI or nudging, reflecting the benefit of updating distributed states through error covariance modelling in KF approaches. The results of this work would be useful in setting up data assimilation systems that employ increasingly abundant real-time observations using distributed hydrological routing models.     

Comparison of Ensemble Kalman Filter groundwater-data assimilation methods based on stochastic moment equations and Monte Carlo simulation

Traditional Ensemble Kalman Filter (EnKF) data assimilation requires computationally intensive Monte Carlo (MC) sampling, which suffers from filter inbreeding unless the number of simulations is large. Recently we proposed an alternative EnKF groundwater-data assimilation method that obviates the need for sampling and is free of inbreeding issues. In our new approach, theoretical ensemble moments are approximated directly by solving a system of corresponding stochastic groundwater flow equations. Like MC-based EnKF, our moment equations (ME) approach allows Bayesian updating of system states and parameters in real-time as new data become available. Here we compare the performances and accuracies of the two approaches on two-dimensional transient groundwater flow toward a well pumping water in a synthetic, randomly heterogeneous confined aquifer subject to prescribed head and flux boundary conditions.     

Data assimilation with the EnKF in a 1-D numerical model of a North Sea station

A series of numerical experiments for data assimilation with the Ensemble Kalman Filter (EnKF) in a shallow water model are reported. Temperature profiles measured at a North Sea location, 55°30ˊ North and 0°55ˊ East (referred to as the CS station of the NERC North Sea project), are assimilated in 1-D simulations. Comparison of simulations without assimilation to model results obtained when assimilating data with the EnKF allows us to assess the filter performance in reproducing features of the observations not accounted for by the model. The quality of the model error sampling is tested as well as the validity of the Gaussian hypothesis underlying the analysis scheme of the EnKF. The influence of the model error parameters and the frequency of the data assimilation are investigated and discussed. From these experiments, a set of optimal parameters for the model error sampling are deduced and used to test the behavior of the EnKF when propagating surface information into the water column.     

Improving hydrologic predictions of a catchment model via assimilation of surface soil moisture

This paper examines the potential for improving Soil and Water Assessment Tool (SWAT) hydrologic predictions of root-zone soil moisture, evapotranspiration, and stream flow within the 341 km² Cobb Creek Watershed in southwestern Oklahoma through the assimilation of surface soil moisture observations using an Ensemble Kalman filter (EnKF). In a series of synthetic twin experiments assimilating surface soil moisture is shown to effectively update SWAT upper-layer soil moisture predictions and provide moderate improvement to lower layer soil moisture and evapotranspiration estimates. However, insufficient SWAT-predicted vertical coupling results in limited updating of deep soil moisture, regardless of the SWAT parameterization chosen for root-water extraction. Likewise, a real data assimilation experiment using ground-based soil moisture observations has only limited success in updating upper-layer soil moisture and is generally unsuccessful in enhancing SWAT stream flow predictions. Comparisons against ground-based observations suggest that SWAT significantly under-predicts the magnitude of vertical soil water coupling at the site, and this lack of coupling impedes the ability of the EnKF to effectively update deep soil moisture, groundwater flow and surface runoff. The failed attempt to improve stream flow prediction is also attributed to the inability of the EnKF to correct for existing biases in SWAT-predicted stream flow components.     

Comparison between gradient based UCODE_2005 and the ensemble Kalman Filter for transient groundwater flow inverse modeling

Gradient based UCODE_2005 and data assimilation based on the Ensemble Kalman Filter(EnKF) are two different inverse methods. A synthetic two-dimensional flow case with four no-flow boundaries is used to compare the UCODE_2005 with the Ensemble Kalman Filter(EnKF) for their efficiency to inversely calculate and calibrate a hydraulic conductivity field based on hydraulic head data. A zonal, random heterogeneous conductivity field is calibrated by assimilating the time series of heads observed in monitoring wells. The study results indicate that the two inverse methods, UCODE_2005 and EnKF, could be used to calibrate the hydraulic conductivity field to a certain degree. More available observations and information about the conductivity field, more accurate inverse results will be obtained for the UCODE_2005. On the other hand, for a realistic zonal heterogeneous hydraulic conductivity field, EnKF can only efficiently determine the hydraulic conductivity field at the first several assimilated time steps. The results obtained by the UCODE_2005 look better than those by the EnKF. This is possibly due to the fact that the UCODE_2005 uses observed head data at every time step, while EnKF can only use observed heads at first several steps due to the filter divergence problem.     

Dual state-parameter estimation of root zone soil moisture by optimal parameter estimation and extended Kalman filter data assimilation

With well-determined hydraulic parameters in a hydrologic model, a traditional data assimilation method (such as the Kalman filter and its extensions) can be used to retrieve root zone soil moisture under uncertain initial state variables (e.g., initial soil moisture content) and good simulated results can be achieved. However, when the key soil hydraulic parameters are incorrect, the error is non-Gaussian, as the Kalman filter will produce a persistent bias in its predictions. In this paper, we propose a method coupling optimal parameters and extended Kalman filter data assimilation (OP-EKF) by combining optimal parameter estimation, the extended Kalman filter (EKF) assimilation method, a particle swarm optimization (PSO) algorithm, and Richards’ equation. We examine the accuracy of estimating root zone soil moisture through the optimal parameters and extended Kalman filter data assimilation method by using observed in situ data at the Meiling experimental station, China. Results indicate that merely using EKF for assimilating surface soil moisture content to obtain soil moisture content in the root zone will produce a persistent bias between simulated and observed values. Using the OP-EKF assimilation method, estimates were clearly improved. If the soil profile is heterogeneous, soil moisture retrieval is accurate in the 0-50 cm soil profile and is inaccurate at 100 cm depth. Results indicate that the method is useful for retrieving root zone soil moisture over large areas and long timescales even when available soil moisture data are limited to the surface layer, and soil moisture content are uncertain and soil hydraulic parameters are incorrect.     

Dual state–parameter estimation of hydrological models using ensemble Kalman filter

Hydrologic models are twofold: models for understanding physical processes and models for prediction. This study addresses the latter, which modelers use to predict, for example, streamflow at some future time given knowledge of the current state of the system and model parameters. In this respect, good estimates of the parameters and state variables are needed to enable the model to generate accurate forecasts. In this paper, a dual state–parameter estimation approach is presented based on the Ensemble Kalman Filter (EnKF) for sequential estimation of both parameters and state variables of a hydrologic model. A systematic approach for identification of the perturbation factors used for ensemble generation and for selection of ensemble size is discussed. The dual EnKF methodology introduces a number of novel features: (1) both model states and parameters can be estimated simultaneously; (2) the algorithm is recursive and therefore does not require storage of all past information, as is the case in the batch calibration procedures; and (3) the various sources of uncertainties can be properly addressed, including input, output, and parameter uncertainties. The applicability and usefulness of the dual EnKF approach for ensemble streamflow forecasting is demonstrated using a conceptual rainfall-runoff model.     

Reduction of Used Memory Ensemble Kalman Filtering (RumEnKF): A data assimilation scheme for memory intensive,high performance computing

Reduction of Used Memory Ensemble Kalman Filtering (RumEnKF) is introduced as a variant on the Ensemble Kalman Filter (EnKF). RumEnKF differs from EnKF in that it does not store the entire ensemble, but rather only saves the first two moments of the ensemble distribution. In this way, the number of ensemble members that can be calculated is less dependent on available memory, and mainly on available computing power (CPU). RumEnKF is developed to make optimal use of current generation super computer architecture, where the number of available floating point operations (flops) increases more rapidly than the available memory and where inter-node communication can quickly become a bottleneck. RumEnKF reduces the used memory compared to the EnKF when the number of ensemble members is greater than half the number of state variables. In this paper, three simple models are used (auto-regressive, low dimensional Lorenz and high dimensional Lorenz) to show that RumEnKF performs similarly to the EnKF. Furthermore, it is also shown that increasing the ensemble size has a similar impact on the estimation error from the three algorithms.     

Comparison of deterministic ensemble Kalman filters for assimilating hydrogeological data

Groundwater models are critical decision support tools for water resources management and environmental remediation. However, limitations in site characterization data and conceptual models can adversely affect the reliability of groundwater models. Therefore, there is a strong need for continuous model uncertainty reduction. Ensemble filters have recently emerged as promising high-dimensional data assimilation techniques. Two general categories of ensemble filters exist in the literature: perturbation-based and deterministic. Deterministic ensemble filters have been extensively studied for their better performance and robustness in assimilating oceanographic and atmospheric data. In hydrogeology, while a number of previous studies demonstrated the usefulness of the perturbation-based ensemble Kalman filter (EnKF) for joint parameter and state estimation, there have been few systematic studies investigating the performance of deterministic ensemble filters. This paper presents a comparative study of four commonly used deterministic ensemble filters for sequentially estimating the hydraulic conductivity parameter in low- and moderately high-dimensional groundwater models. The performance of the filters is assessed on the basis of twin experiments in which the true hydraulic conductivity field is assumed known. The test results indicate that the deterministic ensemble Kalman filter (DEnKF) is the most robust filter and achieves the best performance at relatively small ensemble sizes. Deterministic ensemble filters often make use of covariance inflation and localization to stabilize filter performance. Sensitivity studies demonstrate the effects of covariance inflation, localization, observation density, and conditioning on filter performance.     

Hydrological data assimilation with the Ensemble Square-Root-Filter: Use of streamflow observations to update model states for real-time flash flood forecasting

The objective of the study is to evaluate the potential of a data assimilation system for real-time flash flood forecasting over small watersheds by updating model states. To this end, the Ensemble Square-Root-Filter (EnSRF) based on the Ensemble Kalman Filter (EnKF) technique was coupled to a widely used conceptual rainfall-runoff model called HyMOD. Two small watersheds susceptible to flash flooding from America and China were selected in this study. The modeling and observational errors were considered in the framework of data assimilation, followed by an ensemble size sensitivity experiment. Once the appropriate model error and ensemble size was determined, a simulation study focused on the performance of a data assimilation system, based on the correlation between streamflow observation and model states, was conducted. The EnSRF method was implemented within HyMOD and results for flash flood forecasting were analyzed, where the calibrated streamflow simulation without state updating was treated as the benchmark or nature run. Results for twenty-four flash-flood events in total from the two watersheds indicated that the data assimilation approach effectively improved the predictions of peak flows and the hydrographs in general. This study demonstrated the benefit and efficiency of implementing data assimilation into a hydrological model to improve flash flood forecasting over small, instrumented basins with potential application to real-time alert systems.     

Data assimilation for distributed hydrological catchment modeling via ensemble Kalman filter

Catchment scale hydrological models are critical decision support tools for water resources management and environment remediation. However, the reliability of hydrological models is inevitably affected by limited measurements and imperfect models. Data assimilation techniques combine complementary information from measurements and models to enhance the model reliability and reduce predictive uncertainties. As a sequential data assimilation technique, the ensemble Kalman filter (EnKF) has been extensively studied in the earth sciences for assimilating in-situ measurements and remote sensing data. Although the EnKF has been demonstrated in land surface data assimilations, there are no systematic studies to investigate its performance in distributed modeling with high dimensional states and parameters. In this paper, we present an assessment on the EnKF with state augmentation for combined state-parameter estimation on the basis of a physical-based hydrological model, Soil and Water Assessment Tool (SWAT). Through synthetic simulation experiments, the capability of the EnKF is demonstrated by assimilating the runoff and other measurements, and its sensitivities are analyzed with respect to the error specification, the initial realization and the ensemble size. It is found that the EnKF provides an efficient approach for obtaining a set of acceptable model parameters and satisfactory runoff, soil water content and evapotranspiration estimations. The EnKF performance could be improved after augmenting with other complementary data, such as soil water content and evapotranspiration from remote sensing retrieval. Sensitivity studies demonstrate the importance of consistent error specification and the potential with small ensemble size in the data assimilation system.     

Data assimilation for unsaturated flow models with restart adaptive probabilistic collocation based Kalman filter

The ensemble Kalman filter (EnKF) has gained popularity in hydrological data assimilation problems. As a Monte Carlo based method, a sufficiently large ensemble size is usually required to guarantee the accuracy. As an alternative approach, the probabilistic collocation based Kalman filter (PCKF) employs the polynomial chaos expansion (PCE) to represent and propagate the uncertainties in parameters and states. However, PCKF suffers from the so-called “curse of dimensionality”. Its computational cost increases drastically with the increasing number of parameters and system nonlinearity. Furthermore, PCKF may fail to provide accurate estimations due to the joint updating scheme for strongly nonlinear models. Motivated by recent developments in uncertainty quantification and EnKF, we propose a restart adaptive probabilistic collocation based Kalman filter (RAPCKF) for data assimilation in unsaturated flow problems. During the implementation of RAPCKF, the important parameters are identified and active PCE basis functions are adaptively selected at each assimilation step; the “restart” scheme is utilized to eliminate the inconsistency between updated model parameters and states variables. The performance of RAPCKF is systematically tested with numerical cases of unsaturated flow models. It is shown that the adaptive approach and restart scheme can significantly improve the performance of PCKF. Moreover, RAPCKF has been demonstrated to be more efficient than EnKF with the same computational cost.     

Assimilation of SLA and SST data into an OGCM for the Indian Ocean

 Remotely sensed observations of sea-level anomaly and sea-surface temperature have been assimilated into an implementation of the Miami Isopycnic Coordinate Ocean Model (MICOM) for the Indian Ocean using the Ensemble Kalman Filter (EnKF). The system has been applied in a hindcast validation experiment to examine the properties of the assimilation scheme when used with a full ocean general circulation model and real observations. This work is considered as a first step towards an operational ocean monitoring and forecasting system for the Indian Ocean. The assimilation of real data has demonstrated that the sequential EnKF can efficiently control the model evolution in time. The use of data assimilation requires a significant amount of additional processing and computational resources. However, we have tried to justify the cost of using a sophisticated assimilation scheme by demonstrating strong regional and temporal dependencies of the covariance statistics, which include highly anisotropic and flow-dependent correlation functions. In particular, we observed a marked difference between error statistics in the equatorial region and at off-equatorial latitudes. We have also demonstrated how the assimilation of SLA and SST improves the model fields with respect to real observations. Independent in situ temperature profiles have been used to examine the impact of assimilating the remotely sensed observations. These intercomparisons have shown that the model temperature and salinity fields better resemble in situ observations in the assimilation experiment than in a model free-run case. On the other hand, it is also expected that assimilation of in situ profiles is needed to properly control the deep ocean circulation. Received: 8 January 2002 / Accepted: 8 April 2002     

An application of Ensemble Kalman Filter in integral-balance subsurface modeling

Data assimilation method provides a framework to decrease the uncertainty of hydrological modeling by sequentially incorporating observations into numerical model. Such a process involves estimating statistical moments of different order based on the evolution of conditional probability distribution function. Because of the nonlinearity of many hydrological dynamics, explicit and analytical solutions for moments of state distribution are often impossible. Evensen [J Geophys Res 99(c5): 10143–10162 (1994)] introduced Ensemble Kalman Filtering (EnKF) method to address such problems. We test and evaluate the performance of EnKF in fusing model predictions and observations for a saturated–unsaturated integral-balance subsurface model. We find EnKF improve the model predictions, and also we conclude a good estimate of state variance is essential for the success of EnKF.     

A soil moisture assimilation scheme based on the ensemble Kalman filter using microwave brightness temperature

This study presents a soil moisture assimilation scheme, which could assimilate microwave brightness temperature directly, based on the ensemble Kalman filter and the shuffled complex evolution method (SCE-UA). It uses the soil water model of the land surface model CLM3.0 as the forecast operator, and a radiative transfer model (RTM) as the observation operator in the assimilation system. The assimilation scheme is implemented in two phases: the parameter calibration phase and the pure soil moisture assimilation phase. The vegetation optical thickness and surface roughness parameters in the RTM are calibrated by SCE-UA method and the optimal parameters are used as the final model parameters of the observation operator in the assimilation phase. The ideal experiments with synthetic data indicate that this scheme could significantly improve the simulation of soil moisture at the surface layer. Furthermore, the estimation of soil moisture in the deeper layers could also be improved to a certain extent. The real assimilation experiments with AMSR-E brightness temperature at 10.65 GHz (vertical polarization) show that the root mean square error (RMSE) of soil moisture in the top layer (0–10 cm) by assimilation is 0.03355 m³ · m⁻³, which is reduced by 33.6% compared with that by simulation (0.05052 m³ · m⁻³). The mean RMSE by assimilation for the deeper layers (10–50 cm) is also reduced by 20.9%. All these experiments demonstrate the reasonability of the assimilation scheme developed in this study.     

Dual states estimation of a subsurface flow-transport coupled model using ensemble Kalman filtering

Modeling the spread of subsurface contaminants requires coupling a groundwater flow model with a contaminant transport model. Such coupling may provide accurate estimates of future subsurface hydrologic states if essential flow and contaminant data are assimilated in the model. Assuming perfect flow, an ensemble Kalman filter (EnKF) can be used for direct data assimilation into the transport model. This is, however, a crude assumption as flow models can be subject to many sources of uncertainty. If the flow is not accurately simulated, contaminant predictions will likely be inaccurate even after successive Kalman updates of the contaminant model with the data. The problem is better handled when both flow and contaminant states are concurrently estimated using the traditional joint state augmentation approach. In this paper, we introduce a dual estimation strategy for data assimilation into a one-way coupled system by treating the flow and the contaminant models separately while intertwining a pair of distinct EnKFs, one for each model. The presented strategy only deals with the estimation of state variables but it can also be used for state and parameter estimation problems. This EnKF-based dual state-state estimation procedure presents a number of novel features: (i) it allows for simultaneous estimation of both flow and contaminant states in parallel; (ii) it provides a time consistent sequential updating scheme between the two models (first flow, then transport); (iii) it simplifies the implementation of the filtering system; and (iv) it yields more stable and accurate solutions than does the standard joint approach. We conducted synthetic numerical experiments based on various time stepping and observation strategies to evaluate the dual EnKF approach and compare its performance with the joint state augmentation approach. Experimental results show that on average, the dual strategy could reduce the estimation error of the coupled states by 15% compared with the joint approach. Furthermore, the dual estimation is proven to be very effective computationally, recovering accurate estimates at a reasonable cost.