Variational data assimilation for parameter estimation: application to a simple morphodynamic model

Data assimilation is a sophisticated mathematical technique for combining observational data with model predictions to produce state and parameter estimates that most accurately approximate the current and future states of the true system. The technique is commonly used in atmospheric and oceanic modelling, combining empirical observations with model predictions to produce more accurate and well-calibrated forecasts. Here, we consider a novel application within a coastal environment and describe how the method can also be used to deliver improved estimates of uncertain morphodynamic model parameters. This is achieved using a technique known as state augmentation. Earlier applications of state augmentation have typically employed the 4D-Var, Kalman filter or ensemble Kalman filter assimilation schemes. Our new method is based on a computationally inexpensive 3D-Var scheme, where the specification of the error covariance matrices is crucial for success. A simple 1D model of bed-form propagation is used to demonstrate the method. The scheme is capable of recovering near-perfect parameter values and, therefore, improves the capability of our model to predict future bathymetry. Such positive results suggest the potential for application to more complex morphodynamic models.     

Real-time assimilation of streamflow observations into a hydrological routing model: effects of model structures and updating methods

This paper comparatively assesses the performance of five data assimilation techniques for three-parameter Muskingum routing with a spatially lumped or distributed model structure. The assimilation techniques used include direct insertion (DI), nudging scheme (NS), Kalman filter (KF), ensemble Kalman filter (EnKF) and asynchronous ensemble Kalman filter (AEnKF), which are applied to river reaches in Texas and Louisiana, USA. For both lumped and distributed routing, results from KF, EnKF and AEnKF are sensitive to the error specification. As expected, DI outperformed the other models in the case of lumped modelling, while in distributed routing, KF approaches, particularly AEnKF and EnKF, performed better than DI or nudging, reflecting the benefit of updating distributed states through error covariance modelling in KF approaches. The results of this work would be useful in setting up data assimilation systems that employ increasingly abundant real-time observations using distributed hydrological routing models.     

A land surface data assimilation framework using the land information system: Description and applications

The Land Information System (LIS) is an established land surface modeling framework that integrates various community land surface models, ground measurements, satellite-based observations, high performance computing and data management tools. The use of advanced software engineering principles in LIS allows interoperability of individual system components and thus enables assessment and prediction of hydrologic conditions at various spatial and temporal scales. In this work, we describe a sequential data assimilation extension of LIS that incorporates multiple observational sources, land surface models and assimilation algorithms. These capabilities are demonstrated here in a suite of experiments that use the ensemble Kalman filter (EnKF) and assimilation through direct insertion. In a soil moisture experiment, we discuss the impact of differences in modeling approaches on assimilation performance. Provided careful choice of model error parameters, we find that two entirely different hydrological modeling approaches offer comparable assimilation results. In a snow assimilation experiment, we investigate the relative merits of assimilating different types of observations (snow cover area and snow water equivalent). The experiments show that data assimilation enhancements in LIS are uniquely suited to compare the assimilation of various data types into different land surface models within a single framework. The high performance infrastructure provides adequate support for efficient data assimilation integrations of high computational granularity.     

Argo data assimilation in ocean general circulation model of Northwest Pacific Ocean

The Argo temperature and salinity profiles in 2005–2009 are assimilated into a coastal ocean general circulation model of the Northwest Pacific Ocean using the ensemble adjustment Kalman filter (EAKF). Three numerical tests, including the control run (CTL) (without data assimilation, which serves as the reference experiment), ensemble free run (EnFR) (without data assimilation), and EAKF experiment (with Argo data assimilation using EAKF), are carried out to examine the performance of this system. Using the restarts of different years as the initial conditions of the ensemble integrations, the ensemble spreads from EnFR and EAKF are all kept at a finite value after a sharp decreasing in the first few months because of the sensitive of the model to the initial conditions, and the reducing of the ensemble spread due to Argo data assimilation is not much. The ensemble samples obtained in this way can well represent the probabilities of the real ocean states, and no ensemble inflation is necessary for this EAKF experiment. Different experiment results are compared with satellite sea surface temperature (SST) data and the Global Temperature-Salinity Profile Program (GTSPP) data. The comparison of SST shows that modeled SST errors are reduced after data assimilation; the error reduction percentage after assimilating the Argo profiles is about 10?% on average. The comparison against the GTSPP profiles, which are independent of the Argo profiles, shows improvements in both temperature and salinity. The comparison results indicated a great error reduction in all vertical layers relative to CTL and the ensemble mean of EnFR; the maximum value for temperature and salinity reaches to 85?% and 80?%, respectively. The standard deviations of sea surface height are employed to examine the simulation ability, and it is shown that the mesoscale variability is improved after Argo data assimilation, especially in the Kuroshio extension area and along the section of 10°N. All these results suggest that this system is potentially useful for improving the simulation ability of oceanic numerical models.     

基于集合卡尔曼滤波的湖泊富营养化模型Delft3D-BLOOM数据同化

富营养化模型是进行湖泊水环境质量预测和管理的重要工具,然而模型客观存在的误差一直是应用者关心的重要问题.数据同化作为连接观测数据与数值模型的重要方法,可以有效提高模型的准确性.集合卡尔曼滤波(En KF)是众多数据同化算法中应用最为广泛的一种,可进行非线性系统的数据同化,并能有效降低数据同化的计算量.本研究以太湖作为具体实例,选择Delft3D-BLOOM作为富营养化模型,在数值实验确定En KF集合数为100、观测误差方差为1%、模拟误差方差为10%的基础上分别进行模型状态变量同化以及状态变量与关键参数同步同化.结果显示,仅同化状态变量时,模型预测精度有所增加;同时同化状态变量和关键参数时,可显著提升模型在湖泊水环境质量预测中的精度.该研究为应用集合卡尔曼滤波以提高复杂的湖库富营养化模型模拟精度提供了有效的方法.     

Coupled assimilation for an intermediated coupled ENSO prediction model

The value of coupled assimilation is discussed using an intermediate coupled model in which the wind stress is the only atmospheric state which is slavery to model sea surface temperature (SST). In the coupled assimilation analysis, based on the coupled wind–ocean state covariance calculated from the coupled state ensemble, the ocean state is adjusted by assimilating wind data using the ensemble Kalman filter. As revealed by a series of assimilation experiments using simulated observations, the coupled assimilation of wind observations yields better results than the assimilation of SST observations. Specifically, the coupled assimilation of wind observations can help to improve the accuracy of the surface and subsurface currents because the correlation between the wind and ocean currents is stronger than that between SST and ocean currents in the equatorial Pacific. Thus, the coupled assimilation of wind data can decrease the initial condition errors in the surface/subsurface currents that can significantly contribute to SST forecast errors. The value of the coupled assimilation of wind observations is further demonstrated by comparing the prediction skills of three 12-year (1997–2008) hindcast experiments initialized by the ocean-only assimilation scheme that assimilates SST observations, the coupled assimilation scheme that assimilates wind observations, and a nudging scheme that nudges the observed wind stress data, respectively. The prediction skills of two assimilation schemes are significantly better than those of the nudging scheme. The prediction skills of assimilating wind observations are better than assimilating SST observations. Assimilating wind observations for the 2007/2008 La Niña event triggers better predictions, while assimilating SST observations fails to provide an early warning for that event.     

The Local Ensemble Transform Kalman Filter with the Weather Research and Forecasting Model: Experiments with Real Observations

The local ensemble transform Kalman filter (LETKF) is implemented with the Weather Research and Forecasting (WRF) model, and real observations are assimilated to assess the newly-developed WRF-LETKF system. The WRF model is a widely-used mesoscale numerical weather prediction model, and the LETKF is an ensemble Kalman filter (EnKF) algorithm particularly efficient in parallel computer architecture. This study aims to provide the basis of future research on mesoscale data assimilation using the WRF-LETKF system, an additional testbed to the existing EnKF systems with the WRF model used in the previous studies. The particular LETKF system adopted in this study is based on the system initially developed in 2004 and has been continuously improved through theoretical studies and wide applications to many kinds of dynamical models including realistic geophysical models. Most recent and important improvements include an adaptive covariance inflation scheme which considers the spatial and temporal inhomogeneity of inflation parameters. Experiments show that the LETKF successfully assimilates real observations and that adaptive inflation is advantageous. Additional experiments with various ensemble sizes show that using more ensemble members improves the analyses consistently.     

Calibration framework for a Kalman filter applied to a groundwater model

The paper presents a novel approach to the setup of a Kalman filter by using an automatic calibration framework for estimation of the covariance matrices. The calibration consists of two sequential steps: (1) Automatic calibration of a set of covariance parameters to optimize the performance of the system and (2) adjustment of the model and observation variance to provide an uncertainty analysis relying on the data instead of ad-hoc covariance values. The method is applied to a twin-test experiment with a groundwater model and a colored noise Kalman filter. The filter is implemented in an ensemble framework. It is demonstrated that lattice sampling is preferable to the usual Monte Carlo simulation because its ability to preserve the theoretical mean reduces the size of the ensemble needed. The resulting Kalman filter proves to be efficient in correcting dynamic error and bias over the whole domain studied. The uncertainty analysis provides a reliable estimate of the error in the neighborhood of assimilation points but the simplicity of the covariance models leads to underestimation of the errors far from assimilation points.     

One-dimensional soil moisture profile retrieval by assimilation of near-surface observations: a comparison of retrieval algorithms

This paper investigates the ability to retrieve the true soil moisture and temperature profiles by assimilating near-surface soil moisture and surface temperature data into a soil moisture and heat transfer model. The direct insertion and Kalman filter assimilation schemes have been used most frequently in assimilation studies, but no comparisons of these schemes have been made. This study investigates which of these approaches is able to retrieve the soil moisture and temperature profiles the fastest, over what depth soil moisture observations are required, and the effect of update interval on profile retrieval. These questions are addressed by a desktop study using synthetic data. The study shows that the Kalman filter assimilation scheme is superior to the direct insertion assimilation scheme, with retrieval of the soil moisture profile being achieved in 12 h as compared to 8 days or more, depending on observation depth, for hourly observations. It was also found that profile retrieval could not be realised for direct insertion of the surface node alone, and that observation depth does not have a significant effect on profile retrieval time for the Kalman filter. The observation interval was found to be unimportant for profile retrieval with the Kalman filter when the forcing data is accurate, whilst for direct insertion the continuous Dirichlet boundary condition was required for an increasingly longer period of time. It was also found that the Kalman filter assimilation scheme was less susceptible to unstable updates if volumetric soil moisture was modelled as the dependent state rather than matric head, because the volumetric soil moisture state is more linear in the forecasting model.     

Evaluating the utility of the ensemble transform Kalman filter for adaptive sampling when updating a hydrodynamic model

This paper compares two Monte Carlo sequential data assimilation methods based on the Kalman filter, for estimating the effect of measurements on simulations of state error variance made by a one-dimensional hydrodynamic model. The first method used an ensemble Kalman filter (EnKF) to update state estimates, which were then used as initial conditions for further simulations. The second method used an ensemble transform Kalman filter (ETKF) to quickly estimate the effect of measurement error covariance on forecast error covariance without the need to re-run the simulation model. The ETKF gave an unbiased estimate of EnKF analysed error variance, although differences in the treatment of measurement errors meant the results were not identical. Estimates of forecast error variance could also be made, but their accuracy deteriorated as the time from measurements increased due in part to model non-linearity and the decreasing signal variance. The motivation behind the study was to assess the ability of the ETKF to target possible measurements, as part of an adaptive sampling framework, before they are assimilated by an EnKF-based forecasting model on the River Crouch, Essex, UK. The ETKF was found to be a useful tool for quickly estimating the error covariance expected after assimilating measurements into the hydrodynamic model. It, thus, provided a means of quantifying the ‘usefulness’ (in terms of error variance) of possible sampling schemes.     

Application of data assimilation for improving forecast of water levels and residual currents in Singapore regional waters

Hydrodynamic models are commonly used for predicting water levels and currents in the deep ocean, ocean margins and shelf seas. Their accuracy is typically limited by factors, such as the complexity of the coastal geometry and bathymetry, plus the uncertainty in the flow forcing (deep ocean tide, winds and pressure). In Southeast Asian waters with its strongly hydrodynamic characteristics, the lack of detailed marine observations (bathymetry and tides) for model validation is an additional factor limiting flow representation. This paper deals with the application of ensemble Kalman filter (EnKF)-based data assimilation with the purpose of improving the deterministic model forecast. The efficacy of the EnKF is analysed via a twin experiment conducted with the 2D barotropic Singapore regional model. The results show that the applied data assimilation can improve the forecasts significantly in this complex flow regime.     

Data assimilation for distributed hydrological catchment modeling via ensemble Kalman filter

Catchment scale hydrological models are critical decision support tools for water resources management and environment remediation. However, the reliability of hydrological models is inevitably affected by limited measurements and imperfect models. Data assimilation techniques combine complementary information from measurements and models to enhance the model reliability and reduce predictive uncertainties. As a sequential data assimilation technique, the ensemble Kalman filter (EnKF) has been extensively studied in the earth sciences for assimilating in-situ measurements and remote sensing data. Although the EnKF has been demonstrated in land surface data assimilations, there are no systematic studies to investigate its performance in distributed modeling with high dimensional states and parameters. In this paper, we present an assessment on the EnKF with state augmentation for combined state-parameter estimation on the basis of a physical-based hydrological model, Soil and Water Assessment Tool (SWAT). Through synthetic simulation experiments, the capability of the EnKF is demonstrated by assimilating the runoff and other measurements, and its sensitivities are analyzed with respect to the error specification, the initial realization and the ensemble size. It is found that the EnKF provides an efficient approach for obtaining a set of acceptable model parameters and satisfactory runoff, soil water content and evapotranspiration estimations. The EnKF performance could be improved after augmenting with other complementary data, such as soil water content and evapotranspiration from remote sensing retrieval. Sensitivity studies demonstrate the importance of consistent error specification and the potential with small ensemble size in the data assimilation system.     

Assimilation of ice concentration in a coupled ice–ocean model,using the Ensemble Kalman filter

An implementation of the Ensemble Kalman filter (EnKF) with a coupled ice–ocean model is presented. The model system consists of a dynamic–thermodynamic ice model using the elastic-viscous-plastic (EVP) rheology coupled with the HYbrid Coordinate Ocean Model (HYCOM). The observed variable is ice concentration from passive microwave sensor data (SSM/I). The assimilation of ice concentration has the desired effect of reducing the difference between observations and model. Comparison of the assimilation experiment with a free-run experiment shows that there are large differences, especially in summer. In winter the differences are relatively small, partly because the atmospheric forcing used to run the model depends upon SSM/I data. The assimilation has the strongest impact close to the ice edge, where it ensures a correct location of the ice edge throughout the simulation. An inspection of the model ensemble statistics reveals that the error estimates of the model are too small in winter, partly a result of too low model ice-concentration variance in the central ice pack. It is found that the ensemble covariance between ice concentration and sea-surface temperature in the same grid cell is of the same sign (negative) throughout the year. The ensemble covariance between ice concentration and salinity is more dependent upon the physical mechanisms involved, with ice transport and freeze/melt giving different signs of the covariances. The ice-transport and ice-melt mechanisms also impact the ice-concentration variance and the covariance between ice concentration and ice thickness. The ensemble statistics show a high degree of complexity, which to some extent merits the use of computationally expensive assimilation methods, such as the Ensemble Kalman filter. The present study focuses on the assimilation of ice concentration, but it is understood that assimilation of other datasets, such as sea-surface temperature, would be beneficial.Responsible Editor: Jin-Song von Storch     

Shock capturing data assimilation algorithm for 1D shallow water equations

We propose a new data assimilation algorithm for shallow water equations in one dimension. The algorithm is based upon Discontinuous Galerkin spatial discretization of shallow water equations (DG-SW model) and the continuous formulation of the minimax filter. The latter allows for construction of a robust estimation of the state of the DG-SW model and computes worst-case bounds for the estimation error, provided the uncertain parameters belong to a given bounding set. Numerical studies show that, given sparse observations from numerical or physical experiments, the proposed algorithm quickly reconstructs the true solution even in the presence of shocks, rarefaction waves and unknown values of model parameters. The minimax filter is compared against the ensemble Kalman filter (EnKF) for a benchmark dam-break problem and the results show that the minimax filter converges faster to the true solution for sparse observations.     

Parameter estimation in ensemble based snow data assimilation: A synthetic study

Estimating erroneous parameters in ensemble based snow data assimilation system has been given little attention in the literature. Little is known about the related methods’ effectiveness, performance, and sensitivity to other error sources such as model structural error. This research tackles these questions by running synthetic one-dimensional snow data assimilation with the ensemble Kalman filter (EnKF), in which both state and parameter are simultaneously updated. The first part of the paper investigates the effectiveness of this parameter estimation approach in a perfect-model-structure scenario, and the second part focuses on its dependence on model structure error. The results from first part research demonstrate the advantages of this parameter estimation approach in reducing the systematic error of snow water equivalent (SWE) estimates, and retrieving the correct parameter value. The second part results indicate that, at least in our experiment, there is an evident dependence of parameter search convergence on model structural error. In the imperfect-model-structure run, the parameter search diverges, although it can simulate the state variable well. This result suggest that, good data assimilation performance in estimating state variables is not a sufficient indicator of reliable parameter retrieval in the presence of model structural error. The generality of this conclusion needs to be tested by data assimilation experiments with more complex structural error configurations.     

The role of prior model calibration on predictions with ensemble Kalman filter

This paper, based on a real world case study (Limmat aquifer, Switzerland), compares inverse groundwater flow models calibrated with specified numbers of monitoring head locations. These models are updated in real time with the ensemble Kalman filter (EnKF) and the prediction improvement is assessed in relation to the amount of monitoring locations used for calibration and updating. The prediction errors of the models calibrated in transient state are smaller if the amount of monitoring locations used for the calibration is larger. For highly dynamic groundwater flow systems a transient calibration is recommended as a model calibrated in steady state can lead to worse results than a noncalibrated model with a well-chosen uniform conductivity. The model predictions can be improved further with the assimilation of new measurement data from on-line sensors with the EnKF. Within all the studied models the reduction of 1-day hydraulic head prediction error (in terms of mean absolute error [MAE]) with EnKF lies between 31% (assimilation of head data from 5 locations) and 72% (assimilation of head data from 85 locations). The largest prediction improvements are expected for models that were calibrated with only a limited amount of historical information. It is worthwhile to update the model even with few monitoring locations as it seems that the error reduction with EnKF decreases exponentially with the amount of monitoring locations used. These results prove the feasibility of data assimilation with EnKF also for a real world case and show that improved predictions of groundwater levels can be obtained.     

Comparison of deterministic ensemble Kalman filters for assimilating hydrogeological data

Groundwater models are critical decision support tools for water resources management and environmental remediation. However, limitations in site characterization data and conceptual models can adversely affect the reliability of groundwater models. Therefore, there is a strong need for continuous model uncertainty reduction. Ensemble filters have recently emerged as promising high-dimensional data assimilation techniques. Two general categories of ensemble filters exist in the literature: perturbation-based and deterministic. Deterministic ensemble filters have been extensively studied for their better performance and robustness in assimilating oceanographic and atmospheric data. In hydrogeology, while a number of previous studies demonstrated the usefulness of the perturbation-based ensemble Kalman filter (EnKF) for joint parameter and state estimation, there have been few systematic studies investigating the performance of deterministic ensemble filters. This paper presents a comparative study of four commonly used deterministic ensemble filters for sequentially estimating the hydraulic conductivity parameter in low- and moderately high-dimensional groundwater models. The performance of the filters is assessed on the basis of twin experiments in which the true hydraulic conductivity field is assumed known. The test results indicate that the deterministic ensemble Kalman filter (DEnKF) is the most robust filter and achieves the best performance at relatively small ensemble sizes. Deterministic ensemble filters often make use of covariance inflation and localization to stabilize filter performance. Sensitivity studies demonstrate the effects of covariance inflation, localization, observation density, and conditioning on filter performance.     

Snow water equivalent prediction using Bayesian data assimilation methods

Using the U.S. National Weather Service’s SNOW-17 model, this study compares common sequential data assimilation methods, the ensemble Kalman filter (EnKF), the ensemble square root filter (EnSRF), and four variants of the particle filter (PF), to predict seasonal snow water equivalent (SWE) within a small watershed near Lake Tahoe, California. In addition to SWE estimation, the various data assimilation methods are used to estimate five of the most sensitive parameters of SNOW-17 by allowing them to evolve with the dynamical system. Unlike Kalman filters, particle filters do not require Gaussian assumptions for the posterior distribution of the state variables. However, the likelihood function used to scale particle weights is often assumed to be Gaussian. This study evaluates the use of an empirical cumulative distribution function (ECDF) based on the Kaplan–Meier survival probability method to compute particle weights. These weights are then used in different particle filter resampling schemes. Detailed analyses are conducted for synthetic and real data assimilation and an assessment of the procedures is made. The results suggest that the particle filter, especially the empirical likelihood variant, is superior to the ensemble Kalman filter based methods for predicting model states, as well as model parameters.     

An approach to handling non-Gaussianity of parameters and state variables in ensemble Kalman filtering

The ensemble Kalman filter (EnKF) is a commonly used real-time data assimilation algorithm in various disciplines. Here, the EnKF is applied, in a hydrogeological context, to condition log-conductivity realizations on log-conductivity and transient piezometric head data. In this case, the state vector is made up of log-conductivities and piezometric heads over a discretized aquifer domain, the forecast model is a groundwater flow numerical model, and the transient piezometric head data are sequentially assimilated to update the state vector. It is well known that all Kalman filters perform optimally for linear forecast models and a multiGaussian-distributed state vector. Of the different Kalman filters, the EnKF provides a robust solution to address non-linearities; however, it does not handle well non-Gaussian state-vector distributions. In the standard EnKF, as time passes and more state observations are assimilated, the distributions become closer to Gaussian, even if the initial ones are clearly non-Gaussian. A new method is proposed that transforms the original state vector into a new vector that is univariate Gaussian at all times. Back transforming the vector after the filtering ensures that the initial non-Gaussian univariate distributions of the state-vector components are preserved throughout. The proposed method is based in normal-score transforming each variable for all locations and all time steps. This new method, termed the normal-score ensemble Kalman filter (NS-EnKF), is demonstrated in a synthetic bimodal aquifer resembling a fluvial deposit, and it is compared to the standard EnKF. The proposed method performs better than the standard EnKF in all aspects analyzed (log-conductivity characterization and flow and transport predictions).