Snow water equivalent prediction using Bayesian data assimilation methods

Using the U.S. National Weather Service’s SNOW-17 model, this study compares common sequential data assimilation methods, the ensemble Kalman filter (EnKF), the ensemble square root filter (EnSRF), and four variants of the particle filter (PF), to predict seasonal snow water equivalent (SWE) within a small watershed near Lake Tahoe, California. In addition to SWE estimation, the various data assimilation methods are used to estimate five of the most sensitive parameters of SNOW-17 by allowing them to evolve with the dynamical system. Unlike Kalman filters, particle filters do not require Gaussian assumptions for the posterior distribution of the state variables. However, the likelihood function used to scale particle weights is often assumed to be Gaussian. This study evaluates the use of an empirical cumulative distribution function (ECDF) based on the Kaplan–Meier survival probability method to compute particle weights. These weights are then used in different particle filter resampling schemes. Detailed analyses are conducted for synthetic and real data assimilation and an assessment of the procedures is made. The results suggest that the particle filter, especially the empirical likelihood variant, is superior to the ensemble Kalman filter based methods for predicting model states, as well as model parameters.     

Evaluating the utility of the ensemble transform Kalman filter for adaptive sampling when updating a hydrodynamic model

This paper compares two Monte Carlo sequential data assimilation methods based on the Kalman filter, for estimating the effect of measurements on simulations of state error variance made by a one-dimensional hydrodynamic model. The first method used an ensemble Kalman filter (EnKF) to update state estimates, which were then used as initial conditions for further simulations. The second method used an ensemble transform Kalman filter (ETKF) to quickly estimate the effect of measurement error covariance on forecast error covariance without the need to re-run the simulation model. The ETKF gave an unbiased estimate of EnKF analysed error variance, although differences in the treatment of measurement errors meant the results were not identical. Estimates of forecast error variance could also be made, but their accuracy deteriorated as the time from measurements increased due in part to model non-linearity and the decreasing signal variance. The motivation behind the study was to assess the ability of the ETKF to target possible measurements, as part of an adaptive sampling framework, before they are assimilated by an EnKF-based forecasting model on the River Crouch, Essex, UK. The ETKF was found to be a useful tool for quickly estimating the error covariance expected after assimilating measurements into the hydrodynamic model. It, thus, provided a means of quantifying the ‘usefulness’ (in terms of error variance) of possible sampling schemes.     

Variational data assimilation for parameter estimation: application to a simple morphodynamic model

Data assimilation is a sophisticated mathematical technique for combining observational data with model predictions to produce state and parameter estimates that most accurately approximate the current and future states of the true system. The technique is commonly used in atmospheric and oceanic modelling, combining empirical observations with model predictions to produce more accurate and well-calibrated forecasts. Here, we consider a novel application within a coastal environment and describe how the method can also be used to deliver improved estimates of uncertain morphodynamic model parameters. This is achieved using a technique known as state augmentation. Earlier applications of state augmentation have typically employed the 4D-Var, Kalman filter or ensemble Kalman filter assimilation schemes. Our new method is based on a computationally inexpensive 3D-Var scheme, where the specification of the error covariance matrices is crucial for success. A simple 1D model of bed-form propagation is used to demonstrate the method. The scheme is capable of recovering near-perfect parameter values and, therefore, improves the capability of our model to predict future bathymetry. Such positive results suggest the potential for application to more complex morphodynamic models.     

Data assimilation with the ensemble Kalman filter in a numerical model of the North Sea

Coastal management and maritime safety strongly rely on accurate representations of the sea state. Both dynamical models and observations provide abundant pieces of information. However, none of them provides the complete picture. The assimilation of observations into models is one way to improve our knowledge of the ocean state. Its application in coastal models remains challenging because of the wide range of temporal and spatial variabilities of the processes involved. This study investigates the assimilation of temperature profiles with the ensemble Kalman filter in 3-D North Sea simulations. The model error is represented by the standard deviation of an ensemble of model states. Parameters’ values for the ensemble generation are first computed from the misfit between the data and the model results without assimilation. Then, two square root algorithms are applied to assimilate the data. The impact of data assimilation on the simulated temperature is assessed. Results show that the ensemble Kalman filter is adequate for improving temperature forecasts in coastal areas, under adequate model error specification.     

Comparison of deterministic ensemble Kalman filters for assimilating hydrogeological data

Groundwater models are critical decision support tools for water resources management and environmental remediation. However, limitations in site characterization data and conceptual models can adversely affect the reliability of groundwater models. Therefore, there is a strong need for continuous model uncertainty reduction. Ensemble filters have recently emerged as promising high-dimensional data assimilation techniques. Two general categories of ensemble filters exist in the literature: perturbation-based and deterministic. Deterministic ensemble filters have been extensively studied for their better performance and robustness in assimilating oceanographic and atmospheric data. In hydrogeology, while a number of previous studies demonstrated the usefulness of the perturbation-based ensemble Kalman filter (EnKF) for joint parameter and state estimation, there have been few systematic studies investigating the performance of deterministic ensemble filters. This paper presents a comparative study of four commonly used deterministic ensemble filters for sequentially estimating the hydraulic conductivity parameter in low- and moderately high-dimensional groundwater models. The performance of the filters is assessed on the basis of twin experiments in which the true hydraulic conductivity field is assumed known. The test results indicate that the deterministic ensemble Kalman filter (DEnKF) is the most robust filter and achieves the best performance at relatively small ensemble sizes. Deterministic ensemble filters often make use of covariance inflation and localization to stabilize filter performance. Sensitivity studies demonstrate the effects of covariance inflation, localization, observation density, and conditioning on filter performance.     

Calibration framework for a Kalman filter applied to a groundwater model

The paper presents a novel approach to the setup of a Kalman filter by using an automatic calibration framework for estimation of the covariance matrices. The calibration consists of two sequential steps: (1) Automatic calibration of a set of covariance parameters to optimize the performance of the system and (2) adjustment of the model and observation variance to provide an uncertainty analysis relying on the data instead of ad-hoc covariance values. The method is applied to a twin-test experiment with a groundwater model and a colored noise Kalman filter. The filter is implemented in an ensemble framework. It is demonstrated that lattice sampling is preferable to the usual Monte Carlo simulation because its ability to preserve the theoretical mean reduces the size of the ensemble needed. The resulting Kalman filter proves to be efficient in correcting dynamic error and bias over the whole domain studied. The uncertainty analysis provides a reliable estimate of the error in the neighborhood of assimilation points but the simplicity of the covariance models leads to underestimation of the errors far from assimilation points.     

The Ensemble Kalman Filter for Groundwater Plume Characterization: A Case Study

The Kalman filter is an efficient data assimilation tool to refine an estimate of a state variable using measured data and the variable's correlations in space and/or time. The ensemble Kalman filter (EnKF) (Evensen 2004, 2009) is a Kalman filter variant that employs Monte Carlo analysis to define the correlations that help to refine the updated state. While use of EnKF in hydrology is somewhat limited, it has been successfully applied in other fields of engineering (e.g., oil reservoir modeling, weather forecasting). Here, EnKF is used to refine a simulated groundwater tetrachloroethylene (TCE) plume that underlies the Tooele Army Depot‐North (TEAD‐N) in Utah, based on observations of TCE in the aquifer. The resulting EnKF‐based assimilated plume is simulated forward in time to predict future plume migration. The correlations that underpin EnKF updating implicitly contain information about how the plume developed over time under the influence of complex site hydrology and variable source history, as they are predicated on multiple realizations of a well‐calibrated numerical groundwater flow and transport model. The EnKF methodology is compared to an ordinary kriging‐based assimilation method with respect to the accurate representation of plume concentrations in order to determine the relative efficacy of EnKF for water quality data assimilation.     

The Local Ensemble Transform Kalman Filter with the Weather Research and Forecasting Model: Experiments with Real Observations

The local ensemble transform Kalman filter (LETKF) is implemented with the Weather Research and Forecasting (WRF) model, and real observations are assimilated to assess the newly-developed WRF-LETKF system. The WRF model is a widely-used mesoscale numerical weather prediction model, and the LETKF is an ensemble Kalman filter (EnKF) algorithm particularly efficient in parallel computer architecture. This study aims to provide the basis of future research on mesoscale data assimilation using the WRF-LETKF system, an additional testbed to the existing EnKF systems with the WRF model used in the previous studies. The particular LETKF system adopted in this study is based on the system initially developed in 2004 and has been continuously improved through theoretical studies and wide applications to many kinds of dynamical models including realistic geophysical models. Most recent and important improvements include an adaptive covariance inflation scheme which considers the spatial and temporal inhomogeneity of inflation parameters. Experiments show that the LETKF successfully assimilates real observations and that adaptive inflation is advantageous. Additional experiments with various ensemble sizes show that using more ensemble members improves the analyses consistently.     

Data assimilation for unsaturated flow models with restart adaptive probabilistic collocation based Kalman filter

The ensemble Kalman filter (EnKF) has gained popularity in hydrological data assimilation problems. As a Monte Carlo based method, a sufficiently large ensemble size is usually required to guarantee the accuracy. As an alternative approach, the probabilistic collocation based Kalman filter (PCKF) employs the polynomial chaos expansion (PCE) to represent and propagate the uncertainties in parameters and states. However, PCKF suffers from the so-called “curse of dimensionality”. Its computational cost increases drastically with the increasing number of parameters and system nonlinearity. Furthermore, PCKF may fail to provide accurate estimations due to the joint updating scheme for strongly nonlinear models. Motivated by recent developments in uncertainty quantification and EnKF, we propose a restart adaptive probabilistic collocation based Kalman filter (RAPCKF) for data assimilation in unsaturated flow problems. During the implementation of RAPCKF, the important parameters are identified and active PCE basis functions are adaptively selected at each assimilation step; the “restart” scheme is utilized to eliminate the inconsistency between updated model parameters and states variables. The performance of RAPCKF is systematically tested with numerical cases of unsaturated flow models. It is shown that the adaptive approach and restart scheme can significantly improve the performance of PCKF. Moreover, RAPCKF has been demonstrated to be more efficient than EnKF with the same computational cost.     

Real-time assimilation of streamflow observations into a hydrological routing model: effects of model structures and updating methods

This paper comparatively assesses the performance of five data assimilation techniques for three-parameter Muskingum routing with a spatially lumped or distributed model structure. The assimilation techniques used include direct insertion (DI), nudging scheme (NS), Kalman filter (KF), ensemble Kalman filter (EnKF) and asynchronous ensemble Kalman filter (AEnKF), which are applied to river reaches in Texas and Louisiana, USA. For both lumped and distributed routing, results from KF, EnKF and AEnKF are sensitive to the error specification. As expected, DI outperformed the other models in the case of lumped modelling, while in distributed routing, KF approaches, particularly AEnKF and EnKF, performed better than DI or nudging, reflecting the benefit of updating distributed states through error covariance modelling in KF approaches. The results of this work would be useful in setting up data assimilation systems that employ increasingly abundant real-time observations using distributed hydrological routing models.     

Subsurface characterization with localized ensemble Kalman filter employing adaptive thresholding

Ensemble Kalman filter, EnKF, as a Monte Carlo sequential data assimilation method has emerged promisingly for subsurface media characterization during past decade. Due to high computational cost of large ensemble size, EnKF is limited to small ensemble set in practice. This results in appearance of spurious correlation in covariance structure leading to incorrect or probable divergence of updated realizations. In this paper, a universal/adaptive thresholding method is presented to remove and/or mitigate spurious correlation problem in the forecast covariance matrix. This method is, then, extended to regularize Kalman gain directly. Four different thresholding functions have been considered to threshold forecast covariance and gain matrices. These include hard, soft, lasso and Smoothly Clipped Absolute Deviation (SCAD) functions. Three benchmarks are used to evaluate the performances of these methods. These benchmarks include a small 1D linear model and two 2D water flooding (in petroleum reservoirs) cases whose levels of heterogeneity/nonlinearity are different. It should be noted that beside the adaptive thresholding, the standard distance dependant localization and bootstrap Kalman gain are also implemented for comparison purposes. We assessed each setup with different ensemble sets to investigate the sensitivity of each method on ensemble size. The results indicate that thresholding of forecast covariance yields more reliable performance than Kalman gain. Among thresholding function, SCAD is more robust for both covariance and gain estimation. Our analyses emphasize that not all assimilation cycles do require thresholding and it should be performed wisely during the early assimilation cycles. The proposed scheme of adaptive thresholding outperforms other methods for subsurface characterization of underlying benchmarks.     

Hydrological data assimilation with the Ensemble Square-Root-Filter: Use of streamflow observations to update model states for real-time flash flood forecasting

The objective of the study is to evaluate the potential of a data assimilation system for real-time flash flood forecasting over small watersheds by updating model states. To this end, the Ensemble Square-Root-Filter (EnSRF) based on the Ensemble Kalman Filter (EnKF) technique was coupled to a widely used conceptual rainfall-runoff model called HyMOD. Two small watersheds susceptible to flash flooding from America and China were selected in this study. The modeling and observational errors were considered in the framework of data assimilation, followed by an ensemble size sensitivity experiment. Once the appropriate model error and ensemble size was determined, a simulation study focused on the performance of a data assimilation system, based on the correlation between streamflow observation and model states, was conducted. The EnSRF method was implemented within HyMOD and results for flash flood forecasting were analyzed, where the calibrated streamflow simulation without state updating was treated as the benchmark or nature run. Results for twenty-four flash-flood events in total from the two watersheds indicated that the data assimilation approach effectively improved the predictions of peak flows and the hydrographs in general. This study demonstrated the benefit and efficiency of implementing data assimilation into a hydrological model to improve flash flood forecasting over small, instrumented basins with potential application to real-time alert systems.     

An investigation of ensemble-based assimilation of satellite altimetry and tide gauge data in storm surge prediction

Cyclogenesis and long-fetched winds along the southeastern coast of South America may lead to floods in populated areas, as the Buenos Aires Province, with important economic and social impacts. A numerical model (SMARA) has already been implemented in the region to forecast storm surges. The propagation time of the surge in such extensive and shallow area allows the detection of anomalies based on observations from several hours up to the order of a day prior to the event. Here, we investigate the impact and potential benefit of storm surge level data assimilation into the SMARA model, with the objective of improving the forecast. In the experiments, the surface wind stress from an ensemble prediction system drives a storm surge model ensemble, based on the operational 2-D depth-averaged SMARA model. A 4-D Local Ensemble Transform Kalman Filter (4D-LETKF) initializes the ensemble in a 6-h cycle, assimilating the very few tide gauge observations available along the northern coast and satellite altimeter data. The sparse coverage of the altimeters is a challenge to data assimilation; however, the 4D-LETKF evolving covariance of the ensemble perturbations provides realistic cross-track analysis increments. Improvements on the forecast ensemble mean show the potential of an effective use of the sparse satellite altimeter and tidal gauges observations in the data assimilation prototype. Furthermore, the effects of the localization scale and of the observational errors of coastal altimetry and tidal gauges in the data assimilation approach are assessed.     

An approach to handling non-Gaussianity of parameters and state variables in ensemble Kalman filtering

The ensemble Kalman filter (EnKF) is a commonly used real-time data assimilation algorithm in various disciplines. Here, the EnKF is applied, in a hydrogeological context, to condition log-conductivity realizations on log-conductivity and transient piezometric head data. In this case, the state vector is made up of log-conductivities and piezometric heads over a discretized aquifer domain, the forecast model is a groundwater flow numerical model, and the transient piezometric head data are sequentially assimilated to update the state vector. It is well known that all Kalman filters perform optimally for linear forecast models and a multiGaussian-distributed state vector. Of the different Kalman filters, the EnKF provides a robust solution to address non-linearities; however, it does not handle well non-Gaussian state-vector distributions. In the standard EnKF, as time passes and more state observations are assimilated, the distributions become closer to Gaussian, even if the initial ones are clearly non-Gaussian. A new method is proposed that transforms the original state vector into a new vector that is univariate Gaussian at all times. Back transforming the vector after the filtering ensures that the initial non-Gaussian univariate distributions of the state-vector components are preserved throughout. The proposed method is based in normal-score transforming each variable for all locations and all time steps. This new method, termed the normal-score ensemble Kalman filter (NS-EnKF), is demonstrated in a synthetic bimodal aquifer resembling a fluvial deposit, and it is compared to the standard EnKF. The proposed method performs better than the standard EnKF in all aspects analyzed (log-conductivity characterization and flow and transport predictions).     

Bias aware Kalman filters: Comparison and improvements

This paper reviews two different approaches that have been proposed to tackle the problems of model bias with the Kalman filter: the use of a colored noise model and the implementation of a separate bias filter. Both filters are implemented with and without feedback of the bias into the model state. The colored noise filter formulation is extended to correct both time correlated and uncorrelated model error components. A more stable version of the separate filter without feedback is presented. The filters are implemented in an ensemble framework using Latin hypercube sampling. The techniques are illustrated on a simple one-dimensional groundwater problem. The results show that the presented filters outperform the standard Kalman filter and that the implementations with bias feedback work in more general conditions than the implementations without feedback.     

Coupled assimilation for an intermediated coupled ENSO prediction model

The value of coupled assimilation is discussed using an intermediate coupled model in which the wind stress is the only atmospheric state which is slavery to model sea surface temperature (SST). In the coupled assimilation analysis, based on the coupled wind–ocean state covariance calculated from the coupled state ensemble, the ocean state is adjusted by assimilating wind data using the ensemble Kalman filter. As revealed by a series of assimilation experiments using simulated observations, the coupled assimilation of wind observations yields better results than the assimilation of SST observations. Specifically, the coupled assimilation of wind observations can help to improve the accuracy of the surface and subsurface currents because the correlation between the wind and ocean currents is stronger than that between SST and ocean currents in the equatorial Pacific. Thus, the coupled assimilation of wind data can decrease the initial condition errors in the surface/subsurface currents that can significantly contribute to SST forecast errors. The value of the coupled assimilation of wind observations is further demonstrated by comparing the prediction skills of three 12-year (1997–2008) hindcast experiments initialized by the ocean-only assimilation scheme that assimilates SST observations, the coupled assimilation scheme that assimilates wind observations, and a nudging scheme that nudges the observed wind stress data, respectively. The prediction skills of two assimilation schemes are significantly better than those of the nudging scheme. The prediction skills of assimilating wind observations are better than assimilating SST observations. Assimilating wind observations for the 2007/2008 La Niña event triggers better predictions, while assimilating SST observations fails to provide an early warning for that event.     

基于集合卡尔曼滤波的湖泊富营养化模型Delft3D-BLOOM数据同化

富营养化模型是进行湖泊水环境质量预测和管理的重要工具,然而模型客观存在的误差一直是应用者关心的重要问题.数据同化作为连接观测数据与数值模型的重要方法,可以有效提高模型的准确性.集合卡尔曼滤波(En KF)是众多数据同化算法中应用最为广泛的一种,可进行非线性系统的数据同化,并能有效降低数据同化的计算量.本研究以太湖作为具体实例,选择Delft3D-BLOOM作为富营养化模型,在数值实验确定En KF集合数为100、观测误差方差为1%、模拟误差方差为10%的基础上分别进行模型状态变量同化以及状态变量与关键参数同步同化.结果显示,仅同化状态变量时,模型预测精度有所增加;同时同化状态变量和关键参数时,可显著提升模型在湖泊水环境质量预测中的精度.该研究为应用集合卡尔曼滤波以提高复杂的湖库富营养化模型模拟精度提供了有效的方法.     

Parameter estimation in ensemble based snow data assimilation: A synthetic study

Estimating erroneous parameters in ensemble based snow data assimilation system has been given little attention in the literature. Little is known about the related methods’ effectiveness, performance, and sensitivity to other error sources such as model structural error. This research tackles these questions by running synthetic one-dimensional snow data assimilation with the ensemble Kalman filter (EnKF), in which both state and parameter are simultaneously updated. The first part of the paper investigates the effectiveness of this parameter estimation approach in a perfect-model-structure scenario, and the second part focuses on its dependence on model structure error. The results from first part research demonstrate the advantages of this parameter estimation approach in reducing the systematic error of snow water equivalent (SWE) estimates, and retrieving the correct parameter value. The second part results indicate that, at least in our experiment, there is an evident dependence of parameter search convergence on model structural error. In the imperfect-model-structure run, the parameter search diverges, although it can simulate the state variable well. This result suggest that, good data assimilation performance in estimating state variables is not a sufficient indicator of reliable parameter retrieval in the presence of model structural error. The generality of this conclusion needs to be tested by data assimilation experiments with more complex structural error configurations.     

Efficient ensemble forecasting of marine ecology with clustered 1D models and statistical lateral exchange: application to the Red Sea

Forecasting the state of large marine ecosystems is important for many economic and public health applications. However, advanced three-dimensional (3D) ecosystem models, such as the European Regional Seas Ecosystem Model (ERSEM), are computationally expensive, especially when implemented within an ensemble data assimilation system requiring several parallel integrations. As an alternative to 3D ecological forecasting systems, we propose to implement a set of regional one-dimensional (1D) water-column ecological models that run at a fraction of the computational cost. The 1D model domains are determined using a Gaussian mixture model (GMM)-based clustering method and satellite chlorophyll-a (Chl-a) data. Regionally averaged Chl-a data is assimilated into the 1D models using the singular evolutive interpolated Kalman (SEIK) filter. To laterally exchange information between subregions and improve the forecasting skills, we introduce a new correction step to the assimilation scheme, in which we assimilate a statistical forecast of future Chl-a observations based on information from neighbouring regions. We apply this approach to the Red Sea and show that the assimilative 1D ecological models can forecast surface Chl-a concentration with high accuracy. The statistical assimilation step further improves the forecasting skill by as much as 50%. This general approach of clustering large marine areas and running several interacting 1D ecological models is very flexible. It allows many combinations of clustering, filtering and regression technics to be used and can be applied to build efficient forecasting systems in other large marine ecosystems.