An approach to handling non-Gaussianity of parameters and state variables in ensemble Kalman filtering

The ensemble Kalman filter (EnKF) is a commonly used real-time data assimilation algorithm in various disciplines. Here, the EnKF is applied, in a hydrogeological context, to condition log-conductivity realizations on log-conductivity and transient piezometric head data. In this case, the state vector is made up of log-conductivities and piezometric heads over a discretized aquifer domain, the forecast model is a groundwater flow numerical model, and the transient piezometric head data are sequentially assimilated to update the state vector. It is well known that all Kalman filters perform optimally for linear forecast models and a multiGaussian-distributed state vector. Of the different Kalman filters, the EnKF provides a robust solution to address non-linearities; however, it does not handle well non-Gaussian state-vector distributions. In the standard EnKF, as time passes and more state observations are assimilated, the distributions become closer to Gaussian, even if the initial ones are clearly non-Gaussian. A new method is proposed that transforms the original state vector into a new vector that is univariate Gaussian at all times. Back transforming the vector after the filtering ensures that the initial non-Gaussian univariate distributions of the state-vector components are preserved throughout. The proposed method is based in normal-score transforming each variable for all locations and all time steps. This new method, termed the normal-score ensemble Kalman filter (NS-EnKF), is demonstrated in a synthetic bimodal aquifer resembling a fluvial deposit, and it is compared to the standard EnKF. The proposed method performs better than the standard EnKF in all aspects analyzed (log-conductivity characterization and flow and transport predictions).     

Snow water equivalent prediction using Bayesian data assimilation methods

Using the U.S. National Weather Service’s SNOW-17 model, this study compares common sequential data assimilation methods, the ensemble Kalman filter (EnKF), the ensemble square root filter (EnSRF), and four variants of the particle filter (PF), to predict seasonal snow water equivalent (SWE) within a small watershed near Lake Tahoe, California. In addition to SWE estimation, the various data assimilation methods are used to estimate five of the most sensitive parameters of SNOW-17 by allowing them to evolve with the dynamical system. Unlike Kalman filters, particle filters do not require Gaussian assumptions for the posterior distribution of the state variables. However, the likelihood function used to scale particle weights is often assumed to be Gaussian. This study evaluates the use of an empirical cumulative distribution function (ECDF) based on the Kaplan–Meier survival probability method to compute particle weights. These weights are then used in different particle filter resampling schemes. Detailed analyses are conducted for synthetic and real data assimilation and an assessment of the procedures is made. The results suggest that the particle filter, especially the empirical likelihood variant, is superior to the ensemble Kalman filter based methods for predicting model states, as well as model parameters.     

Prediction and predictability of a catastrophic local extreme precipitation event through cloud-resolving ensemble analysis and forecasting with Doppler radar observations

Local extreme rain usually resulted in disasters such as flash floods and landslides. Upon today, it is still one of the most difficult tasks for operational weather forecast centers to predict those events accurately. In this paper, we simulate an extreme precipitation event with ensemble Kalman filter (EnKF) assimilation of Doppler radial-velocity observations, and analyze the uncertainties of the assimilation. The results demonstrate that, without assimilation radar data, neither a single initialization of deterministic forecast nor an ensemble forecast with adding perturbations or multiple physical parameterizations can predict the location of strong precipitation. However, forecast was significantly improved with assimilation of radar data, especially the location of the precipitation. The direct cause of the improvement is the buildup of a deep mesoscale convection system with EnKF assimilation of radar data. Under a large scale background favorable for mesoscale convection, efficient perturbations of upstream mid-low level meridional wind and moisture are key factors for the assimilation and forecast. Uncertainty still exists for the forecast of this case due to its limited predictability. Both the difference of large scale initial fields and the difference of analysis obtained from EnKF assimilation due to small amplitude of initial perturbations could have critical influences to the event's prediction. Forecast could be improved through more cycles of EnKF assimilation. Sensitivity tests also support that more accurate forecasts are expected through improving numerical models and observations.     

Radiance data assimilation for operational snow and streamflow forecasting

Estimation of seasonal snowpack, in mountainous regions, is crucial for accurate streamflow prediction. This paper examines the ability of data assimilation (DA) of remotely sensed microwave radiance data to improve snow water equivalent prediction, and ultimately operational streamflow forecasts. Operational streamflow forecasts in the National Weather Service River Forecast Center (NWSRFC) are produced with a coupled SNOW17 (snow model) and SACramento Soil Moisture Accounting (SAC-SMA) model. A comparison of two assimilation techniques, the ensemble Kalman filter (EnKF) and the particle filter (PF), is made using a coupled SNOW17 and the microwave emission model for layered snow pack (MEMLS) model to assimilate microwave radiance data. Microwave radiance data, in the form of brightness temperature (TB), is gathered from the advanced microwave scanning radiometer-earth observing system (AMSR-E) at the 36.5 GHz channel. SWE prediction is validated in a synthetic experiment. The distribution of snowmelt from an experiment with real data is then used to run the SAC-SMA model. Several scenarios on state or joint state-parameter updating with TB data assimilation to SNOW-17 and SAC-SMA models were analyzed, and the results show potential benefit for operational streamflow forecasting.     

Comparison of deterministic ensemble Kalman filters for assimilating hydrogeological data

Groundwater models are critical decision support tools for water resources management and environmental remediation. However, limitations in site characterization data and conceptual models can adversely affect the reliability of groundwater models. Therefore, there is a strong need for continuous model uncertainty reduction. Ensemble filters have recently emerged as promising high-dimensional data assimilation techniques. Two general categories of ensemble filters exist in the literature: perturbation-based and deterministic. Deterministic ensemble filters have been extensively studied for their better performance and robustness in assimilating oceanographic and atmospheric data. In hydrogeology, while a number of previous studies demonstrated the usefulness of the perturbation-based ensemble Kalman filter (EnKF) for joint parameter and state estimation, there have been few systematic studies investigating the performance of deterministic ensemble filters. This paper presents a comparative study of four commonly used deterministic ensemble filters for sequentially estimating the hydraulic conductivity parameter in low- and moderately high-dimensional groundwater models. The performance of the filters is assessed on the basis of twin experiments in which the true hydraulic conductivity field is assumed known. The test results indicate that the deterministic ensemble Kalman filter (DEnKF) is the most robust filter and achieves the best performance at relatively small ensemble sizes. Deterministic ensemble filters often make use of covariance inflation and localization to stabilize filter performance. Sensitivity studies demonstrate the effects of covariance inflation, localization, observation density, and conditioning on filter performance.     

Dual state–parameter estimation of hydrological models using ensemble Kalman filter

Hydrologic models are twofold: models for understanding physical processes and models for prediction. This study addresses the latter, which modelers use to predict, for example, streamflow at some future time given knowledge of the current state of the system and model parameters. In this respect, good estimates of the parameters and state variables are needed to enable the model to generate accurate forecasts. In this paper, a dual state–parameter estimation approach is presented based on the Ensemble Kalman Filter (EnKF) for sequential estimation of both parameters and state variables of a hydrologic model. A systematic approach for identification of the perturbation factors used for ensemble generation and for selection of ensemble size is discussed. The dual EnKF methodology introduces a number of novel features: (1) both model states and parameters can be estimated simultaneously; (2) the algorithm is recursive and therefore does not require storage of all past information, as is the case in the batch calibration procedures; and (3) the various sources of uncertainties can be properly addressed, including input, output, and parameter uncertainties. The applicability and usefulness of the dual EnKF approach for ensemble streamflow forecasting is demonstrated using a conceptual rainfall-runoff model.     

BMA集合预报在淮河流域应用及参数规律初探

以淮河流域吴家渡水文站作为试验站点,采用基于贝叶斯平均法(BMA)的集合预报模型处理来源于马斯京根法、一维水动力学方法、BPNN(Back Propagation Neural Network)的预报流量序列,通过分析BMA的参数以及其预报结果,对各方法在淮河典型站点流量预报中的适用性进行验证与分析.经2003—2016年19场洪水模拟检验可知,BMA模型能够有效避免模型选择带来的洪水预报误差放大效应,可以提供高精度、鲁棒性强的洪水预报结果.通过进一步比较各模型统计最优的频率与BMA权重值之间的相关性,发现权重值不适用于对单场洪水预报精度评定,而适用于描述多场洪水预报中,模型为最优的统计频率;基于大量先验信息,提前获取BMA的权重等参数,将是指导模型选择、降低洪水预报不确定性、改进洪水预报技术的有效手段.     

Data assimilation for distributed hydrological catchment modeling via ensemble Kalman filter

Catchment scale hydrological models are critical decision support tools for water resources management and environment remediation. However, the reliability of hydrological models is inevitably affected by limited measurements and imperfect models. Data assimilation techniques combine complementary information from measurements and models to enhance the model reliability and reduce predictive uncertainties. As a sequential data assimilation technique, the ensemble Kalman filter (EnKF) has been extensively studied in the earth sciences for assimilating in-situ measurements and remote sensing data. Although the EnKF has been demonstrated in land surface data assimilations, there are no systematic studies to investigate its performance in distributed modeling with high dimensional states and parameters. In this paper, we present an assessment on the EnKF with state augmentation for combined state-parameter estimation on the basis of a physical-based hydrological model, Soil and Water Assessment Tool (SWAT). Through synthetic simulation experiments, the capability of the EnKF is demonstrated by assimilating the runoff and other measurements, and its sensitivities are analyzed with respect to the error specification, the initial realization and the ensemble size. It is found that the EnKF provides an efficient approach for obtaining a set of acceptable model parameters and satisfactory runoff, soil water content and evapotranspiration estimations. The EnKF performance could be improved after augmenting with other complementary data, such as soil water content and evapotranspiration from remote sensing retrieval. Sensitivity studies demonstrate the importance of consistent error specification and the potential with small ensemble size in the data assimilation system.     

Assimilating transient groundwater flow data via a localized ensemble Kalman filter to calibrate a heterogeneous conductivity field

A localized ensemble Kalman filter (EnKF) method is developed to assimilate transient flow data to calibrate a heterogeneous conductivity field. To update conductivity value at a point in a study domain, instead of assimilating all the measurements in the study domain, only limited measurement data in an area around the point are used for the conductivity updating in the localized EnKF method. The localized EnKF is proposed to solve the problems of the filter divergence usually existing in a data assimilation method without localization. The developed method is applied, in a synthetical two dimensional case, to calibrate a heterogeneous conductivity field by assimilating transient hydraulic head data. The simulations by the data assimilation with and without localized EnKF are compared. The study results indicate that the hydraulic conductivity field can be updated efficiently by the localized EnKF, while it cannot be by the EnKF. The covariance inflation and localization are found to solve the problem of the filter divergence efficiently. In comparison with the EnKF method without localization, the localized EnKF method needs smaller ensemble size to achieve stabilized results. The simulation results by the localized EnKF method are much more sensitive to conductivity correlation length than to the localization radius. The developed localized EnKF method provides an approach to improve EnKF method in conductivity calibration.     

The Local Ensemble Transform Kalman Filter with the Weather Research and Forecasting Model: Experiments with Real Observations

The local ensemble transform Kalman filter (LETKF) is implemented with the Weather Research and Forecasting (WRF) model, and real observations are assimilated to assess the newly-developed WRF-LETKF system. The WRF model is a widely-used mesoscale numerical weather prediction model, and the LETKF is an ensemble Kalman filter (EnKF) algorithm particularly efficient in parallel computer architecture. This study aims to provide the basis of future research on mesoscale data assimilation using the WRF-LETKF system, an additional testbed to the existing EnKF systems with the WRF model used in the previous studies. The particular LETKF system adopted in this study is based on the system initially developed in 2004 and has been continuously improved through theoretical studies and wide applications to many kinds of dynamical models including realistic geophysical models. Most recent and important improvements include an adaptive covariance inflation scheme which considers the spatial and temporal inhomogeneity of inflation parameters. Experiments show that the LETKF successfully assimilates real observations and that adaptive inflation is advantageous. Additional experiments with various ensemble sizes show that using more ensemble members improves the analyses consistently.     

Data assimilation for unsaturated flow models with restart adaptive probabilistic collocation based Kalman filter

The ensemble Kalman filter (EnKF) has gained popularity in hydrological data assimilation problems. As a Monte Carlo based method, a sufficiently large ensemble size is usually required to guarantee the accuracy. As an alternative approach, the probabilistic collocation based Kalman filter (PCKF) employs the polynomial chaos expansion (PCE) to represent and propagate the uncertainties in parameters and states. However, PCKF suffers from the so-called “curse of dimensionality”. Its computational cost increases drastically with the increasing number of parameters and system nonlinearity. Furthermore, PCKF may fail to provide accurate estimations due to the joint updating scheme for strongly nonlinear models. Motivated by recent developments in uncertainty quantification and EnKF, we propose a restart adaptive probabilistic collocation based Kalman filter (RAPCKF) for data assimilation in unsaturated flow problems. During the implementation of RAPCKF, the important parameters are identified and active PCE basis functions are adaptively selected at each assimilation step; the “restart” scheme is utilized to eliminate the inconsistency between updated model parameters and states variables. The performance of RAPCKF is systematically tested with numerical cases of unsaturated flow models. It is shown that the adaptive approach and restart scheme can significantly improve the performance of PCKF. Moreover, RAPCKF has been demonstrated to be more efficient than EnKF with the same computational cost.     

Real-time assimilation of streamflow observations into a hydrological routing model: effects of model structures and updating methods

This paper comparatively assesses the performance of five data assimilation techniques for three-parameter Muskingum routing with a spatially lumped or distributed model structure. The assimilation techniques used include direct insertion (DI), nudging scheme (NS), Kalman filter (KF), ensemble Kalman filter (EnKF) and asynchronous ensemble Kalman filter (AEnKF), which are applied to river reaches in Texas and Louisiana, USA. For both lumped and distributed routing, results from KF, EnKF and AEnKF are sensitive to the error specification. As expected, DI outperformed the other models in the case of lumped modelling, while in distributed routing, KF approaches, particularly AEnKF and EnKF, performed better than DI or nudging, reflecting the benefit of updating distributed states through error covariance modelling in KF approaches. The results of this work would be useful in setting up data assimilation systems that employ increasingly abundant real-time observations using distributed hydrological routing models.     

Application of ensemble kalman filter to geophysical parameters retrieval in remote sensing:A case study of kernel-driven BRDF model inversion

Remote sensing can provide multi-spatial resolution, multi-temporal resolution multi-spectral band and multi-angular data for the observation of land surface. At present, one of research focuses is how to make the best of these data to retrieve geophysical parameters in conjunction with their a priori knowledge and simul-taneously consider the influence of data uncertainties on inversion results[1-5]. The essence of remote sensing lies in inversion. It is difficult to precisely retrieve parame…     

The role of prior model calibration on predictions with ensemble Kalman filter

This paper, based on a real world case study (Limmat aquifer, Switzerland), compares inverse groundwater flow models calibrated with specified numbers of monitoring head locations. These models are updated in real time with the ensemble Kalman filter (EnKF) and the prediction improvement is assessed in relation to the amount of monitoring locations used for calibration and updating. The prediction errors of the models calibrated in transient state are smaller if the amount of monitoring locations used for the calibration is larger. For highly dynamic groundwater flow systems a transient calibration is recommended as a model calibrated in steady state can lead to worse results than a noncalibrated model with a well-chosen uniform conductivity. The model predictions can be improved further with the assimilation of new measurement data from on-line sensors with the EnKF. Within all the studied models the reduction of 1-day hydraulic head prediction error (in terms of mean absolute error [MAE]) with EnKF lies between 31% (assimilation of head data from 5 locations) and 72% (assimilation of head data from 85 locations). The largest prediction improvements are expected for models that were calibrated with only a limited amount of historical information. It is worthwhile to update the model even with few monitoring locations as it seems that the error reduction with EnKF decreases exponentially with the amount of monitoring locations used. These results prove the feasibility of data assimilation with EnKF also for a real world case and show that improved predictions of groundwater levels can be obtained.     

A semi-empirical wind set-up forecasting model for Lake Champlain

The precision of Lake Champlain's water level estimation is a key component in the flood forecasting process for the Richelieu River. Hydrological models do not typically take into consideration the effects of the wind on the water level (also known as the wind set-up). The objective of this study is to create an empirical wind set-up forecast model for Lake Champlain during high wind events. The proposed model uses wind speed and direction across the Lake, as well as wind gusts as inputs. The model is calibrated to a subset of observations and evaluated on an independent sample, considering four wind speed bins. It is tested and compared to a variant of the Zuider Zee equation on 20 wind set-up events that occurred between 2017 and 2019 using hindcast data from five different numerical weather prediction systems (GDPS, RDPS, HRDPS, NOAA and ECMWF). A quantile mapping-based forecast calibration scheme is implemented for each of the forecast products to correct their biases. Results show that events are successfully predicted by the proposed model at least 72 h in advance. These results are better than the other comparative models found in the literature and tested herein. Overall, significant improvements are obtained by including wind speed and wind gusts from different weather stations.     

Subsurface characterization with localized ensemble Kalman filter employing adaptive thresholding

Ensemble Kalman filter, EnKF, as a Monte Carlo sequential data assimilation method has emerged promisingly for subsurface media characterization during past decade. Due to high computational cost of large ensemble size, EnKF is limited to small ensemble set in practice. This results in appearance of spurious correlation in covariance structure leading to incorrect or probable divergence of updated realizations. In this paper, a universal/adaptive thresholding method is presented to remove and/or mitigate spurious correlation problem in the forecast covariance matrix. This method is, then, extended to regularize Kalman gain directly. Four different thresholding functions have been considered to threshold forecast covariance and gain matrices. These include hard, soft, lasso and Smoothly Clipped Absolute Deviation (SCAD) functions. Three benchmarks are used to evaluate the performances of these methods. These benchmarks include a small 1D linear model and two 2D water flooding (in petroleum reservoirs) cases whose levels of heterogeneity/nonlinearity are different. It should be noted that beside the adaptive thresholding, the standard distance dependant localization and bootstrap Kalman gain are also implemented for comparison purposes. We assessed each setup with different ensemble sets to investigate the sensitivity of each method on ensemble size. The results indicate that thresholding of forecast covariance yields more reliable performance than Kalman gain. Among thresholding function, SCAD is more robust for both covariance and gain estimation. Our analyses emphasize that not all assimilation cycles do require thresholding and it should be performed wisely during the early assimilation cycles. The proposed scheme of adaptive thresholding outperforms other methods for subsurface characterization of underlying benchmarks.     

Impact of assimilating Taiwan’s coastal radar radial velocity on forecasting Typhoon Morakot (2009) in southeastern China using a WRF-based EnKF

This study explores for the first time the impact of assimilating radial velocity (Vr) observations from a single or multiple Taiwan’s coastal radars on tropical cyclone (TC) forecasting after landfall in the Chinese mainland by using a Weather Research and Forecasting model (WRF)-based ensemble Kalman filter (EnKF) data assimilation system. Typhoon Morakot (2009), which caused widespread damage in the southeastern coastal regions of the mainland after devastating Taiwan, was chosen as a case study. The results showed that assimilating Taiwan’s radar Vr data improved environmental field and steering flow and produced a more realistic TC position and structure in the final EnKF cycling analysis. Thus, the subsequent TC track and rainfall forecasts in southeastern China were improved. In addition, better observations of the TC inner core by Taiwan’s radar was a primary factor in improving TC rainfall forecast in the Chinese mainland.     

Future developments in modelling and monitoring of volcanic ash clouds: outcomes from the first IAVCEI-WMO workshop on Ash Dispersal Forecast and Civil Aviation

As a result of the serious consequences of the 2010 Eyjafjallajökull eruption (Iceland) on civil aviation, 52 volcanologists, meteorologists, atmospheric dispersion modellers and space and ground-based monitoring specialists from 12 different countries (including representatives from 6 Volcanic Ash Advisory Centres and related institutions) gathered to discuss the needs of the ash dispersal modelling community, investigate new data-acquisition strategies (i.e. quantitative measurements and observations) and discuss how to improve communication between the research community and institutions with an operational mandate. Based on a dedicated benchmark exercise and on 3 days of in-depth discussion, recommendations have been made for future model improvements, new strategies of ash cloud forecasting, multidisciplinary data acquisition and more efficient communication between different communities. Issues addressed in the workshop include ash dispersal modelling, uncertainty, ensemble forecasting, combining dispersal models and observations, sensitivity analysis, model variability, data acquisition, pre-eruption forecasting, first simulation and data assimilation, research priorities and new communication strategies to improve information flow and operational routines. As a main conclusion, model developers, meteorologists, volcanologists and stakeholders need to work closely together to develop new and improved strategies for ash dispersal forecasting and, in particular, to: (1) improve the definition of the source term, (2) design models and forecasting strategies that can better characterize uncertainties, (3) explore and identify the best ensemble strategies that can be adapted to ash dispersal forecasting, (4) identify optimized strategies for the combination of models and observations and (5) implement new critical operational strategies.     

Parameter estimation of subsurface flow models using iterative regularized ensemble Kalman filter

A new parameter estimation algorithm based on ensemble Kalman filter (EnKF) is developed. The developed algorithm combined with the proposed problem parametrization offers an efficient parameter estimation method that converges using very small ensembles. The inverse problem is formulated as a sequential data integration problem. Gaussian process regression is used to integrate the prior knowledge (static data). The search space is further parameterized using Karhunen–Loève expansion to build a set of basis functions that spans the search space. Optimal weights of the reduced basis functions are estimated by an iterative regularized EnKF algorithm. The filter is converted to an optimization algorithm by using a pseudo time-stepping technique such that the model output matches the time dependent data. The EnKF Kalman gain matrix is regularized using truncated SVD to filter out noisy correlations. Numerical results show that the proposed algorithm is a promising approach for parameter estimation of subsurface flow models.