Error covariance modeling in sequential data assimilation

 The efficiency of a sequential data assimilation scheme relies on the capability to describe the error covariance. This aspect is all the more relevant if one needs accurate statistics on the estimation error. Frequently an ad hoc function depending on a few parameters is proposed, and these parameters are tuned, estimated or updated. This usually requires that the covariance is second-order stationary (i.e. depends only on the distance between two points). In this paper, we discuss this feature and show that even in simple applications (such as one-dimensional hydrodynamics), this assumption does not hold and may lead to poorly described estimation errors. We propose a method relying on the analysis of the error term and the use of the hydrodynamical model to generate one part of the covariance function, the other part being modeled using a second-order stationary approach. This method is discussed using a twin experiment in the case where a physical parameter is erroneous, and improves significantly the results: the model bias is strongly reduced and the estimation error is well described. Moreover, it enables a better adaptation of the Kalman gain to the actual estimation error.     

Flow relevant covariance localization during dynamic data assimilation using EnKF

Multiphase dynamic data integration into high resolution subsurface models is an integral aspect of reservoir and groundwater management strategies and uncertainty assessment. Over the past two decades, advances in computing and the development and implementation of robust algorithms for automatic history matching have considerably reduced the time and effort associated with subsurface characterization and reduced the subjectivity associated with manual model calibration. However, reliable and accurate subsurface characterization continues to be challenging due to the large number of model unknowns to be estimated using a relatively smaller set of measurements. For ensemble-based methods in particular, the difficulties are compounded by the need for a large number of model replicates to estimate sample-based statistical measures, specifically the covariances and cross-covariances that directly impact the spread of information from the measurement locations to the model parameters. Statistical noise resulting from modest ensemble sizes can overwhelm and degrade the model updates leading to geologically inconsistent subsurface models. In this work we propose to address the difficulties in the implementation of the ensemble Kalman filter (EnKF) for operational data integration problems. The methods described here use streamline-derived information to identify regions within the reservoir that will have a maximum impact on the dynamic response. This is achieved through spatial localization of the sample-based cross-covariance estimates between the measurements and the model unknowns using streamline trajectories. We illustrate the approach with a synthetic example and a large field-study that demonstrate the difficulties with the traditional EnKF implementation. In both the numerical experiments, it is shown that these challenges are addressed using flow relevant conditioning of the cross-covariance matrix. By mitigating sampling error in the cross-covariance estimates, the proposed approach provides significant computational savings through the use of modest ensemble sizes, and consequently offers the opportunity for use with large field-scale groundwater and reservoir characterization studies.     

Error covariance calculation for forecast bias estimation in hydrologic data assimilation

To date, an outstanding issue in hydrologic data assimilation is a proper way of dealing with forecast bias. A frequently used method to bypass this problem is to rescale the observations to the model climatology. While this approach improves the variability in the modeled soil wetness and discharge, it is not designed to correct the results for any bias. Alternatively, attempts have been made towards incorporating dynamic bias estimates into the assimilation algorithm. Persistent bias models are most often used to propagate the bias estimate, where the a priori forecast bias error covariance is calculated as a constant fraction of the unbiased a priori state error covariance. The latter approach is a simplification to the explicit propagation of the bias error covariance. The objective of this paper is to examine to which extent the choice for the propagation of the bias estimate and its error covariance influence the filter performance. An Observation System Simulation Experiment (OSSE) has been performed, in which ground water storage observations are assimilated into a biased conceptual hydrologic model. The magnitudes of the forecast bias and state error covariances are calibrated by optimizing the innovation statistics of groundwater storage. The obtained bias propagation models are found to be identical to persistent bias models. After calibration, both approaches for the estimation of the forecast bias error covariance lead to similar results, with a realistic attribution of error variances to the bias and state estimate, and significant reductions of the bias in both the estimates of groundwater storage and discharge. Overall, the results in this paper justify the use of the traditional approach for online bias estimation with a persistent bias model and a simplified forecast bias error covariance estimation.     

Ocean satellite data assimilation experiments in FIO-ESM using ensemble adjustment Kalman filter

Using Ensemble Adjustment Kalman Filter (EAKF), two types of ocean satellite datasets were assimilated into the First Institute of Oceanography Earth System Model (FIO-ESM), v1.0. One control experiment without data assimilation and four assimilation experiments were conducted. All the experiments were ensemble runs for 1-year period and each ensemble started from different initial conditions. One assimilation experiment was designed to assimilate sea level anomaly (SLA); another, to assimilate sea surface temperature (SST); and the other two assimilation experiments were designed to assimilate both SLA and SST but in different orders. To examine the effects of data assimilation, all the results were compared with an objective analysis dataset of EN3. Different from the ocean model without coupling, the momentum and heat fluxes were calculated via air-sea coupling in FIO-ESM, which makes the relations among variables closer to the reality. The outputs after the assimilation of satellite data were improved on the whole, especially at depth shallower than 1000 m. The effects due to the assimilation of different kinds of satellite datasets were somewhat different. The improvement due to SST assimilation was greater near the surface, while the improvement due to SLA assimilation was relatively great in the subsurface. The results after the assimilation of both SLA and SST were much better than those only assimilated one kind of dataset, but the difference due to the assimilation order of the two kinds of datasets was not significant.     

Root-zone soil moisture estimation from assimilation of downscaled Soil Moisture and Ocean Salinity data

The crucial role of root-zone soil moisture is widely recognized in land–atmosphere interaction, with direct practical use in hydrology, agriculture and meteorology. But it is difficult to estimate the root-zone soil moisture accurately because of its space-time variability and its nonlinear relationship with surface soil moisture. Typically, direct satellite observations at the surface are extended to estimate the root-zone soil moisture through data assimilation. But the results suffer from low spatial resolution of the satellite observation. While advances have been made recently to downscale the satellite soil moisture from Soil Moisture and Ocean Salinity (SMOS) mission using methods such as the Disaggregation based on Physical And Theoretical scale Change (DisPATCh), the assimilation of such data into high spatial resolution land surface models has not been examined to estimate the root-zone soil moisture. Consequently, this study assimilates the 1-km DisPATCh surface soil moisture into the Joint UK Land Environment Simulator (JULES) to better estimate the root-zone soil moisture. The assimilation is demonstrated using the advanced Evolutionary Data Assimilation (EDA) procedure for the Yanco area in south eastern Australia. When evaluated using in-situ OzNet soil moisture, the open loop was found to be 95% as accurate as the updated output, with the updated estimate improving the DisPATCh data by 14%, all based on the root mean square error (RMSE). Evaluation of the root-zone soil moisture with in-situ OzNet data found the updated output to improve the open loop estimate by 34% for the 0–30 cm soil depth, 59% for the 30–60 cm soil depth, and 63% for the 60–90 cm soil depth, based on RMSE. The increased performance of the updated output over the open loop estimate is associated with (i) consistent estimation accuracy across the three soil depths for the updated output, and (ii) the deterioration of the open loop output for deeper soil depths. Thus, the findings point to a combined positive impact from the DisPATCh data and the EDA procedure, which together provide an improved soil moisture with consistent accuracy both at the surface and at the root-zone.     

Comparison of reduced-order, sequential and variational data assimilation methods in the tropical Pacific Ocean

This paper presents a comparison of two reduced-order, sequential, and variational data assimilation methods: the singular evolutive extended Kalman filter (SEEK) and the reduced 4D-Var (R-4D-Var). A hybridization of the two, combining the variational framework and the sequential evolution of covariance matrices, is also preliminarily investigated and assessed in the same experimental conditions. The comparison is performed using the twin-experiment approach on a model of the tropical Pacific domain. The assimilated data are simulated temperature profiles at the locations of the TAO/TRITON array moorings. It is shown that, in a quasilinear regime, both methods produce similarly good results. However, the hybrid approach provides slightly better results and thus appears as potentially fruitful. In a more nonlinear regime, when tropical instability waves develop, the global nature of the variational approach helps control model dynamics better than the sequential approach of the SEEK filter. This aspect is probably enhanced by the context of the experiments in that there is a limited amount of assimilated data and no model error.     

Scale-recursive assimilation of precipitation data

This paper presents a new statistical method for assimilating precipitation data from different sensors operating over a range of scales. The technique is based on a scale-recursive estimation algorithm which is computationally efficient and able to account for the nested spatial structure of precipitation fields. The version of the algorithm described here relies on a static multiplicative cascade model which relates rainrates at different scales. Bayesian estimation techniques are used to condition rainrate estimates on measurements. The conditioning process is carried out recursively in two sweeps: first from fine to coarse scales and then from coarse to fine scales. The complete estimation algorithm is similar to a fixed interval smoother although it processes data over scale rather than time. We use this algorithm to assimilate radar and satellite microwave data collected during the tropical ocean–global atmosphere coupled ocean–atmosphere response experiment (TOGA-COARE). The resulting rainrate estimates reproduce withheld radar measurements to within the level of accuracy predicted by the assimilation algorithm.     

Towards a data assimilation system for morphodynamic modeling: bathymetric data assimilation for wave property estimation

Data assimilation is mainly concerned with the proper management of uncertainties. The main objective of the present work is to implement and analyze a data assimilation technique capable of assimilating bathymetric data into a coupled flow, wave, and morphodynamic model. For the case presented here, wave significant height, wave direction of incidence, and wave peak period are being optimized based on bathymetric data taken from a twin experiment. An adjoint-free variational scheme is used. In this approach, a linear reduced order model (ROM) is constructed as an approximation of the full model. The ROM is an autoregressive model of order 1 (AR1) that preserves the parametrization. Since the ROM is linear, the construction of its adjoint is straightforward, making the implementation of 4D variational data assimilation effortless. The scheme is able to update the morphodynamic model satisfactorily despite the fact that the model shows nonlinear behavior even for very small perturbations of all three parameters. The size and direction of the perturbations necessary for constructing the ROM have a significant impact on the performance of the technique.     

Turbulent viscosity optimized by data assimilation

As an alternative approach to classical turbulence modelling using a first or second order closure, the data assimilation method of optimal control is applied to estimate a time and space-dependent turbulent viscosity in a three-dimensional oceanic circulation model. The optimal control method, described for a 3-D primitive equation model, involves the minimization of a cost function that quantifies the discrepancies between the simulations and the observations. An iterative algorithm is obtained via the adjoint model resolution. In a first experiment, a k ± L model is used to simulate the one-dimensional development of inertial oscillations resulting from a wind stress at the sea surface and with the presence of a halocline. These results are used as synthetic observations to be assimilated. The turbulent viscosity is then recovered without the k + L closure, even with sparse and noisy observations. The problems of controllability and of the dimensions of the control are then discussed. A second experiment consists of a two-dimensional schematic simulation. A 2-D turbulent viscosity field is estimated from data on the initial and final states of a coastal upwelling event.     

OpenDA-NEMO framework for ocean data assimilation

Data assimilation methods provide a means to handle the modeling errors and uncertainties in sophisticated ocean models. In this study, we have created an OpenDA-NEMO framework unlocking the data assimilation tools available in OpenDA for use with NEMO models. This includes data assimilation methods, automatic parallelization, and a recently implemented automatic localization algorithm that removes spurious correlations in the model based on uncertainties in the computed Kalman gain matrix. We have set up a twin experiment where we assimilate sea surface height (SSH) satellite measurements. From the experiments, we can conclude that the OpenDA-NEMO framework performs as expected and that the automatic localization significantly improves the performance of the data assimilation algorithm by successfully removing spurious correlations. Based on these results, it looks promising to extend the framework with new kinds of observations and work on improving the computational speed of the automatic localization technique such that it becomes feasible to include large number of observations.     

Neural networks in auroral data assimilation

Data assimilation is an essential step for improving space weather forecasting by means of a weighted combination between observational data and data from a mathematical model. In the present work data assimilation methods based on Kalman filter (KF) and artificial neural networks are applied to a three-wave model of auroral radio emissions. A novel data assimilation method is presented, whereby a multilayer perceptron neural network is trained to emulate a KF for data assimilation by using cross-validation. The results obtained render support for the use of neural networks as an assimilation technique for space weather prediction.     

Observing system experiments over the Atlantic Ocean with the REMO ocean data assimilation system (RODAS) into HYCOM

Ocean Dynamics - The Oceanographic Modeling and Observation Network (REMO) focuses on scientific and technological development of operational oceanography in Brazil considering both numerical...     

Toward a global ocean data assimilation system based on ensemble optimum interpolation: altimetry data assimilation experiment

A global ocean data assimilation system based on the ensemble optimum interpolation (EnOI) has been under development as the Chinese contribution to the Global Ocean Data Assimilation Experiment. The system uses a global ocean general circulation model, which is eddy permitting, developed by the Institute of Atmospheric Physics of the Chinese Academy of Sciences. In this paper, the implementation of the system is described in detail. We describe the sampling strategy to generate the stationary ensembles for EnOI. In addition, technical methods are introduced to deal with the requirement of massive memory space to hold the stationary ensembles of the global ocean. The system can assimilate observations such as satellite altimetry, sea surface temperature (SST), in situ temperature and salinity from Argo, XBT, Tropical Atmosphere Ocean (TAO), and other sources in a straightforward way. As a first step, an assimilation experiment from 1997 to 2001 is carried out by assimilating the sea level anomaly (SLA) data from TOPEX/Poseidon. We evaluate the performance of the system by comparing the results with various types of observations. We find that SLA assimilation shows very positive impact on the modeled fields. The SST and sea surface height fields are clearly improved in terms of both the standard deviation and the root mean square difference. In addition, the assimilation produces some improvements in regions where mesoscale processes cannot be resolved with the horizontal resolution of this model. Comparisons with TAO profiles in the Pacific show that the temperature and salinity fields have been improved to varying degrees in the upper ocean. The biases with respect to the independent TAO profiles are reduced with a maximum magnitude of about 0.25°C and 0.1 psu for the time-averaged temperature and salinity. The improvements on temperature and salinity also lead to positive impact on the subsurface currents. The equatorial under current is enhanced in the Pacific although it is still underestimated after the assimilation.     

An explicit four-dimensional variational data assimilation method

A new data assimilation method called the explicit four-dimensional variational (4DVAR) method is proposed. In this method, the singular value decomposition (SVD) is used to construct the orthogonal basis vectors from a forecast ensemble in a 4D space. The basis vectors represent not only the spatial structure of the analysis variables but also the temporal evolution. After the analysis variables are ex-pressed by a truncated expansion of the basis vectors in the 4D space, the control variables in the cost function appear explicitly, so that the adjoint model, which is used to derive the gradient of cost func-tion with respect to the control variables, is no longer needed. The new technique significantly simpli-fies the data assimilation process. The advantage of the proposed method is demonstrated by several experiments using a shallow water numerical model and the results are compared with those of the conventional 4DVAR. It is shown that when the observation points are very dense, the conventional 4DVAR is better than the proposed method. However, when the observation points are sparse, the proposed method performs better. The sensitivity of the proposed method with respect to errors in the observations and the numerical model is lower than that of the conventional method.     

Gamma random field simulation by a covariance matrix transformation method

In studies involving environmental risk assessment, Gaussian random field generators are often used to yield realizations of a Gaussian random field, and then realizations of the non-Gaussian target random field are obtained by an inverse-normal transformation. Such simulation process requires a set of observed data for estimation of the empirical cumulative distribution function (ECDF) and covariance function of the random field under investigation. However, if realizations of a non-Gaussian random field with specific probability density and covariance function are needed, such observed-data-based simulation process will not work when no observed data are available. In this paper we present details of a gamma random field simulation approach which does not require a set of observed data. A key element of the approach lies on the theoretical relationship between the covariance functions of a gamma random field and its corresponding standard normal random field. Through a set of devised simulation scenarios, the proposed technique is shown to be capable of generating realizations of the given gamma random fields.     

Review of the Kalman-type hydrological data assimilation

ABSTRACT

There is great potential in Data Assimilation (DA) for the purposes of uncertainty identification, reduction and real-time correction of hydrological models. This paper reviews the latest developments in Kalman filters (KFs), particularly the Extended KF (EKF) and the Ensemble KF (EnKF) in hydrological DA. The hydrological DA targets, methodologies and their applicability are examined. The recent applications of the EKF and EnKF in hydrological DA are summarized and assessed critically. Furthermore, this review highlights the existing challenges in the implementation of the EKF and EnKF, especially error determination and joint parameter estimation. A detailed review of these issues would benefit not only the Kalman-type DA but also provide an important reference to other hydrological DA types.

Editor D. Koutsoyiannis; Associate editor F. Pappenberger     

On the assimilation of total-ozone satellite data

A two-dimensional model for advection and data assimilation of total-ozone data has been developed. The Assimilation Model KNMI (AMK) is a global model describing the transport of the column amounts of ozone, by a wind field at a single pressure level, assuming that total ozone behaves as a passive tracer. In this study, ozone column amounts measured by the TIROS Operational Vertical Sounder (TOVS) instrument on the National Oceanic and Atmospheric Administration (NOAA) polar satellites and wind fields from the Meteorological Archive and Retrieval System (MARS) archives at ECMWF have been used. By means of the AMK, the incomplete space-time distribution of the TOVS measurements is filled in and global total-ozone maps at any given time can be obtained. The choice of wind field to be used for transporting column amounts of ozone is extensively discussed. It is shown that the 200-hPa wind field is the optimal single-pressure-level wind field for advecting total ozone. Assimilated ozone fields are the basic information for research on atmospheric chemistry and dynamics, but are also important for the validation of ozone measurements.