Ensemble filter methods with perturbed observations applied to nonlinear problems

The ensemble Kalman filter (EnKF) has been shown repeatedly to be an effective method for data assimilation in large-scale problems, including those in petroleum engineering. Data assimilation for multiphase flow in porous media is particularly difficult, however, because the relationships between model variables (e.g., permeability and porosity) and observations (e.g., water cut and gas–oil ratio) are highly nonlinear. Because of the linear approximation in the update step and the use of a limited number of realizations in an ensemble, the EnKF has a tendency to systematically underestimate the variance of the model variables. Various approaches have been suggested to reduce the magnitude of this problem, including the application of ensemble filter methods that do not require perturbations to the observed data. On the other hand, iterative least-squares data assimilation methods with perturbations of the observations have been shown to be fairly robust to nonlinearity in the data relationship. In this paper, we present EnKF with perturbed observations as a square root filter in an enlarged state space. By imposing second-order-exact sampling of the observation errors and independence constraints to eliminate the cross-covariance with predicted observation perturbations, we show that it is possible in linear problems to obtain results from EnKF with observation perturbations that are equivalent to ensemble square-root filter results. Results from a standard EnKF, EnKF with second-order-exact sampling of measurement errors that satisfy independence constraints (EnKF (SIC)), and an ensemble square-root filter (ETKF) are compared on various test problems with varying degrees of nonlinearity and dimensions. The first test problem is a simple one-variable quadratic model in which the nonlinearity of the observation operator is varied over a wide range by adjusting the magnitude of the coefficient of the quadratic term. The second problem has increased observation and model dimensions to test the EnKF (SIC) algorithm. The third test problem is a two-dimensional, two-phase reservoir flow problem in which permeability and porosity of every grid cell (5,000 model parameters) are unknown. The EnKF (SIC) and the mean-preserving ETKF (SRF) give similar results when applied to linear problems, and both are better than the standard EnKF. Although the ensemble methods are expected to handle the forecast step well in nonlinear problems, the estimates of the mean and the variance from the analysis step for all variants of ensemble filters are also surprisingly good, with little difference between ensemble methods when applied to nonlinear problems.     

Ensemble Kalman filtering with shrinkage regression techniques

The classical ensemble Kalman filter (EnKF) is known to underestimate the prediction uncertainty. This can potentially lead to low forecast precision and an ensemble collapsing into a single realisation. In this paper, we present alternative EnKF updating schemes based on shrinkage methods known from multivariate linear regression. These methods reduce the effects caused by collinear ensemble members and have the same computational properties as the fastest EnKF algorithms previously suggested. In addition, the importance of model selection and validation for prediction purposes is investigated, and a model selection scheme based on cross-validation is introduced. The classical EnKF scheme is compared with the suggested procedures on two-toy examples and one synthetic reservoir case study. Significant improvements are seen, both in terms of forecast precision and prediction uncertainty estimates.     

Fast linearized forecasts for subsurface flow data assimilation with ensemble Kalman filter

Ensemble methods present a practical framework for parameter estimation, performance prediction, and uncertainty quantification in subsurface flow and transport modeling. In particular, the ensemble Kalman filter (EnKF) has received significant attention for its promising performance in calibrating heterogeneous subsurface flow models. Since an ensemble of model realizations is used to compute the statistical moments needed to perform the EnKF updates, large ensemble sizes are needed to provide accurate updates and uncertainty assessment. However, for realistic problems that involve large-scale models with computationally demanding flow simulation runs, the EnKF implementation is limited to small-sized ensembles. As a result, spurious numerical correlations can develop and lead to inaccurate EnKF updates, which tend to underestimate or even eliminate the ensemble spread. Ad hoc practical remedies, such as localization, local analysis, and covariance inflation schemes, have been developed and applied to reduce the effect of sampling errors due to small ensemble sizes. In this paper, a fast linear approximate forecast method is proposed as an alternative approach to enable the use of large ensemble sizes in operational settings to obtain more improved sample statistics and EnKF updates. The proposed method first clusters a large number of initial geologic model realizations into a small number of groups. A representative member from each group is used to run a full forward flow simulation. The flow predictions for the remaining realizations in each group are approximated by a linearization around the full simulation results of the representative model (centroid) of the respective cluster. The linearization can be performed using either adjoint-based or ensemble-based gradients. Results from several numerical experiments with two-phase and three-phase flow systems in this paper suggest that the proposed method can be applied to improve the EnKF performance in large-scale problems where the number of full simulation is constrained.     

Multimodal ensemble Kalman filtering using Gaussian mixture models

In this paper we present an extension of the ensemble Kalman filter (EnKF) specifically designed for multimodal systems. EnKF data assimilation scheme is less accurate when it is used to approximate systems with multimodal distribution such as reservoir facies models. The algorithm is based on the assumption that both prior and posterior distribution can be approximated by Gaussian mixture and it is validated by the introduction of the concept of finite ensemble representation. The effectiveness of the approach is shown with two applications. The first example is based on Lorenz model. In the second example, the proposed methodology combined with a localization technique is used to update a 2D reservoir facies models. Both applications give evidence of an improved performance of the proposed method respect to the EnKF.     

Balance-aware covariance localisation for atmospheric and oceanic ensemble Kalman filters

Covariance localisation is used in many implementations of the ensemble Kalman filter (EnKF) but has been shown by Lorenc and by Kepert to significantly degrade the main balances in the atmosphere and ocean. Kepert recently introduced an improved form of localisation that reduced or eliminated this problem. This paper presents an extension to that approach, in which the background state is decomposed into balanced and unbalanced parts as part of the localisation. This new balance-aware localisation is shown to be a slight improvement on the earlier work of Kepert and a substantial improvement on the standard Schur-product localisation. Balance-aware localisation also enables the use of some sets of alternative analysis variables that do not work well with conventional localisation in the EnKF. It is shown using identical-twin experiments with a global spectral shallow-water model and no separate initialisation step that analysis to geopotential, streamfunction and velocity potential is slightly more accurate than is analysis to geopotential and the wind components. Analysis to unbalanced (instead of total) geopotential, streamfunction and velocity potential leads to slightly less accurate but significantly better balanced analyses than the other choices of analysis variables. If nonlinear normal modes initialisation is incorporated in the analysis cycling, then the conventional localisation becomes the most accurate method. However, initialisation may be undesirable or unavailable, and the comparison of system performance without localisation is useful since it helps improve understanding of the balance issues in EnKF-based assimilation systems.     

Comparing the adaptive Gaussian mixture filter with the ensemble Kalman filter on synthetic reservoir models

Over the last years, the ensemble Kalman filter (EnKF) has become a very popular tool for history matching petroleum reservoirs. EnKF is an alternative to more traditional history matching techniques as it is computationally fast and easy to implement. Instead of seeking one best model estimate, EnKF is a Monte Carlo method that represents the solution with an ensemble of state vectors. Lately, several ensemble-based methods have been proposed to improve upon the solution produced by EnKF. In this paper, we compare EnKF with one of the most recently proposed methods, the adaptive Gaussian mixture filter (AGM), on a 2D synthetic reservoir and the Punq-S3 test case. AGM was introduced to loosen up the requirement of a Gaussian prior distribution as implicitly formulated in EnKF. By combining ideas from particle filters with EnKF, AGM extends the low-rank kernel particle Kalman filter. The simulation study shows that while both methods match the historical data well, AGM is better at preserving the geostatistics of the prior distribution. Further, AGM also produces estimated fields that have a higher empirical correlation with the reference field than the corresponding fields obtained with EnKF.     

基于集合卡尔曼平滑算法的土壤水分同化

为研究观测资料稀少情况下土壤质地及有机质对土壤水分同化的影响,发展了集合卡尔曼平滑(Ensemble Kalman Smooth, EnKS)的土壤水分同化方案。利用黑河上游阿柔冻融观测站2008年6月1日至10月29日的观测数据,使用EnKS算法将表层土壤水分观测数据同化到简单生物圈模型(Simple Biosphere Model 2, SiB2)中,分析不同方案对土壤水分估计的影响,并与集合卡尔曼滤波算法(EnKF)的结果进行比较。研究结果表明,土壤质地和有机质对表层土壤水分模拟结果影响最大而对深层的影响相对较小;利用EnKF和EnKS算法同化表层土壤水分观测数据,均能够显著提高表层和根区土壤水分估计的精度,EnKS算法的精度略高于EnKF且所受土壤质地和有机质的影响小于EnKF;当观测数据稀少时,EnKS算法仍然可以得到较高精度的土壤水分估计。     

评估集合卡曼滤波反演土壤湿度廓线的性能

集合卡曼滤波由于易于使用而被广泛地应用到陆面数据同化研究中,它是建立在模型为线性、误差为正态分布的假设上,而实际土壤湿度方程是高度非线性的,并且当土壤过干或过湿时会发生样本偏斜.为了全面评估它在同化表层土壤湿度观测来反演土壤湿度廓线的性能,特引入不需要上述假设的采样重要性重采样粒子滤波,比较非线性和偏斜性对同化算法的影响.结果显示:不管是小样本还是大样本,集合卡曼滤波都能快速、准确地逼近样本均值,而粒子滤波只有在大样本时才能缓慢地趋近;此外,集合卡曼滤波的粒子边缘概率密度及其偏度和峰度与粒子滤波完全不同,前者粒子虽不完全满足正态分布,但始终为单峰状态,而后者粒子随同化推进经历了单峰到双峰再到单峰的变化.     

Ensemble Kalman filtering for non-linear likelihood models using kernel-shrinkage regression techniques

One of the major limitations of the classical ensemble Kalman filter (EnKF) is the assumption of a linear relationship between the state vector and the observed data. Thus, the classical EnKF algorithm can suffer from poor performance when considering highly non-linear and non-Gaussian likelihood models. In this paper, we have formulated the EnKF based on kernel-shrinkage regression techniques. This approach makes it possible to handle highly non-linear likelihood models efficiently. Moreover, a solution to the pre-image problem, essential in previously suggested EnKF schemes based on kernel methods, is not required. Testing the suggested procedure on a simple, illustrative problem with a non-linear likelihood model, we were able to obtain good results when the classical EnKF failed.     

Improving the Ensemble Estimate of the Kalman Gain by Bootstrap Sampling

Using a small ensemble size in the ensemble Kalman filter methodology is efficient for updating numerical reservoir models but can result in poor updates following spurious correlations between observations and model variables. The most common approach for reducing the effect of spurious correlations on model updates is multiplication of the estimated covariance by a tapering function that eliminates all correlations beyond a prespecified distance. Distance-dependent tapering is not always appropriate, however. In this paper, we describe efficient methods for discriminating between the real and the spurious correlations in the Kalman gain matrix by using the bootstrap method to assess the confidence level of each element from the Kalman gain matrix. The new method is tested on a small linear problem, and on a water flooding reservoir history matching problem. For the water flooding example, a small ensemble size of 30 was used to compute the Kalman gain in both the screened EnKF and standard EnKF methods. The new method resulted in significantly smaller root mean squared errors of the estimated model parameters and greater variability in the final updated ensemble.     

集合卡尔曼滤波估计水文地质参数的局域化修正

集合卡尔曼滤波(Ensemble Kalman Filter,EnKF)作为一种有效的数据同化方法,在众多数值实验中体现优势的同时,也暴露了它使用小集合估计协方差情况下精度较低的缺陷。为了降低取样噪声对协方差估计的干扰并提高滤波精度,应用局域化函数对小集合估计的协方差进行修正,即在协方差矩阵中以舒尔积的形式增加空间距离权重以限制远距离相关。在一个二维理想孔隙承压含水层模型中的运行结果表明,局域化对集合卡尔曼滤波估计地下水参数的修正十分有效,局域化可以很好地过滤小集合估计中噪声的影响,节省计算量的同时又可以防止滤波发散。相关长度较小的水文地质参数(如对数渗透系数)更容易受到噪声的干扰,更有必要进行局域化修正。     

A parallel ensemble-based framework for reservoir history matching and uncertainty characterization

We present a parallel framework for history matching and uncertainty characterization based on the Kalman filter update equation for the application of reservoir simulation. The main advantages of ensemble-based data assimilation methods are that they can handle large-scale numerical models with a high degree of nonlinearity and large amount of data, making them perfectly suited for coupling with a reservoir simulator. However, the sequential implementation is computationally expensive as the methods require relatively high number of reservoir simulation runs. Therefore, the main focus of this work is to develop a parallel data assimilation framework with minimum changes into the reservoir simulator source code. In this framework, multiple concurrent realizations are computed on several partitions of a parallel machine. These realizations are further subdivided among different processors, and communication is performed at data assimilation times. Although this parallel framework is general and can be used for different ensemble techniques, we discuss the methodology and compare results of two algorithms, the ensemble Kalman filter (EnKF) and the ensemble smoother (ES). Computational results show that the absolute runtime is greatly reduced using a parallel implementation versus a serial one. In particular, a parallel efficiency of about 35 % is obtained for the EnKF, and an efficiency of more than 50 % is obtained for the ES.     

History matching time-lapse seismic data using the ensemble Kalman filter with multiple data assimilations

The ensemble Kalman filter (EnKF) has become a popular method for history matching production and seismic data in petroleum reservoir models. However, it is known that EnKF may fail to give acceptable data matches especially for highly nonlinear problems. In this paper, we introduce a procedure to improve EnKF data matches based on assimilating the same data multiple times with the covariance matrix of the measurement errors multiplied by the number of data assimilations. We prove the equivalence between single and multiple data assimilations for the linear-Gaussian case and present computational evidence that multiple data assimilations can improve EnKF estimates for the nonlinear case. The proposed procedure was tested by assimilating time-lapse seismic data in two synthetic reservoir problems, and the results show significant improvements compared to the standard EnKF. In addition, we review the inversion schemes used in the EnKF analysis and present a rescaling procedure to avoid loss of information during the truncation of small singular values.     

Continuous Facies Updating Using the Ensemble Kalman Filter and the Level Set Method

We present a methodology based on the ensemble Kalman filter (EnKF) and the level set method for the continuous model updating of geological facies with respect to production data. Geological facies are modeled using an implicit surface representation and conditioned to production data using the ensemble Kalman filter. The methodology is based on Gaussian random fields used to deform the facies boundaries. The Gaussian random fields are used as the model parameter vector to be updated sequentially within the EnKF when new measurements are available. We show the successful application of the methodology to two synthetic reservoir models.     

Groundwater parameter estimation using the ensemble Kalman filter with localization

The ensemble Kalman filter (EnKF), an efficient data assimilation method showing advantages in many numerical experiments, is deficient when used in approximating covariance from an ensemble of small size. Implicit localization is used to add distance-related weight to covariance and filter spurious correlations which weaken the EnKF??s capability to estimate uncertainty correctly. The effect of this kind of localization is studied in two-dimensional (2D) and three-dimensional (3D) synthetic cases. It is found that EnKF with localization can capture reliably both the mean and variance of the hydraulic conductivity field with higher efficiency; it can also greatly stabilize the assimilation process as a small-size ensemble is used. Sensitivity experiments are conducted to explore the effect of localization function format and filter lengths. It is suggested that too long or too short filter lengths will prevent implicit localization from modifying the covariance appropriately. Steep localization functions will greatly disturb local dynamics like the 0-1 function even if the function is continuous; four relatively gentle localization functions succeed in avoiding obvious disturbance to the system and improve estimation. As the degree of localization of the L function increases, the parameter sensitivity becomes weak, making parameter selection easier, but more information may be lost in the assimilation process.     

基于局域化EnKF同时同化地下水水流和溶质运移数据的渗透系数场识别研究

局域化改进集合卡尔曼滤波（EnKF）可以克服EnKF方法在使用小集合时,对参数识别精度较低的缺陷,其能同化 地下水位观测数据有效识别渗透系数场。实际工作中,溶质运移数据也较容易获得。崔凯鹏（2013）尝试增加溶质运移 数据以改进只同化水流数据对渗透系数的估计结果,但是精度提高有限。本文在其基础上修改模型,进一步增加溶质注 入井,探究同时同化水头和溶质运移数据,对渗透系数场识别效果,之后对比了局域化EnKF与非局域化EnKF参数识别结 果,并分析了溶质影响范围与参数识别的关系。结果表明：同时同化溶质运移和水头资料,比同化单一种类观测数据识别 的渗透系数精度更高;相同实现数目下,局域化EnKF比EnKF对渗透系数场的估计结果与真实场更为接近;仅考虑溶质影 响范围内的渗透系数,同化水头数据在最后时刻参数识别结果好于同化溶质运移数据参数识别结果,但差别不大。     

基于集合卡尔曼滤波的多相流模型参数估计——以室内二维砂箱中重质非水相污染物入渗为例

重质非水相有机污染物(DNAPL)泄漏到地下后,其运移与分布特征受渗透率非均质性影响显著。为刻画DNAPL污染源区结构特征,需进行参数估计以描述水文地质参数的非均质性。本研究构建了基于集合卡尔曼滤波方法(EnKF)与多相流运移模型的同化方案,通过融合DNAPL饱和度观测数据推估非均质介质渗透率空间分布。通过二维砂箱实际与理想算例,验证了同化方法的推估效果,并探讨了不同因素对同化的影响。研究结果表明:基于EnKF方法同化饱和度观测资料可有效地推估非均质渗透率场;参数推估精度随观测时空密度的增大而提高;观测点位置分布对同化效果有所影响,布置在污染集中区域的观测数据对于参数估计具有较高的数据价值。     

Multiscale ensemble filtering for reservoir engineering applications

Reservoir management requires periodic updates of the simulation models using the production data available over time. Traditionally, validation of reservoir models with production data is done using a history matching process. Uncertainties in the data, as well as in the model, lead to a nonunique history matching inverse problem. It has been shown that the ensemble Kalman filter (EnKF) is an adequate method for predicting the dynamics of the reservoir. The EnKF is a sequential Monte-Carlo approach that uses an ensemble of reservoir models. For realistic, large-scale applications, the ensemble size needs to be kept small due to computational inefficiency. Consequently, the error space is not well covered (poor cross-correlation matrix approximations) and the updated parameter field becomes scattered and loses important geological features (for example, the contact between high- and low-permeability values). The prior geological knowledge present in the initial time is not found anymore in the final updated parameter. We propose a new approach to overcome some of the EnKF limitations. This paper shows the specifications and results of the ensemble multiscale filter (EnMSF) for automatic history matching. EnMSF replaces, at each update time, the prior sample covariance with a multiscale tree. The global dependence is preserved via the parent–child relation in the tree (nodes at the adjacent scales). After constructing the tree, the Kalman update is performed. The properties of the EnMSF are presented here with a 2D, two-phase (oil and water) small twin experiment, and the results are compared to the EnKF. The advantages of using EnMSF are localization in space and scale, adaptability to prior information, and efficiency in case many measurements are available. These advantages make the EnMSF a practical tool for many data assimilation problems.     

An efficient matrix-free algorithm for the ensemble Kalman filter

In this work, we present an efficient matrix-free ensemble Kalman filter (EnKF) algorithm for the assimilation of large data sets. The EnKF has increasingly become an essential tool for data assimilation of numerical models. It is an attractive assimilation method because it can evolve the model covariance matrix for a non-linear model, through the use of an ensemble of model states, and it is easy to implement for any numerical model. Nevertheless, the computational cost of the EnKF can increase significantly for cases involving the assimilation of large data sets. As more data become available for assimilation, a potential bottleneck in most EnKF algorithms involves the operation of the Kalman gain matrix. To reduce the complexity and cost of assimilating large data sets, a matrix-free EnKF algorithm is proposed. The algorithm uses an efficient matrix-free linear solver, based on the Sherman–Morrison formulas, to solve the implicit linear system within the Kalman gain matrix and compute the analysis. Numerical experiments with a two-dimensional shallow water model on the sphere are presented, where results show the matrix-free implementation outperforming an singular value decomposition-based implementation in computational time.