集合卡尔曼滤波估计水文地质参数的局域化修正

集合卡尔曼滤波(Ensemble Kalman Filter,EnKF)作为一种有效的数据同化方法,在众多数值实验中体现优势的同时,也暴露了它使用小集合估计协方差情况下精度较低的缺陷。为了降低取样噪声对协方差估计的干扰并提高滤波精度,应用局域化函数对小集合估计的协方差进行修正,即在协方差矩阵中以舒尔积的形式增加空间距离权重以限制远距离相关。在一个二维理想孔隙承压含水层模型中的运行结果表明,局域化对集合卡尔曼滤波估计地下水参数的修正十分有效,局域化可以很好地过滤小集合估计中噪声的影响,节省计算量的同时又可以防止滤波发散。相关长度较小的水文地质参数(如对数渗透系数)更容易受到噪声的干扰,更有必要进行局域化修正。     

Groundwater parameter estimation using the ensemble Kalman filter with localization

The ensemble Kalman filter (EnKF), an efficient data assimilation method showing advantages in many numerical experiments, is deficient when used in approximating covariance from an ensemble of small size. Implicit localization is used to add distance-related weight to covariance and filter spurious correlations which weaken the EnKF??s capability to estimate uncertainty correctly. The effect of this kind of localization is studied in two-dimensional (2D) and three-dimensional (3D) synthetic cases. It is found that EnKF with localization can capture reliably both the mean and variance of the hydraulic conductivity field with higher efficiency; it can also greatly stabilize the assimilation process as a small-size ensemble is used. Sensitivity experiments are conducted to explore the effect of localization function format and filter lengths. It is suggested that too long or too short filter lengths will prevent implicit localization from modifying the covariance appropriately. Steep localization functions will greatly disturb local dynamics like the 0-1 function even if the function is continuous; four relatively gentle localization functions succeed in avoiding obvious disturbance to the system and improve estimation. As the degree of localization of the L function increases, the parameter sensitivity becomes weak, making parameter selection easier, but more information may be lost in the assimilation process.     

Cross-covariances and localization for EnKF in multiphase flow data assimilation

The ensemble Kalman filter has been successfully applied for data assimilation in very large models, including those in reservoir simulation and weather. Two problems become critical in a standard implementation of the ensemble Kalman filter, however, when the ensemble size is small. The first is that the ensemble approximation to cross-covariances of model and state variables to data can indicate the presence of correlations that are not real. These spurious correlations give rise to model or state variable updates in regions that should not be updated. The second problem is that the number of degrees of freedom in the ensemble is only as large as the size of the ensemble, so the assimilation of large amounts of precise, independent data is impossible. Localization of the Kalman gain is almost universal in the weather community, but applications of localization for the ensemble Kalman filter in porous media flow have been somewhat rare. It has been shown, however, that localization of updates to regions of non-zero sensitivity or regions of non-zero cross-covariance improves the performance of the EnKF when the ensemble size is small. Localization is necessary for assimilation of large amounts of independent data. The problem is to define appropriate localization functions for different types of data and different types of variables. We show that the knowledge of sensitivity alone is not sufficient for determination of the region of localization. The region depends also on the prior covariance for model variables and on the past history of data assimilation. Although the goal is to choose localization functions that are large enough to include the true region of non-zero cross-covariance, for EnKF applications, the choice of localization function needs to balance the harm done by spurious covariance resulting from small ensembles and the harm done by excluding real correlations. In this paper, we focus on the distance-based localization and provide insights for choosing suitable localization functions for data assimilation in multiphase flow problems. In practice, we conclude that it is reasonable to choose localization functions based on well patterns, that localization function should be larger than regions of non-zero sensitivity and should extend beyond a single well pattern.     

History matching time-lapse seismic data using the ensemble Kalman filter with multiple data assimilations

The ensemble Kalman filter (EnKF) has become a popular method for history matching production and seismic data in petroleum reservoir models. However, it is known that EnKF may fail to give acceptable data matches especially for highly nonlinear problems. In this paper, we introduce a procedure to improve EnKF data matches based on assimilating the same data multiple times with the covariance matrix of the measurement errors multiplied by the number of data assimilations. We prove the equivalence between single and multiple data assimilations for the linear-Gaussian case and present computational evidence that multiple data assimilations can improve EnKF estimates for the nonlinear case. The proposed procedure was tested by assimilating time-lapse seismic data in two synthetic reservoir problems, and the results show significant improvements compared to the standard EnKF. In addition, we review the inversion schemes used in the EnKF analysis and present a rescaling procedure to avoid loss of information during the truncation of small singular values.     

An efficient matrix-free algorithm for the ensemble Kalman filter

In this work, we present an efficient matrix-free ensemble Kalman filter (EnKF) algorithm for the assimilation of large data sets. The EnKF has increasingly become an essential tool for data assimilation of numerical models. It is an attractive assimilation method because it can evolve the model covariance matrix for a non-linear model, through the use of an ensemble of model states, and it is easy to implement for any numerical model. Nevertheless, the computational cost of the EnKF can increase significantly for cases involving the assimilation of large data sets. As more data become available for assimilation, a potential bottleneck in most EnKF algorithms involves the operation of the Kalman gain matrix. To reduce the complexity and cost of assimilating large data sets, a matrix-free EnKF algorithm is proposed. The algorithm uses an efficient matrix-free linear solver, based on the Sherman–Morrison formulas, to solve the implicit linear system within the Kalman gain matrix and compute the analysis. Numerical experiments with a two-dimensional shallow water model on the sphere are presented, where results show the matrix-free implementation outperforming an singular value decomposition-based implementation in computational time.     

A stochastic collocation based Kalman filter for data assimilation

In this paper, a stochastic collocation-based Kalman filter (SCKF) is developed to estimate the hydraulic conductivity from direct and indirect measurements. It combines the advantages of the ensemble Kalman filter (EnKF) for dynamic data assimilation and the polynomial chaos expansion (PCE) for efficient uncertainty quantification. In this approach, the random log hydraulic conductivity field is first parameterized by the Karhunen–Loeve (KL) expansion and the hydraulic pressure is expressed by the PCE. The coefficients of PCE are solved with a collocation technique. Realizations are constructed by choosing collocation point sets in the random space. The stochastic collocation method is non-intrusive in that such realizations are solved forward in time via an existing deterministic solver independently as in the Monte Carlo method. The needed entries of the state covariance matrix are approximated with the coefficients of PCE, which can be recovered from the collocation results. The system states are updated by updating the PCE coefficients. A 2D heterogeneous flow example is used to demonstrate the applicability of the SCKF with respect to different factors, such as initial guess, variance, correlation length, and the number of observations. The results are compared with those from the EnKF method. It is shown that the SCKF is computationally more efficient than the EnKF under certain conditions. Each approach has its own advantages and limitations. The performance of the SCKF decreases with larger variance, smaller correlation ratio, and fewer observations. Hence, the choice between the two methods is problem dependent. As a non-intrusive method, the SCKF can be easily extended to multiphase flow problems.     

Ensemble filter methods with perturbed observations applied to nonlinear problems

The ensemble Kalman filter (EnKF) has been shown repeatedly to be an effective method for data assimilation in large-scale problems, including those in petroleum engineering. Data assimilation for multiphase flow in porous media is particularly difficult, however, because the relationships between model variables (e.g., permeability and porosity) and observations (e.g., water cut and gas–oil ratio) are highly nonlinear. Because of the linear approximation in the update step and the use of a limited number of realizations in an ensemble, the EnKF has a tendency to systematically underestimate the variance of the model variables. Various approaches have been suggested to reduce the magnitude of this problem, including the application of ensemble filter methods that do not require perturbations to the observed data. On the other hand, iterative least-squares data assimilation methods with perturbations of the observations have been shown to be fairly robust to nonlinearity in the data relationship. In this paper, we present EnKF with perturbed observations as a square root filter in an enlarged state space. By imposing second-order-exact sampling of the observation errors and independence constraints to eliminate the cross-covariance with predicted observation perturbations, we show that it is possible in linear problems to obtain results from EnKF with observation perturbations that are equivalent to ensemble square-root filter results. Results from a standard EnKF, EnKF with second-order-exact sampling of measurement errors that satisfy independence constraints (EnKF (SIC)), and an ensemble square-root filter (ETKF) are compared on various test problems with varying degrees of nonlinearity and dimensions. The first test problem is a simple one-variable quadratic model in which the nonlinearity of the observation operator is varied over a wide range by adjusting the magnitude of the coefficient of the quadratic term. The second problem has increased observation and model dimensions to test the EnKF (SIC) algorithm. The third test problem is a two-dimensional, two-phase reservoir flow problem in which permeability and porosity of every grid cell (5,000 model parameters) are unknown. The EnKF (SIC) and the mean-preserving ETKF (SRF) give similar results when applied to linear problems, and both are better than the standard EnKF. Although the ensemble methods are expected to handle the forecast step well in nonlinear problems, the estimates of the mean and the variance from the analysis step for all variants of ensemble filters are also surprisingly good, with little difference between ensemble methods when applied to nonlinear problems.     

Improved initial ensemble generation coupled with ensemble square root filters and inflation to estimate uncertainty

The performance of the Ensemble Kalman Filter method (EnKF) depends on the sample size compared to the dimension of the parameters space. In real applications insufficient sampling may result in spurious correlations which reduce the accuracy of the filter with a strong underestimation of the uncertainty. Covariance localization and inflation are common solutions to these problems. The Ensemble Square Root Filters (ESRF) is also better to estimate uncertainty with respect to the EnKF. In this work we propose a method that limits the consequences of sampling errors by means of a convenient generation of the initial ensemble. This regeneration is based on a Stationary Orthogonal-Base Representation (SOBR) obtained via a singular value decomposition of a stationary covariance matrix estimated from the ensemble. The technique is tested on a 2D single phase reservoir and compared with the other common techniques. The evaluation is based on a reference solution obtained with a very large ensemble (one million members) which remove the spurious correlations. The example gives evidence that the SOBR technique is a valid alternative to reduce the effect of sampling error. In addition, when the SOBR method is applied in combination with the ESRF and inflation, it gives the best performance in terms of uncertainty estimation and oil production forecast.     

The temperature-based localization for the application of EnKF on automatic history matching of the SAGD process

The ensemble Kalman filter (EnKF) has been successfully applied to data assimilation in steam-assisted gravity drainage (SAGD) process, but applications of localization for the EnKF in the SAGD process have not been studied. Distance-based localization has been reported to be very efficient for assimilation of large amounts of independent data with a small ensemble in water flooding process, but it is not applicable to the SAGD process, since in the SAGD process, oil is produced mainly from the transition zone steam chamber to cold oil instead of the regions around the producer. As the oil production rate is mainly affected by the temperature distribution in the transition zone, temperature-based localization was proposed for automatic history matching of the SAGD process. The regions of the localization function were determined through sensitivity analysis by using a large ensemble with 1000 members. The sensitivity analysis indicated that the regions of cross-correlations between oil production and state variables are much wider than the correlations between production data and model variables. To choose localization regions that are large enough to include the true regions of non-zero cross-covariance, the localization function is defined based on the regions of non-zero covariances of production data to state variables. The non-zero covariances between production data and state variables are distributed in accordance with the steam chamber. This makes the definition of a universal localization function for different state variables easier. Based on the cross-correlation analysis, the temperature range in which oil production is contributed is determined, and beyond or below this range, the localization function reduces from one, and at the critical temperature or steam temperature, the localization function reduces to zero. The temperature-based localization function was obtained through modifying the distance-based localization function. Localization is applied to covariance of data with permeability, saturation, and temperature, as well as the covariance of data with data. A small ensemble (10 ensemble members) was employed in several case studies. Without localization, the variability in the ensemble collapsed very quickly and lost the ability to assimilate later data. The mean variance of model variables dropped dramatically by 95 %, and there was almost no variability in ensemble forecasts, while the prediction was far from the reference with data mismatch keeping up at a high level. At least 50 ensemble members are needed to keep the qualities of matches and forecasts, which significantly increases the computation time. The EnKF with temperature-based localization is able to avoid the collapse of ensemble variability with a small ensemble (10 members), which saves the computation time and gives better history match and prediction results.     

考虑预报偏差的迭代式集合卡尔曼滤波在地下水水流数据同化中的应用

集合卡尔曼滤波（Ensemble Kalman Filter,EnKF）方法已广泛应用于地下水水流和污染物运移模拟相关问题的求解。但前人研究多建立在同化系统预报模型是准确的基础上,忽视了模型概化的不确定性。当模型概化不准确时,将导致预报偏差,可能带来错误的系统估计。因此,文章提出考虑模型预报偏差的迭代式集合卡尔曼滤波（Bias aware Ensemble Kalman Filter with Confirming Option,Bias-CEnKF）方法。以地下水水流数据同化为例,研究模型概化存在不确定条件下,边界条件、初始条件、源汇项概化不准确时新方法的有效性。结果表明,当预报模型概化不准确时,使用标准EnKF方法进行数据同化,可能会导致滤波发散,造成同化失败。Bias-CEnKF方法不仅保留了较好的同化性能,同时减小了参数、变量、偏差项非线性关系带来的不一致性。针对文章中4种情景,Bias-CEnKF同化获得的含水层渗透系数场以及水头场均接近真实场,且预报结果可靠。本研究进一步提升了模型概化不确定时EnKF方法的适用性,为实际野外复杂条件下地下水水流数据同化问题提供了可靠的方法。     

Multiscale ensemble filtering for reservoir engineering applications

Reservoir management requires periodic updates of the simulation models using the production data available over time. Traditionally, validation of reservoir models with production data is done using a history matching process. Uncertainties in the data, as well as in the model, lead to a nonunique history matching inverse problem. It has been shown that the ensemble Kalman filter (EnKF) is an adequate method for predicting the dynamics of the reservoir. The EnKF is a sequential Monte-Carlo approach that uses an ensemble of reservoir models. For realistic, large-scale applications, the ensemble size needs to be kept small due to computational inefficiency. Consequently, the error space is not well covered (poor cross-correlation matrix approximations) and the updated parameter field becomes scattered and loses important geological features (for example, the contact between high- and low-permeability values). The prior geological knowledge present in the initial time is not found anymore in the final updated parameter. We propose a new approach to overcome some of the EnKF limitations. This paper shows the specifications and results of the ensemble multiscale filter (EnMSF) for automatic history matching. EnMSF replaces, at each update time, the prior sample covariance with a multiscale tree. The global dependence is preserved via the parent–child relation in the tree (nodes at the adjacent scales). After constructing the tree, the Kalman update is performed. The properties of the EnMSF are presented here with a 2D, two-phase (oil and water) small twin experiment, and the results are compared to the EnKF. The advantages of using EnMSF are localization in space and scale, adaptability to prior information, and efficiency in case many measurements are available. These advantages make the EnMSF a practical tool for many data assimilation problems.     

Fast linearized forecasts for subsurface flow data assimilation with ensemble Kalman filter

Ensemble methods present a practical framework for parameter estimation, performance prediction, and uncertainty quantification in subsurface flow and transport modeling. In particular, the ensemble Kalman filter (EnKF) has received significant attention for its promising performance in calibrating heterogeneous subsurface flow models. Since an ensemble of model realizations is used to compute the statistical moments needed to perform the EnKF updates, large ensemble sizes are needed to provide accurate updates and uncertainty assessment. However, for realistic problems that involve large-scale models with computationally demanding flow simulation runs, the EnKF implementation is limited to small-sized ensembles. As a result, spurious numerical correlations can develop and lead to inaccurate EnKF updates, which tend to underestimate or even eliminate the ensemble spread. Ad hoc practical remedies, such as localization, local analysis, and covariance inflation schemes, have been developed and applied to reduce the effect of sampling errors due to small ensemble sizes. In this paper, a fast linear approximate forecast method is proposed as an alternative approach to enable the use of large ensemble sizes in operational settings to obtain more improved sample statistics and EnKF updates. The proposed method first clusters a large number of initial geologic model realizations into a small number of groups. A representative member from each group is used to run a full forward flow simulation. The flow predictions for the remaining realizations in each group are approximated by a linearization around the full simulation results of the representative model (centroid) of the respective cluster. The linearization can be performed using either adjoint-based or ensemble-based gradients. Results from several numerical experiments with two-phase and three-phase flow systems in this paper suggest that the proposed method can be applied to improve the EnKF performance in large-scale problems where the number of full simulation is constrained.     

Sampling error distribution for the ensemble Kalman filter update step

In recent years, data assimilation techniques have been applied to an increasingly wider specter of problems. Monte Carlo variants of the Kalman filter, in particular, the ensemble Kalman filter (EnKF), have gained significant popularity. EnKF is used for a wide variety of applications, among them for updating reservoir simulation models. EnKF is a Monte Carlo method, and its reliability depends on the actual size of the sample. In applications, a moderately sized sample (40–100 members) is used for computational convenience. Problems due to the resulting Monte Carlo effects require a more thorough analysis of the EnKF. Earlier we presented a method for the assessment of the error emerging at the EnKF update step (Kovalenko et al., SIAM J Matrix Anal Appl, in press). A particular energy norm of the EnKF error after a single update step was studied. The energy norm used to assess the error is hard to interpret. In this paper, we derive the distribution of the Euclidean norm of the sampling error under the same assumptions as before, namely normality of the forecast distribution and negligibility of the observation error. The distribution depends on the ensemble size, the number and spatial arrangement of the observations, and the prior covariance. The distribution is used to study the error propagation in a single update step on several synthetic examples. The examples illustrate the changes in reliability of the EnKF, when the parameters governing the error distribution vary.     

Relation between two common localisation methods for the EnKF

This study investigates the relation between two common localisation methods in ensemble Kalman filter (EnKF) systems: covariance localisation and local analysis. Both methods are popular in large-scale applications with the EnKF. The case of local observations with non-correlated errors is considered. Both methods are formulated in terms of tapering of ensemble anomalies, which provides a framework for their comparison. Based on analytical considerations and experimental evidence, we conclude that in practice the two methods should yield very similar results, so that the choice between them should be based on other criteria, such as numerical effectiveness and scalability.     

Application of the EnKF and Localization to Automatic History Matching of Facies Distribution and Production Data

The performance of the ensemble Kalman filter (EnKF) for continuous updating of facies location and boundaries in a reservoir model based on production and facies data for a 3D synthetic problem is presented. The occurrence of the different facies types is treated as a random process and the initial distribution was obtained by truncating a bi-Gaussian random field. Because facies data are highly non-Gaussian, re-parameterization was necessary in order to use the EnKF algorithm for data assimilation; two Gaussian random fields are updated in lieu of the static facies parameters. The problem of history matching applied to facies is difficult due to (1) constraints to facies observations at wells are occasionally violated when productions data are assimilated; (2) excessive reduction of variance seems to be a bigger problem with facies than with Gaussian random permeability and porosity fields; and (3) the relationship between facies variables and data is so highly non-linear that the final facies field does not always honor early production data well. Consequently three issues are investigated in this work. Is it possible to iteratively enforce facies constraints when updates due to production data have caused them to be violated? Can localization of adjustments be used for facies to prevent collapse of the variance during the data-assimilation period? Is a forecast from the final state better than a forecast from time zero using the final parameter fields?To investigate these issues, a 3D reservoir simulation model is coupled with the EnKF technique for data assimilation. One approach to enforcing the facies constraint is continuous iteration on all available data, which may lead to inconsistent model states, incorrect weighting of the production data and incorrect adjustment of the state vector. A sequential EnKF where the dynamic and static data are assimilated sequentially is presented and this approach seems to have solved the highlighted problems above. When the ensemble size is small compared to the number of independent data, the localized adjustment of the state vector is a very important technique that may be used to mitigate loss of rank in the ensemble. Implementing a distance-based localization of the facies adjustment appears to mitigate the problem of variance deficiency in the ensembles by ensuring that sufficient variability in the ensemble is maintained throughout the data assimilation period. Finally, when data are assimilated without localization, the prediction results appear to be independent of the starting point. When localization is applied, it is better to predict from the start using the final parameter field rather than continue from the final state.     

A comparison between a Monte Carlo implementation of retrospective optimal interpolation and an ensemble Kalman filter in nonlinear dynamics

To more correctly estimate the error covariance of an evolved state of a nonlinear dynamical system, the second and higher-order moments of the prior error need to be known. Retrospective optimal interpolation (ROI) may require relatively less information on the higher-order moments of the prior errors than an ensemble Kalman filter (EnKF) because it uses the initial conditions as the background states instead of forecasts. Analogous to the extension of a Kalman filter into an EnKF, an ensemble retrospective optimal interpolation (EnROI) technique was derived using the Monte Carlo method from ROI. In contrast to the deterministic version of ROI, the background error covariance is represented by a background ensemble in EnROI. By sequentially applying EnROI to a moving limited analysis window and exploiting the forecast from the average of the background ensemble of EnROI as a guess field, the computation costs for EnROI can be reduced. In the numerical experiment using a Lorenz-96 model and a Model-III of Lorenz with a perfect-model assumption, the cost-effectiveness of the suboptimal version of EnROI is demonstrated to be superior to that of EnKF using perturbed observations.     

水动力学模型与集合卡尔曼滤波耦合的实时校正多变量分析方法

为减少非恒定水流计算中的不确定性,在水流随机运动系统状态空间模型基础上,应用集合卡尔曼滤波(EnKF)技术建立了非恒定水流分析的实时更新(校正)方法。该方法适用于非线性的随机微分方程,过程和观测噪声可以是非正态分布。同时,为充分利用水位、流量等误差量级相差巨大的观测中所蕴含的有效信息,导出了EnKF多变量分析格式。以明渠单峰洪水过程的合成数据为例,考察了运用建立的实时更新方法分析预报一维洪水演进的性能。重点对比了采用不同精度等级下的水位和流量观测资料进行滤波的效果。在中国现行标准规定的允许观测误差范围内,以水位观测进行一维洪水动力学模型的滤波分析可有效地控制误差、估计流量、识别水流运动系统状态。长江干流清溪场至万县江段实际洪水计算还证实:该方法通过插入即时观测,可实时更新模型状态,给出与实际更为接近的计算。     

评估集合卡曼滤波反演土壤湿度廓线的性能

集合卡曼滤波由于易于使用而被广泛地应用到陆面数据同化研究中,它是建立在模型为线性、误差为正态分布的假设上,而实际土壤湿度方程是高度非线性的,并且当土壤过干或过湿时会发生样本偏斜.为了全面评估它在同化表层土壤湿度观测来反演土壤湿度廓线的性能,特引入不需要上述假设的采样重要性重采样粒子滤波,比较非线性和偏斜性对同化算法的影响.结果显示:不管是小样本还是大样本,集合卡曼滤波都能快速、准确地逼近样本均值,而粒子滤波只有在大样本时才能缓慢地趋近;此外,集合卡曼滤波的粒子边缘概率密度及其偏度和峰度与粒子滤波完全不同,前者粒子虽不完全满足正态分布,但始终为单峰状态,而后者粒子随同化推进经历了单峰到双峰再到单峰的变化.     

Improving the Ensemble Estimate of the Kalman Gain by Bootstrap Sampling

Using a small ensemble size in the ensemble Kalman filter methodology is efficient for updating numerical reservoir models but can result in poor updates following spurious correlations between observations and model variables. The most common approach for reducing the effect of spurious correlations on model updates is multiplication of the estimated covariance by a tapering function that eliminates all correlations beyond a prespecified distance. Distance-dependent tapering is not always appropriate, however. In this paper, we describe efficient methods for discriminating between the real and the spurious correlations in the Kalman gain matrix by using the bootstrap method to assess the confidence level of each element from the Kalman gain matrix. The new method is tested on a small linear problem, and on a water flooding reservoir history matching problem. For the water flooding example, a small ensemble size of 30 was used to compute the Kalman gain in both the screened EnKF and standard EnKF methods. The new method resulted in significantly smaller root mean squared errors of the estimated model parameters and greater variability in the final updated ensemble.     

Closed-loop reservoir management on the Brugge test case

This paper proposes an augmented Lagrangian method for production optimization in which the cost function to be maximized is defined as an augmented Lagrangian function consisting of the net present value (NPV) and all the equality and inequality constraints except the bound constraints. The bound constraints are dealt with using a trust-region gradient projection method. The paper also presents a way to eliminate the need to convert the inequality constraints to equality constraints with slack variables in the augmented Lagrangian function, which greatly reduces the size of the optimization problem when the number of inequality constraints is large. The proposed method is tested in the context of closed-loop reservoir management benchmark problem based on the Brugge reservoir setup by TNO. In the test, we used the ensemble Kalman filter (EnKF) with covariance localization for data assimilation. Production optimization is done on the updated ensemble mean model from EnKF. The production optimization resulted in a substantial increase in the NPV for the expected reservoir life compared to the base case with reactive control.