Evaluation and error analysis: Kalman gain regularization versus covariance regularization

Ensemble size is critical to the efficiency and performance of the ensemble Kalman filter, but when the ensemble size is small, the Kalman gain generally cannot be well estimated. To reduce the negative effect of spurious correlations, a regularization process applied on either the covariance or the Kalman gain seems to be necessary. In this paper, we evaluate and compare the estimation errors when two regularization methods including the distance-dependent localization and the bootstrap-based screening are applied on the covariance and on the Kalman gain. The investigations were carried out through two examples: 1D linear problem without dynamics but for which the true Kalman gain can be computed and a 2D highly nonlinear reservoir fluid flow problem. The investigation resulted in three primary conclusions. First, if localizations of two covariance matrices are not consistent, the estimate of the Kalman gain will generally be poor at the observation location. The consistency condition can be difficult to apply for nonlocal observations. Second, the estimate of the Kalman gain that results from covariance regularization is generally subject to greater errors than the estimate of the Kalman gain that results from Kalman gain regularization. Third, in terms of removing spurious correlations in the estimation of spatially correlated variables, the performance of screening Kalman gain is comparable as the performance of localization methods (applied on either covariance or Kalman gain), but screening Kalman gain outperforms the localization methods in terms of generality for application, as the screening method can be used for estimating both spatially correlated and uncorrelated variables, and moreover, no assumption about the prior covariance is required for the screening method.