Entropy and spatial disorder

The majority of geostatistical estimation and simulation algorithms rely on a covariance model as the sole characteristic of the spatial distribution of the attribute under study. The limitation to a single covariance implicitly calls for a multivariate Gaussian model for either the attribute itself or for its normal scores transform. The Gaussian model could be justified on the basis that it is both analytically simple and it is a maximum entropy model, i.e., a model that minimizes unwarranted structural properties. As a consequence, the Gaussian model also maximizes spatial disorder (beyond the imposed covariance) which can cause flow simulation results performed on multiple stochastic images to be very similar; thus, the space of response uncertainty could be too narrow entailing a misleading sense of safety. The ability of the sole covariance to adequately describe spatial distributions for flow studies, and the assumption that maximum spatial disorder amounts to either no additional information or a safe prior hypothesis are questioned. This paper attempts to clarify the link between entropy and spatial disorder and to provide, through a detailed case study, an appreciation for the impact of entropy of prior random function models on the resulting response distributions.     

Entropy and spatial disorder

The majority of geostatistical estimation and simulation algorithms rely on a covariance model as the sole characteristic of the spatial distribution of the attribute under study. The limitation to a single covariance implicitly calls for a multivariate Gaussian model for either the attribute itself or for its normal scores transform. The Gaussian model could be justified on the basis that it is both analytically simple and it is a maximum entropy model, i.e., a model that minimizes unwarranted structural properties. As a consequence, the Gaussian model also maximizes spatial disorder (beyond the imposed covariance) which can cause flow simulation results performed on multiple stochastic images to be very similar; thus, the space of response uncertainty could be too narrow entailing a misleading sense of safety. The ability of the sole covariance to adequately describe spatial distributions for flow studies, and the assumption that maximum spatial disorder amounts to either no additional information or a safe prior hypothesis are questioned. This paper attempts to clarify the link between entropy and spatial disorder and to provide, through a detailed case study, an appreciation for the impact of entropy of prior random function models on the resulting response distributions.     

Computationally efficient restricted maximum likelihood estimation of generalized covariance functions

Computational aspects of the estimation of generalized covariance functions by the method of restricted maximum likelihood (REML) are considered in detail. In general, REML estimation is computationally intensive, but significant computational savings are available in important special cases. The approach taken here restricts attention to data whose spatial configuration is a regular lattice, but makes no restrictions on the number of parameters involved in the generalized covariance nor (with the exception of one result) on the nature of the generalized covariance function's dependence on those parameters. Thus, this approach complements the recent work of L. G. Barendregt (1987), who considered computational aspects of REML estimation in the context of arbitrary spatial data configurations, but restricted attention to generalized covariances which are linear functions of only two parameters.     

Empirical Maximum Likelihood Kriging: The General Case

Although linear kriging is a distribution-free spatial interpolator, its efficiency is maximal only when the experimental data follow a Gaussian distribution. Transformation of the data to normality has thus always been appealing. The idea is to transform the experimental data to normal scores, krige values in the “Gaussian domain” and then back-transform the estimates and uncertainty measures to the “original domain.” An additional advantage of the Gaussian transform is that spatial variability is easier to model from the normal scores because the transformation reduces effects of extreme values. There are, however, difficulties with this methodology, particularly, choosing the transformation to be used and back-transforming the estimates in such a way as to ensure that the estimation is conditionally unbiased. The problem has been solved for cases in which the experimental data follow some particular type of distribution. In general, however, it is not possible to verify distributional assumptions on the basis of experimental histograms calculated from relatively few data and where the uncertainty is such that several distributional models could fit equally well. For the general case, we propose an empirical maximum likelihood method in which transformation to normality is via the empirical probability distribution function. Although the Gaussian domain simple kriging estimate is identical to the maximum likelihood estimate, we propose use of the latter, in the form of a likelihood profile, to solve the problem of conditional unbiasedness in the back-transformed estimates. Conditional unbiasedness is achieved by adopting a Bayesian procedure in which the likelihood profile is the posterior distribution of the unknown value to be estimated and the mean of the posterior distribution is the conditionally unbiased estimate. The likelihood profile also provides several ways of assessing the uncertainty of the estimation. Point estimates, interval estimates, and uncertainty measures can be calculated from the posterior distribution.     

极大熵准则在降雨强度-历时-频率模型参数估计中的应用

根据频率与重现期的关系推导出三类常用降雨强度-历时-频率模型的无条件及条件概率分布(概率密度)函数及与模型相对应的约束。极大熵与极大似然准则产生一致估计,本文尝试基于极大熵准则建立降雨强度-历时-频率模型参数估计的优化模型,应用模拟退火算法求解该优化模型。根据比较发现,极大熵估计有时比常用的极大似然估计和最小二乘估计更精确。     

The Second-Order Stationary Universal Kriging Model Revisited

Universal kriging originally was developed for problems of spatial interpolation if a drift seemed to be justified to model the experimental data. But its use has been questioned in relation to the bias of the estimated underlying variogram (variogram of the residuals), and furthermore universal kriging came to be considered an old-fashioned method after the theory of intrinsic random functions was developed. In this paper the model is reexamined together with methods for handling problems in the inference of parameters. The efficiency of the inference of covariance parameters is shown in terms of bias, variance, and mean square error of the sampling distribution obtained by Monte Carlo simulation for three different estimators (maximum likelihood, bias corrected maximum likelihood, and restricted maximum likelihood). It is shown that unbiased estimates for the covariance parameters may be obtained but if the number of samples is small there can be no guarantee of good estimates (estimates close to the true value) because the sampling variance usually is large. This problem is not specific to the universal kriging model but rather arises in any model where parameters are inferred from experimental data. The validity of the estimates may be evaluated statistically as a risk function as is shown in this paper.     

A robust cylindrical fitting to point cloud data

Environmental, engineering and industrial modelling of natural features (e.g. trees) and man-made features (e.g. pipelines) requires some form of fitting of geometrical objects such as cylinders, which is commonly undertaken using a least-squares method that—in order to get optimal estimation—assumes normal Gaussian distribution. In the presence of outliers, however, this assumption is violated leading to a Gaussian mixture distribution. This study proposes a robust parameter estimation method, which is an improved and extended form of vector algebraic modelling. The proposed method employs expectation maximisation and maximum likelihood estimation (MLE) to find cylindrical parameters in case of Gaussian mixture distribution. MLE computes the model parameters assuming that the distribution of model errors is a Gaussian mixture corresponding to inlier and outlier points. The parameters of the Gaussian mixture distribution and the membership functions of the inliers and outliers are computed using an expectation maximisation algorithm from the histogram of the model error distribution, and the initial guess values for the model parameters are obtained using total least squares. The method, illustrated by a practical example from a terrestrial laser scanning point cloud, is novel in that it is algebraic (i.e. provides a non-iterative solution to the global maximisation problem of the likelihood function), is practically useful for any type of error distribution model and is capable of separating points of interest and outliers.     

A non-parametric bivariate entropy estimator for spatial processes

In this paper, entropy is presented as an alternative measure to characterize the bivariate distribution of a stationary spatial process. This non-parametric estimator attempts to quantify the concept of spatial ordering, and it provides a measure of how Gaussian the experimental bivariate distribution is. The concept of entropy is explained and the classical definition presented, along with some important results. In particular, the reader is reminded that, for a known mean and covariance, the bivariate Gaussian distribution maximizes entropy. A relative entropy estimator is introduced in order to measure departure of an experimental bivariate distribution from the bivariate Gaussian. Two case studies are presented as examples.     

先验概率分布及似然函数模型的选择对边坡可靠度评价影响的定量评估

受工程勘察成本及试验场地限制,可获得的试验数据通常有限,基于有限的试验数据难以准确估计岩土参数统计特征和边坡可靠度。贝叶斯方法可以融合有限的场地信息降低对岩土参数不确定性的估计进而提高边坡可靠度水平。但是,目前的贝叶斯更新研究大多假定参数先验概率分布为正态、对数正态和均匀分布,似然函数为多维正态分布,这种做法的合理性有待进一步验证。总结了岩土工程贝叶斯分析常用的参数先验概率分布及似然函数模型,以一个不排水黏土边坡为例,采用自适应贝叶斯更新方法系统探讨了参数先验概率分布和似然函数对空间变异边坡参数后验概率分布推断及可靠度更新的影响。计算结果表明：参数先验概率分布对空间变异边坡参数后验概率分布推断及可靠度更新均有一定的影响,选用对数正态和极值I型分布作为先验概率分布推断的参数后验概率分布离散性较小。选用Beta分布和极值I型分布获得的边坡可靠度计算结果分别偏于保守和危险,选用对数正态分布获得的边坡可靠度计算结果居中。相比之下,似然函数的影响更加显著。与其他类型似然函数相比,由多维联合正态分布构建的似然函数可在降低对岩土参数不确定性估计的同时,获得与场地信息更为吻合的计算结果。另外,构建似然函数时不同位置处测量误差之间的自相关性对边坡后验失效概率也具有一定的影响。     

Projection Pursuit Multivariate Transform

Transforming complex multivariate geological data to a Gaussian distribution is an important and challenging problem in geostatistics. A variety of transforms are available for this goal, but struggle with high dimensional data sets. Projection pursuit density estimation (PPDE) is a well-established nonparametric method for estimating the joint density of multivariate data. A central component of the PPDE algorithm transforms the original data toward a multivariate Gaussian distribution. The PPDE approach is modified to map complex data to a multivariate Gaussian distribution within a geostatistical modeling context. Traditional modeling may then take place on the transformed Gaussian data, with a back-transform used to return simulated variables to their original units. This approach is referred to as the projection pursuit multivariate transform (PPMT). The PPMT shows the potential to be an effective means for modeling high dimensional and complex geologic data. The PPMT algorithm is developed before discussing considerations and limitations. A case study compares modeling results against more common techniques to demonstrate the value and place of the PPMT within geostatistics.     

The Correlation Structure of Matheron's Classical Variogram Estimator Under Elliptically Contoured Distributions

The classical variogram estimator proposed by Matheron can be written as a quadratic form of the observations. When data have an elliptically contoured distribution with constant mean, the correlation between the classical variogram estimator at two different lags is a function of the spatial design matrix, the covariance matrix, and the kurtosis. Several specific cases are studied closely. A subclass of elliptically contoured distributions with a particular family of covariance matrices is shown to possess exactly the same correlation structure for the classical variogram estimator as the multivariate independent Gaussian distribution. The consequences on variogram fitting by generalized least squares are discussed.     

The Correlation Structure of Matheron's Classical Variogram Estimator Under Elliptically Contoured Distributions

The classical variogram estimator proposed by Matheron can be written as a quadratic form of the observations. When data have an elliptically contoured distribution with constant mean, the correlation between the classical variogram estimator at two different lags is a function of the spatial design matrix, the covariance matrix, and the kurtosis. Several specific cases are studied closely. A subclass of elliptically contoured distributions with a particular family of covariance matrices is shown to possess exactly the same correlation structure for the classical variogram estimator as the multivariate independent Gaussian distribution. The consequences on variogram fitting by generalized least squares are discussed.     

Bayesian Spatial Prediction for Discrete Closed Skew Gaussian Random Field

Most approaches in statistical spatial prediction assume that the spatial data are realizations of a Gaussian random field. However, this assumption is hard to justify for most applications. When the distribution of data is skewed but otherwise has similar properties to the normal distribution, a closed skew normal distribution can be used for modeling their skewness. Closed skew normal distribution is an extension of the multivariate skew normal distribution and has the advantage of being closed under marginalization and conditioning. In this paper, we generalize Bayesian prediction methods using closed skew normal distributions. A simulation study is performed to check the validity of the model and performance of the Bayesian spatial predictor. Finally, our prediction method is applied to Bayesian spatial prediction on the strain data near Semnan, Iran. The mean-square error of cross-validation is improved by the closed skew Gaussian model on the strain data.     

Calculation of the Inverse of the Covariance

In reservoir characterization, the covariance is often used to describe the spatial correlation and variation in rock properties or the uncertainty in rock properties. The inverse of the covariance, on the other hand, is seldom discussed in geostatistics. In this paper, I show that the inverse is required for simulation and estimation of Gaussian random fields, and that it can be identified with the differential operator in regularized inverse theory. Unfortunately, because the covariance matrix for parameters in reservoir models can be extremely large, calculation of the inverse can be a problem. In this paper, I discuss four methods of calculating the inverse of the covariance, two of which are analytical, and two of which are purely numerical. By taking advantage of the assumed stationarity of the covariance, none of the methods require inversion of the full covariance matrix.     

Non-stationary Geostatistical Modeling Based on Distance Weighted Statistics and Distributions

A common assumption in geostatistics is that the underlying joint distribution of possible values of a geological attribute at different locations is stationary within a homogeneous domain. This joint distribution is commonly modeled as multi-Gaussian, with correlations defined by a stationary covariance function. This results in attribute maps that fail to reproduce local changes in the mean, in the variance and, particularly, in the spatial continuity. The proposed alternative is to build local distributions, variograms, and correlograms. These are inferred by weighting the samples depending on their distance to selected locations. The local distributions are locally transformed into Gaussian distributions embedding information on the local histogram. The distance weighted experimental variograms and correlograms are able to adapt to local changes in the direction and range of spatial continuity. The automatically fitted local variogram models and the local Gaussian transformation parameters are used in spatial estimation algorithms assuming local stationarity. The resulting maps are rich in nonstationary spatial features. The proposed process implies a higher computational effort than traditional stationary techniques, but if data availability allows for a reliable inference of the local distributions and statistics, a higher accuracy of estimates can be achieved.     

Quadratic covariance estimation and equivalence of predictions

This article illustrates the use of linear and nonlinear regression models to obtain quadratic estimates of covariance parameters. These models lead to new insights into the motivation behind estimation methods, the relationships between different methods, and the relationship of covariance estimation to prediction. In particular, we derive the standard estimating equations for minimum norm quadratic unbiased translation invariant estimates (MINQUEs) from an appropriate linear model. Connections between the linear model, minimum variance quadratic unbiased translation invariant estimates (MIVQUEs), and MINQUEs are examined and we provide a minimum norm justification for the use of one-step normal theory maximum likelihood estimates. A nonlinear regression model is used to define MINQUEs for nonlinear covariance structures and obtain REML estimates. Finally, the equivalence of predictions under various models is examined when covariance parameters are estimated. In particular, we establish that when using MINQUE, iterative MINQUE, or restricted maximum likelihood (REML) estimates, the choice between a stationary covariance function and an intrinsically stationary semivariogram is irrelevant to predictions and estimated prediction variances.     

Polynomial expansions in likelihoods for spatial data: A case study

This paper illustrates the computational benefits of polynomial representations for quantities in the likelihood function for the spatial linear model based on the power covariance scheme. These benefits include a comprehensive study of likelihoods and maximum likelihood estimators for data. For simplicity, we focus on a relatively simple covariance scheme and data observed at equal intervals along a transect; we briefly indicate how generalizations to more complicated covariance functions and higher dimensions will operate.     

Positive definiteness is not enough

Geostatisticians know that the mathematical functions chosen to represent spatial covariances and variograms must have the appropriate type of positive definiteness, but they may not realize that there are restrictions on the types of covariances and variograms that are compatible with particular distributions. This paper gives some examples showing that (1) the spherical model is not compatible with the multivariate lognormal distribution if the coefficient of variation is 2.0 or more (even in 1-D), and (2) the Gaussian covariance and several other models are not compatible with indicator random functions. As these examples concern quite different types of random functions, it is clear that there is a general problem of compatibility between spatial covariance models (or variograms) and a specified multivariate distribution. The problem arises with all distributions except the multivariate normal, and not just the two cited here. The need for a general theorem giving the necessary and sufficient conditions for a covariance or a variogram to be compatible with a particular distribution is stressed.     

Multivariate Spatial Modeling for Geostatistical Data Using Convolved Covariance Functions

Soil pollution data collection typically studies multivariate measurements at sampling locations, e.g., lead, zinc, copper or cadmium levels. With increased collection of such multivariate geostatistical spatial data, there arises the need for flexible explanatory stochastic models. Here, we propose a general constructive approach for building suitable models based upon convolution of covariance functions. We begin with a general theorem which asserts that, under weak conditions, cross convolution of covariance functions provides a valid cross covariance function. We also obtain a result on dependence induced by such convolution. Since, in general, convolution does not provide closed-form integration, we discuss efficient computation. We then suggest introducing such specification through a Gaussian process to model multivariate spatial random effects within a hierarchical model. We note that modeling spatial random effects in this way is parsimonious relative to say, the linear model of coregionalization. Through a limited simulation, we informally demonstrate that performance for these two specifications appears to be indistinguishable, encouraging the parsimonious choice. Finally, we use the convolved covariance model to analyze a trivariate pollution dataset from California.