Algorithms for box constrained M-estimation: Fitting large data sets with application to geodynamics

Because of occurrence of ill-conditioning and outliers, use of direct Least Squares fit is now in decline, while robust M-estimators are currently attracting attention. We present here new algorithms based on the Spingarn Partial Inverse proximal decomposition method for L1 and Huber-M estimation that take into account both primal and dual aspects of the underlying optimization problem. The result is a family of highly parallel algorithms. Globally convergent, they are attractive for large scale problems as encountered in geodesy, especially in the field of Earth Orientation data analysis. The method is extended to handle box constrained problems. To obtain an efficient implementation, remedies are introduced to ensure efficiency in the case of models with less than full rank. Numerical results are discussed. Robust data pre-conditioning is shown to induce faster algorithm convergence. Practical implementation aspects are presented with application to series describing the Earth Rotation.     

Visual data exploration for hydrological analysis

Hydrological research projects for integrated water resources management such as the IWAS initiative often accumulate large amounts of heterogeneous data from different sources. Given the number of partners taking part in such projects, surveying and accessing the available data sets, as well as searching for a defined subset, becomes increasingly difficult. We propose an integrated approach for a system combining visual data management and numerical simulation which allows to survey and select data sets based on keywords such as a region of interest or given indicators. An adequate 3D visualisation of such subsets helps to convey information and significantly supports the assessment of relations between different types of data. Furthermore, the interface between the visual data management system and finite element codes allows for the straightforward integration of information into the numerical simulation process and the subsequent visualisation of simulation results in a geographical context. We demonstrate typical workflows for integration and processing within the system based on data from the IWAS model region in Saudi Arabia and the TERENO Bode Observatory in the Harz Mountains in Germany. In addition, we present examples for data import and export based on established standard file formats.     

Error analysis for bracketed phase equilibrium data

The proper treatment of experimental errors associated with experimental phase boundary determinations and derived results is necessary for a full interpretation and for critical comparison with other data. In the past, mathematical approximations used in treating phase equilibrium data have overestimated standard deviations and confidence limits by as much as a factor of two, thus hindering the proper interpretation of results. We have derived the correct expressions for calculating uncertainties in the location of a phase boundary and in quantities derived from a boundary such as the free energy of formation of a phase. For the special case of a pair of experimental; points with equal errors bracketing the phase boundary, we have prepared equations and graphs which are particularly easy to use. Our approach can be extended to more problems involving the analysis of phase equilibrium experiments.     

Quantifying a critical marl thickness for vertical fracture extension using field data and numerical experiments

In fractured reservoirs characterized by low matrix permeability,fracture networks control the main fluid flow paths.However,in layered reservoirs,the vertical extension of fractures is often restricted to single layers.In this study,we explored the effect of changing marl/shale thickness on fracture extension using comprehensive field data and numerical modeling.The field data were sampled from coastal exposures of Liassic limestone-marl/shale alternations in Wales and Somerset(Bristol Channel Basin,UK).The vertical fracture traces of more than 4000 fractures were mapped in detail.Six sections were selected to represent a variety of layer thicknesses.Besides the field data also thin sections were analyzed.Numerical models of fracture extension in a two-layer limestone-marl system were based on field data and laboratory measurements of Young's moduli.The modeled principal stress magnitude σ_3 along the lithological contact was used as an indication for fracture extension through marls.Field data exhibit good correlation(R~2=0.76) between fracture extension and marl thickness,the thicker the marl layer the fewer fractures propagate through.The model results show that almost no tensile stress reaches the top of the marl layer when the marls are thicker than 30 cm.For marls that are less than 20 cm,the propagation of stress is more dependent on the stiffness of the marls.The higher the contrast between limestone and marl stiffness the lower the stress that is transmitted into the marl layer.In both model experiments and field data the critical marl thickness for fracture extension is ca.15-20 cm.This quantification of critical marl thicknesses can be used to improve predictions of fracture networks and permeability in layered rocks.Up-or downsampling methods often ignore spatially continuous impermeable layers with thicknesses that are under the detection limit of seismic data.However,ignoring these layers can lead to overestimates of the overall permeability.Therefore,the understanding of how fractures propagate and terminate through impermeable layers will help to improve the characterization of conventional reservoirs.     

Compositional data analysis

Compositional data occur naturally in the geosciences — tables of chemical analyses, rock-compositions, sedimentary proportions, pollen-analytical tables, etc. The statistical analysis of such data requires special techniques and it is not possible to use standard methods of computing correlation coefficients and carry out multivariate statistical analyses without the risk of incurring grave mistakes. The special property of compositional data, to wit, the fact that the determinations on each specimen sum to a constant, means that the variables involved in the study occur in constrained space defined by the simplex , a restricted part of real space.     

Two GUIs-based analysis tool for spectroradiometer data pre-processing

A new graphical user interface (GUI) for pre-processing reflectance spectra, built using MATLAB and expressly designed for the ASD FieldSpec® spectroradiometer, was developed to solve problems that generally affect experimental ASD data. The GUI is characterised by an easily readable, graphic visualisation of spectra, from which the absorption band depth (ABD) can be obtained for a selected wavelength. The output format of the ASD data is a binary file with an .asd extension. The binary file, that provides a single spectrum, can be processed using a software functionality, by means of a GUI, that allows to select one or more binary files to produce a spectral library in a unique .txt file. The spectral reflectance is re-calibrated with the “convex-hull” methodology to eliminate the convex shape, which is typical of reflectance spectra. Different examples of the use of the new GUI are provided.     

Self-organizing maps for geoscientific data analysis: geological interpretation of multidimensional geophysical data

Data interpretation is a common task in geoscientific disciplines. Interpretation difficulties occur especially if the data that have to be interpreted are of arbitrary dimension. This paper describes the application of a statistical method, called self-organizing mapping (SOM), to interpret multidimensional, non-linear, and highly noised geophysical data for purposes of geological prediction. The underlying theory is explained, and the method is applied to a six-dimensional seismic data set. Results of SOM classifications can be represented as two-dimensional images, called feature maps. Feature maps illustrate the complexity and demonstrate interrelations between single features or clusters of the complete feature space. SOM images can be visually described and easily interpreted. The advantage is that the SOM method considers interdependencies between all geophysical features at each instance. An application example of an automated geological interpretation based on the geophysical data is shown.     

The perils of taxonomic inconsistency in quantitative palaeoecology: experiments with testate amoeba data

Payne, R. J., Lamentowicz, M. & Mitchell, E. A. D. 2010: The perils of taxonomic inconsistency in quantitative palaeoecology: experiments with testate amoeba data. Boreas, 10.1111/j.1502‐3885.2010.00174.x. ISSN 0300‐9483. A fundamental requirement of quantitative palaeoecology is consistent taxonomy between a modern training set and palaeoecological data. In this study we assess the possible consequences of violation of this requirement by simulating taxonomic errors in testate amoeba data. Combinations of easily confused taxa were selected, and data manipulated to reflect confusion of these taxa; transfer functions based on unmodified data were then applied to these modified data sets. Initially these experiments were carried out one error at a time using four modern training sets; subsequently, multiple errors were separately simulated both in four modern training sets and in four palaeoecological data sets. Some plausible taxonomic confusions caused major biases in reconstructed values. In the case of two palaeoecological data sets, a single consistent taxonomic error was capable of changing the pattern of environmental reconstruction beyond all recognition, totally removing any real palaeoenvironmental signal. The issue of taxonomic consistency is one that many researchers would rather ignore; our results show that the consequences of this may ultimately be severe.     

Online weather data analysis and visualization tools for applications in ecoinformatics

As large quantities of physical data are always collected for Ecoinformatics research, it is difficult for them to be cleaned, shared, visualized, and analyzed by research collaborators. To resolve this difficulty, this study presents online weather data analysis and visualization cyber-infrastructures consisting of (1) online weather data analysis and visualization tools and (2) near real-time online weather data portal. Firstly, these online tools at www.twibl.org/weather provide data sharing in three web pages: information on instruments and site; data access protected by simple password security; data analysis and visualization services so-called “Ecoinfows”. Secondly, the near real-time online weather data portal for visualizing and forecasting weather data from cloud storage of many automatic weather stations is online at www.twibl.org/aaportal. To overcome speed and accessibility problems, we developed these tools with many technologies - i.e. cloud computing, online computing XML (webMathematica), and binary access data conversion.     

Method and analysis for the upscaling of structural data

3D geological models are created to integrate a set of input measurements into a single geological model. There are many problems with this approach, as there is uncertainty in all stages of the modelling process, from initial data collection to the approach used in the modelling scheme itself to calculate the geological model. This study looks at the uncertainty inherent in geological models due to data density and introduces a novel method to upscale geological data that optimises the information in the initial dataset. This method also provides the ability for the dominant trend of a geological dataset to be determined at different scales. By using self-organizing maps (SOM's) to examine the different metrics used to quantify a geological model, we allow for a larger range of metrics to be used compared to traditional statistical methods, due to the SOM's ability to deal with incomplete datasets. The classification of the models into clusters based on the geological metrics using k-means clustering provides a useful insight into the models that are most similar and models that are statistical outliers. Our approach is guided and can be calculated on any input dataset of this type to determine the effect that data density will have on a resultant model. These models are all statistical derivations that represent simplifications and different scales of the initial dataset and can be used to interrogate the scale of observations.     

A statistical analysis of exploration geochemical data for uranium

This paper describes a statistical analysis of reconnaissance exploration geochemical data for uranium. Three sets of data were analyzed, as they related to: (a) Charlotte-Winston-Salem and (b) Charlotte two-degree NTMS quadrangles of the south-eastern U.S.A. The coefficient of variation for uranium in each of the three sets of data was less than unity and hence no transformation of the original variable was needed. These data were subjected to correlogram analysis. A first-order Markovian model of the type: Y₀ exp (-a |p|) was fit by the least-squares method to serial correlation coefficients of these data using the method proposed by Deming (1948). The model was tested by computing the variance-volume relationship for assumed individual blocks of unit length. The noise in the input (record) was eliminated by the application of an optimum bilateral exponential smoothing technique developed by Agterberg. A comparison of spectral density estimates obtained by a maximum entropy method employing Yule-Walker equations and the Burg algorithm was also made. The prediction error coefficients needed to determine the order of the autoregressive process and hence the spectral densities were determined in both cases for the three sets of data.On leave from National Geophysical Research Institute, Hyderabad-7, India.     

3D modelling strategy for weather radar data analysis

Weather radar data, which have obvious spatial characteristics, represent an important and essential data source for weather identification and prediction, and the multi-dimensional visualization and analysis of such data in a three-dimensional (3D) environment are important strategies for meteorological assessments of potentially disastrous weather. The previous studies have generally used regular 3D raster grids as a basic structure to represent radar data and reconstruct convective clouds. However, conducting weather radar data analyses based on regular 3D raster grids is time-consuming and inefficient, because such analyses involve considerable amounts of tedious data interpolation, and they cannot be used to address real-time situations or provide rapid-response solutions. Therefore, a new 3D modelling strategy that can be used to efficiently represent and analyse radar data is proposed in this article. According to the mode by which the radar data are obtained, the proposed 3D modelling strategy organizes the radar data using logical objects entitled radar-point, radar-line, radar-sector, and radar-cluster objects. In these logical objects, the radar point is the basic object that carries the real radar data unit detected from the radar scan, and the radar-line, radar-sector, and radar-cluster objects organize the radar-point collection in different spatial levels that are consistent with the intrinsic spatial structure of the radar scan. Radar points can be regarded as spatial points, and their spatial structure can support logical objects; thus, the radar points can be flexibly connected to construct continuous surface data with quads and volume data with hexahedron cells without additional tedious data interpolation. This model can be used to conduct corresponding operations, such as extracting an isosurface with the marching cube method and a radar profile with a designed sectioning algorithm to represent the outer and inner structure of a convective cloud. Finally, a case study is provided to verify that the proposed 3D modelling strategy has a better performance in radar data analysis and can intuitively and effectively represent the 3D structure of convective clouds.     

The use of the Levy-stable distribution for geophysical data analysis

The suitability of the Levy-stable distribution for geophysical data from a sedimentary environment has been investigated. Four types of geophysical well logging measurements from the Choshuichi alluvial fan in central Taiwan were analyzed—long-normal electrical resistivity, short-normal electrical resistivity, natural gamma-ray logging, and spontaneous potential. The results demonstrate that the Levy-stable distribution describes the geophysical data better than a Gaussian distribution. Because the Levy-stable distribution is flexible in fitting data from strongly heterogeneous fields and the traditional Gaussian distribution is only its special case, the Levy-stable distribution should be considered when analyzing geophysical data.     

形变监测高分辨率InSAR数据的精度分析

收集江苏高分辨率InSAR地面沉降监测结果及兴化、淮安同期水准测量资料和基岩标同期监测资料,利用79个水准点测量数据、7个基岩标（分别选取不同时段）测量数据共98个数据样本,将InSAR技术与水准测量及基岩标测量技术的监测结果进行对比发现,监测结果一致性率为86.73%,平均差值为2.08 mm,标准差为2.50 mm,最大差值为18.64 mm,表明InSAR可进行半定量形变监测,但在高层建筑、排水管线、大坝、桥梁等需高精度形变监测的工程方面仍不能替代水准测量。     

小波分析的原理及其在电磁感应仪EM38数据分析中的应用

利用小波变换对澳大利亚新南威尔士Mandamah地区所测得EM38数据进行全局阈值消噪处理,通过分解与重构,从中提取有效信息,去除干扰,得到去噪后的数据。将去噪后的数据与地球化学铜元素的异常特征和已知的铜矿化区域进行分析与对比,推测出该地区的铜矿化预测区。该方法能有效地去除噪声,又不会损失有用信息。实际资料处理结果表明：小波分析方法在EM38数据处理中可以起到独特的作用与效果。     

Rockfalls: analysis of the block fragmentation through field experiments

Landslides - Fragmentation is a common feature of rockfall that exerts a strong control on the trajectories of the generated blocks, the impact energies, and the runout. In this paper, we present a...