Relative model score: a scoring rule for evaluating ensemble simulations with application to microbial soil respiration modeling

This paper defines a new scoring rule, namely relative model score (RMS), for evaluating ensemble simulations of environmental models. RMS implicitly incorporates the measures of ensemble mean accuracy, prediction interval precision, and prediction interval reliability for evaluating the overall model predictive performance. RMS is numerically evaluated from the probability density functions of ensemble simulations given by individual models or several models via model averaging. We demonstrate the advantages of using RMS through an example of soil respiration modeling. The example considers two alternative models with different fidelity, and for each model Bayesian inverse modeling is conducted using two different likelihood functions. This gives four single-model ensembles of model simulations. For each likelihood function, Bayesian model averaging is applied to the ensemble simulations of the two models, resulting in two multi-model prediction ensembles. Predictive performance for these ensembles is evaluated using various scoring rules. Results show that RMS outperforms the commonly used scoring rules of log-score, pseudo Bayes factor based on Bayesian model evidence (BME), and continuous ranked probability score (CRPS). RMS avoids the problem of rounding error specific to log-score. Being applicable to any likelihood functions, RMS has broader applicability than BME that is only applicable to the same likelihood function of multiple models. By directly considering the relative score of candidate models at each cross-validation datum, RMS results in more plausible model ranking than CRPS. Therefore, RMS is considered as a robust scoring rule for evaluating predictive performance of single-model and multi-model prediction ensembles.     

Geophysical flows under location uncertainty,Part II Quasi-geostrophy and efficient ensemble spreading

Models under location uncertainty are derived assuming that a component of the velocity is uncorrelated in time. The material derivative is accordingly modified to include an advection correction, inhomogeneous and anisotropic diffusion terms and a multiplicative noise contribution. In this paper, simplified geophysical dynamics are derived from a Boussinesq model under location uncertainty. Invoking usual scaling approximations and a moderate influence of the subgrid terms, stochastic formulations are obtained for the stratified Quasi-Geostrophy and the Surface Quasi-Geostrophy models. Based on numerical simulations, benefits of the proposed stochastic formalism are demonstrated. A single realization of models under location uncertainty can restore small-scale structures. An ensemble of realizations further helps to assess model error prediction and outperforms perturbed deterministic models by one order of magnitude. Such a high uncertainty quantification skill is of primary interests for assimilation ensemble methods. MATLAB^® code examples are available online.     

扬铜地震构造带中强地震灰色预测

本文用灰色灾变预测方法，以两次中强地震的间隔时间或历次中强地震发生时间作为原始建模数列，分别建立扬铜地震构造带Ｍ＿Ｓ≥４．Ｏ和Ｍ＿ｓ≥５．０地震灰色预测模型。通过近年对该带中强地震发生时间预测实践，证明该模型的预测精度和效果均令人满意，说明灰色预测方法可以作为地震预报综合研究的内容之一。     

The Area Skill Score Statistic for Evaluating Earthquake Predictability Experiments

Rigorous predictability experimentation requires a statistical characterization of the performance metric used to evaluate the participating models. We explore the properties of the area skill score measure and consider issues related to experimental discretization. For the case of continuous alarm functions and continuous observations, we present exact analytical solutions that describe the distribution of the area skill score for unskilled predictors, and we also describe how a Gaussian distribution with known mean and variance can be used to approximate the area skill score distribution. We quantify the deviation of the exact distribution from the Gaussian approximation by specifying the kurtosis excess as a function of the number of observed target earthquakes. For numerical earthquake predictability experiments that involve discretization of the study region and observations, we explore simulation procedures for estimating the area skill score distribution, and we present efficient algorithms for various experimental scenarios. When more than one target earthquake occurs within a given space/time/magnitude bin, the probabilities of predicting individual events are not independent, and this requires special consideration. Having presented the statistical properties of the area skill score, we describe and illustrate a preliminary procedure for comparing earthquake prediction strategies based on alarm functions.     

江苏及邻区中强震的灰色马尔柯夫模型

为了探索新的地震灾变预测途径，探讨了集不同手段之长所构筑的一种新模式，其方法是在灰色模型（ＧＭ（１，１））的基础上划分多个与预测值平行的状态区间，考察下一步状态的转移概率，形成灰色马尔柯预测模型，将震级和时间等作为预测要素，通过模型给出了其变化的灰区间和是了大可能值，实例表明，由于灰色预测和马尔柯夫转换概率矩阵两者互补优势，形成的新模型优化了预测结果，作为新方法，灰色马尔柯夫模型值得深入研究。     

Forecasting Ability of a Multi-Renewal Seismicity Model

The inter-event time (IET) is sometimes used as a basis for prediction of large earthquakes. It is the case when theoretical analysis of prediction is possible. Quite recently, a specific IET model was suggested for dynamic probabilistic prediction of \( M \ge 5.5 \) events in Italy (http://earthquake.bo.ingv.it). In this study we analyze some aspects of the statistical estimation of the model and its predictive ability. We find that more or less effective prediction is possible within four out of 34 seismotectonic zones where seismicity rate or clustering of events is relatively high. We show that, in the framework of the model, one can suggest a simple zone-independent strategy, which practically optimizes the relative number of non-accidental successes, or the Hanssen-Kuiper (HK) skill score. This quasi-optimal strategy declares alarm in a zone for the first 2.67 years just after the occurrence of each large event in the zone. The optimal HK skill score values are about 26 % for the three most active zones, and 2–10 % for the 26 least active zones. However, the number of false alarm time intervals per one event in each of the zones is unusually high: about 0.7 and 0.8–0.95, respectively. Both these theoretical estimations are important because any prospective testing of the model is unrealistic in most of the zones during a reasonable time. This particular analysis requires a discussion of the following issues of general interest: a specific approach to the analysis of predictions vs. the standard CSEP testing approach; prediction vs. forecasting; HK skill score vs. probability gain; the total forecast error diagram and connected false alarms.     

Uncertainty-based multi-criteria calibration of rainfall-runoff models: a comparative study

This study compares formal Bayesian inference to the informal generalized likelihood uncertainty estimation (GLUE) approach for uncertainty-based calibration of rainfall-runoff models in a multi-criteria context. Bayesian inference is accomplished through Markov Chain Monte Carlo (MCMC) sampling based on an auto-regressive multi-criteria likelihood formulation. Non-converged MCMC sampling is also considered as an alternative method. These methods are compared along multiple comparative measures calculated over the calibration and validation periods of two case studies. Results demonstrate that there can be considerable differences in hydrograph prediction intervals generated by formal and informal strategies for uncertainty-based multi-criteria calibration. Also, the formal approach generates definitely preferable validation period results compared to GLUE (i.e., tighter prediction intervals that show higher reliability) considering identical computational budgets. Moreover, non-converged MCMC (based on the standard Gelman–Rubin metric) performance is reasonably consistent with those given by a formal and fully-converged Bayesian approach even though fully-converged results requires significantly larger number of samples (model evaluations) for the two case studies. Therefore, research to define alternative and more practical convergence criteria for MCMC applications to computationally intensive hydrologic models may be warranted.     

Links, comparisons and extensions of the geographically weighted regression model when used as a spatial predictor

In this study, we link and compare the geographically weighted regression (GWR) model with the kriging with an external drift (KED) model of geostatistics. This includes empirical work where models are performance tested with respect to prediction and prediction uncertainty accuracy. In basic forms, GWR and KED (specified with local neighbourhoods) both cater for nonstationary correlations (i.e. the process is heteroskedastic with respect to relationships between the variable of interest and its covariates) and as such, can predict more accurately than models that do not. Furthermore, on specification of an additional heteroskedastic term to the same models (now with respect to a process variance), locally-accurate measures of prediction uncertainty can result. These heteroskedastic extensions of GWR and KED can be preferred to basic constructions, whose measures of prediction uncertainty are only ever likely to be globally-accurate. We evaluate both basic and heteroskedastic GWR and KED models using a case study data set, where data relationships are known to vary across space. Here GWR performs well with respect to the more involved KED model and as such, GWR is considered a viable alternative to the more established model in this particular comparison. Our study adds to a growing body of empirical evidence that GWR can be a worthy predictor; complementing its more usual guise as an exploratory technique for investigating relationships in multivariate spatial data sets.     

Forecast of Low Visibility and Fog from NCEP: Current Status and Efforts

Based on the visibility analysis data during November 2009 through April 2010 over North America from the Aviation Digital Database Service (ADDS), the performance of low visibility/fog predictions from the current operational 12?km-NAM, 13?km-RUC and 32?km-WRF-NMM models at the National Centers for Environmental Prediction (NCEP) was evaluated. The evaluation shows that the performance of the low visibility/fog forecasts from these models is still poor in comparison to those of precipitation forecasts from the same models. In order to improve the skill of the low visibility/fog prediction, three efforts have been made at NCEP, including application of a rule-based fog detection scheme, extension of the NCEP Short Range Ensemble Forecast System (SREF) to fog ensemble probabilistic forecasts, and a combination of these two applications. How to apply these techniques in fog prediction is described and evaluated with the same visibility analysis data over the same period of time. The evaluation results demonstrate that using the multi-rule-based fog detection scheme significantly improves the fog forecast skill for all three models relative to visibility-diagnosed fog prediction, and with a combination of both rule-based fog detection and the ensemble technique, the performance skill of fog forecasting can be further raised.     

Comparative study of monthly inflow prediction methods for the Three Gorges Reservoir

Due to the complexity of influencing factors and the limitation of existing scientific knowledge, current monthly inflow prediction accuracy is unable to meet the requirements of various water users yet. A flow time series is usually considered as a combination of quasi-periodic signals contaminated by noise, so prediction accuracy can be improved by data preprocess. Singular spectrum analysis (SSA), as an efficient preprocessing method, is used to decompose the original inflow series into filtered series and noises. Current application of SSA only selects filtered series as model input without considering noises. This paper attempts to prove that noise may contain hydrological information and it cannot be ignored, a new method that considerers both filtered and noises series is proposed. Support vector machine (SVM), genetic programming (GP), and seasonal autoregressive (SAR) are chosen as the prediction models. Four criteria are selected to evaluate the prediction model performance: Nash–Sutcliffe efficiency, Water Balance efficiency, relative error of annual average maximum (RE_max) monthly flow and relative error of annual average minimum (RE_min) monthly flow. The monthly inflow data of Three Gorges Reservoir is analyzed as a case study. Main results are as following: (1) coupling with the SSA, the performance of the SVM and GP models experience a significant increase in predicting the inflow series. However, there is no significant positive change in the performance of SAR (1) models. (2) After considering noises, both modified SSA-SVM and modified SSA-GP models perform better than SSA-SVM and SSA-GP models. Results of this study indicated that the data preprocess method SSA can significantly improve prediction precision of SVM and GP models, and also proved that noises series still contains some information and has an important influence on model performance.     

Global Optimal Design of Ground Water Monitoring Network Using Embedded Kriging

We present a methodology for global optimal design of ground water quality monitoring networks using a linear mixed-integer formulation. The proposed methodology incorporates ordinary kriging (OK) within the decision model formulation for spatial estimation of contaminant concentration values. Different monitoring network design models incorporating concentration estimation error, variance estimation error, mass estimation error, error in locating plume centroid, and spatial coverage of the designed network are developed. A big-M technique is used for reformulating the monitoring network design model to a linear decision model while incorporating different objectives and OK equations. Global optimality of the solutions obtained for the monitoring network design can be ensured due to the linear mixed-integer programming formulations proposed. Performances of the proposed models are evaluated for both field and hypothetical illustrative systems. Evaluation results indicate that the proposed methodology performs satisfactorily. These performance evaluation results demonstrate the potential applicability of the proposed methodology for optimal ground water contaminant monitoring network design.     

Latin hypercube approach to estimate uncertainty in ground water vulnerability

A methodology is proposed to quantify prediction uncertainty associated with ground water vulnerability models that were developed through an approach that coupled multivariate logistic regression with a geographic information system (GIS). This method uses Latin hypercube sampling (LHS) to illustrate the propagation of input error and estimate uncertainty associated with the logistic regression predictions of ground water vulnerability. Central to the proposed method is the assumption that prediction uncertainty in ground water vulnerability models is a function of input error propagation from uncertainty in the estimated logistic regression model coefficients (model error) and the values of explanatory variables represented in the GIS (data error). Input probability distributions that represent both model and data error sources of uncertainty were simultaneously sampled using a Latin hypercube approach with logistic regression calculations of probability of elevated nonpoint source contaminants in ground water. The resulting probability distribution represents the prediction intervals and associated uncertainty of the ground water vulnerability predictions. The method is illustrated through a ground water vulnerability assessment of the High Plains regional aquifer. Results of the LHS simulations reveal significant prediction uncertainties that vary spatially across the regional aquifer. Additionally, the proposed method enables a spatial deconstruction of the prediction uncertainty that can lead to improved prediction of ground water vulnerability.     

Dairy of forthcoming events

Abstract

Due to its great importance, the availability of long flow records, contemporary as well as older, and the additional historical information of its behaviour, the Nile is an ideal test case for identifying and understanding hydrological behaviours, and for model development. Such behaviours include the long-term persistence, which historically has motivated the discovery of the Hurst phenomenon and has put into question classical statistical results and typical stochastic models. Based on the empirical evidence from the exploration of the Nile flows and on the theoretical insights provided by the principle of maximum entropy, a concept newly employed in hydrological stochastic modelling, an advanced yet simple stochastic methodology is developed. The approach is focused on the prediction of the Nile flow a month ahead, but the methodology is general and can be applied to any type of stochastic prediction. The stochastic methodology is also compared with deterministic approaches, specifically an analogue (local nonlinear chaotic) model and a connectionist (artificial neural network) model based on the same flow record. All models have good performance with the stochastic model outperforming in prediction skills and the analogue model in simplicity. In addition, the stochastic model has other elements of superiority such as the ability to provide long-term simulations and to improve understanding of natural behaviours.     

地震预测的灰色模糊模型

在灰色系统理论GM(1,1)模型的基础上,把模糊数学的表达方式用于灰色灾变预测中,建立了地震预测的灰色模糊模型。用此模型对我国川滇、甘宁青、新疆及华北等地区大地震时间序列进行预测检验,结果表明:灰色模糊预测模型比纯灰色系统理论模型要好,因而可在实际地震预测中试用。最后对所得结果及实际应用中需要注意的问题也进行了讨论。     

Improving monthly streamflow prediction in alpine regions: integrating HBV model with Bayesian neural network

Statistical methods have been widely used to build different streamflow prediction models; however, lacking of physical mechanism prevents precise streamflow prediction in alpine regions dominated by rainfall, snow and glacier. To improve precision, a new hybrid model (HBNN) integrating HBV hydrological model, Bayesian neural network (BNN) and uncertainty analysis is proposed. In this approach, the HBV is mainly used to generate initial snow-melt and glacier-melt runoffs that are regarded as new inputs of BNN for precision improvement. To examine model reliability, a hybrid deterministic model called HLSSVM incorporating the HBV model and least-square support vector machine is also developed and compared with HBNN in a typical region, the Yarkant River basin in Central Asia. The findings suggest that the HBNN model is a robust streamflow prediction model for alpine regions and capable of combining strengths of both the BNN statistical model and the HBV hydrological model, providing not only more precise streamflow prediction but also more reasonable uncertainty intervals than competitors particularly at high flows. It can be used in predicting streamflow for similar regions worldwide.     

Water level observations from unmanned aerial vehicles for improving estimates of surface water–groundwater interaction

Integrated hydrological models are usually calibrated against observations of river discharge and piezometric head in groundwater aquifers. Calibration of such models against spatially distributed observations of river water level can potentially improve their reliability and predictive skill. However, traditional river gauging stations are normally spaced too far apart to capture spatial patterns in the water surface, whereas spaceborne observations have limited spatial and temporal resolution. Unmanned aerial vehicles can retrieve river water level measurements, providing (a) high spatial resolution; (b) spatially continuous profiles along or across the water body, and (c) flexible timing of sampling. A semisynthetic study was conducted to analyse the value of the new unmanned aerial vehicle‐borne datatype for improving hydrological models, in particular estimates of groundwater–surface water (GW–SW) interaction. Mølleåen River (Denmark) and its catchment were simulated using an integrated hydrological model (MIKE 11–MIKE SHE). Calibration against distributed surface water levels using the Differential Evolution Adaptive Metropolis algorithm demonstrated a significant improvement in estimating spatial patterns and time series of GW–SW interaction. After water level calibration, the sharpness of the estimates of GW–SW time series improves by ~50% and root mean square error decreases by ~75% compared with those of a model calibrated against discharge only.     

Application of saddlepoint approximation in reliability analysis of dynamic systems

The application of the saddlepoint approximation to reliability analysis of dynamic systems is investigated. The failure event in reliability problems is formulated as the exceedance of a single performance variable over a prescribed threshold level. The saddlepoint approximation technique provides a choice to estimate the cumulative distribution function (CDF) of the performance variable. The failure probability is obtained as the value of the complement CDF at a specif ied threshold. The method requires computing the saddlepoint from a simple algebraic equation that depends on the cumulant generating function (CGF) of the performance variable. A method for calculating the saddlepoint using random samples of the performance variable is presented. The applicable region of the saddlepoint approximation is discussed in detail. A 10-story shear building model with white noise excitation illustrates the accuracy and effi ciency of the proposed methodology.     

A comparative study of seismic tomography models of the Chinese continental lithosphere

The Chinese mainland is subject to complicated plate interactions that give rise to its complex structure and tectonics. While several seismic velocity models have been developed for the Chinese mainland, apparent discrepancies exist and, so far, little effort has been made to evaluate their reliability and consistency. Such evaluations are important not only for the application and interpretation of model results but also for future model improvement. To address this problem, here we compare five published shear-wave velocity models with a focus on model consistency. The five models were derived from different datasets and methods (i.e., body waves, surface waves from earthquakes, surface waves from noise interferometry, and full waves) and interpolated into uniform horizontal grids (0.5° × 0.5°) with vertical sampling points at 5 km, 10 km, and then 20 km intervals to a depth of 160 km below the surface, from which we constructed an averaged model (AM) as a common reference for comparative study. We compare both the absolute velocity values and perturbation patterns of these models. Our comparisons show that the models have large (> 4%) differences in absolute values, and these differences are independent of data coverage and model resolution. The perturbation patterns of the models also show large differences, although some of the models show a high degree of consistency within certain depth ranges. The observed inconsistencies may reflect limited model resolution but, more importantly, systematic differences in the datasets and methods employed. Thus, despite several seismic models being published for this region, there is significant room for improvement. In particular, the inconsistencies in both data and methodologies need to be resolved in future research. Finally, we constructed a merged model (ChinaM-S1.0) that incorporates the more robust features of the five published models. As the existing models are constrained by different datasets and methods, the merged model serves as a new type of reference model that incorporates the common features from the joint datasets and methods for the shear-wave velocity structure of the Chinese mainland lithosphere.     

Identifying the potential loss of monitoring wells using an uncertainty analysis

From the mid-1940s through the 1980s, large volumes of waste water were discharged at the Hanford Site in southeastern Washington State, causing a large-scale rise (>20 m) in the water table. When waste water discharges ceased in 1988, ground water mounds began to dissipate. This caused a large number of wells to go dry and has made it difficult to monitor contaminant plume migration. To identify monitoring wells that will need replacement, a methodology has been developed using a first-order uncertainty analysis with UCODE, a nonlinear parameter estimation code. Using a three-dimensional, finite-element ground water flow code, key parameters were identified by calibrating to historical hydraulic head data. Results from the calibration period were then used to check model predictions by comparing monitoring wells' wet/dry status with field data. This status was analyzed using a methodology that incorporated the 0.3 cumulative probability derived from the confidence and prediction intervals. For comparison, a nonphysically based trend model was also used as a predictor of wells' wet/dry status. Although the numerical model outperformed the trend model, for both models, the central value of the intervals was a better predictor of a wet well status. The prediction interval, however, was more successful at identifying dry wells. Predictions made through the year 2048 indicated that 46% of the wells in the monitoring well network are likely to go dry in areas near the river and where the ground water mound is dissipating.