A Model-Averaging Method for Assessing Groundwater Conceptual Model Uncertainty

This study evaluates alternative groundwater models with different recharge and geologic components at the northern Yucca Flat area of the Death Valley Regional Flow System (DVRFS), USA. Recharge over the DVRFS has been estimated using five methods, and five geological interpretations are available at the northern Yucca Flat area. Combining the recharge and geological components together with additional modeling components that represent other hydrogeological conditions yields a total of 25 groundwater flow models. As all the models are plausible given available data and information, evaluating model uncertainty becomes inevitable. On the other hand, hydraulic parameters (e.g., hydraulic conductivity) are uncertain in each model, giving rise to parametric uncertainty. Propagation of the uncertainty in the models and model parameters through groundwater modeling causes predictive uncertainty in model predictions (e.g., hydraulic head and flow). Parametric uncertainty within each model is assessed using Monte Carlo simulation, and model uncertainty is evaluated using the model averaging method. Two model-averaging techniques (on the basis of information criteria and GLUE) are discussed. This study shows that contribution of model uncertainty to predictive uncertainty is significantly larger than that of parametric uncertainty. For the recharge and geological components, uncertainty in the geological interpretations has more significant effect on model predictions than uncertainty in the recharge estimates. In addition, weighted residuals vary more for the different geological models than for different recharge models. Most of the calibrated observations are not important for discriminating between the alternative models, because their weighted residuals vary only slightly from one model to another.     

Including prior information in the estimation of effective soil parameters in unsaturated zone modelling

In this paper we propose a methodology to include prior information in the estimation of effective soil parameters for modelling the soil moisture content in the unsaturated zone. Laboratory measurements on undisturbed soil cores were used to estimate the moisture retention curve and hydraulic conductivity curve parameters. The soil moisture content was measured at 25 locations along three transects and at three different depths (surface, 30 and 60 cm) on an 80×20 m hillslope for the year 2001. Soil cores were collected in 84 locations situated in three profile pits along the hillslope. For the estimation of the effective soil hydraulic parameters the joint probability distribution of measured parameter values was used as prior information. A two-horizon single column 1D MIKE SHE model based on Richards' equation was set-up for nine soil moisture measurement locations along the middle transect of the hillslope. The goal of the model is to simulate the soil moisture profile at each location. The shuffled complex evolution (SCE) algorithm has been applied to estimate effective model parameters using either wide parameter ranges, referred to as the ‘no-prior’ case, or the joint probability distribution of measured parameter values as prior information (‘prior’ case). When the prior information is incorporated in the SCE optimisation the goodness-of-fit of the model predictions is only slightly worse compared to when no-prior information is incorporated. However, the effective parameter estimates are more realistic when the prior information is incorporated. For both the no-prior and prior case the generalised likelihood uncertainty estimation procedure (GLUE) was subsequently used to estimate the uncertainty bounds (UB) on the model predictions. When incorporating the prior information more parameter sets were accepted for the estimation of the predictive uncertainty and the parameter values were more realistic. Moreover, UB better enclosed the observations. Thus, incorporating prior information in GLUE reduces the amount of model evaluations needed to obtain sufficient behavioural parameter sets. The results indicate the importance of prior information in the SCE and GLUE parameter estimation strategies.     

Ground water model calibration using pilot points and regularization

Use of nonlinear parameter estimation techniques is now commonplace in ground water model calibration. However, there is still ample room for further development of these techniques in order to enable them to extract more information from calibration datasets, to more thoroughly explore the uncertainty associated with model predictions, and to make them easier to implement in various modeling contexts. This paper describes the use of "pilot points" as a methodology for spatial hydraulic property characterization. When used in conjunction with nonlinear parameter estimation software that incorporates advanced regularization functionality (such as PEST), use of pilot points can add a great deal of flexibility to the calibration process at the same time as it makes this process easier to implement. Pilot points can be used either as a substitute for zones of piecewise parameter uniformity, or in conjunction with such zones. In either case, they allow the disposition of areas of high and low hydraulic property value to be inferred through the calibration process, without the need for the modeler to guess the geometry of such areas prior to estimating the parameters that pertain to them. Pilot points and regularization can also be used as an adjunct to geostatistically based stochastic parameterization methods. Using the techniques described herein, a series of hydraulic property fields can be generated, all of which recognize the stochastic characterization of an area at the same time that they satisfy the constraints imposed on hydraulic property values by the need to ensure that model outputs match field measurements. Model predictions can then be made using all of these fields as a mechanism for exploring predictive uncertainty.     

Demystifying Fisher Information: What Observation Data Reveal about Our Models

Information theory is the basis for understanding how information is transmitted as observations. Observation data can be used to compare uncertainty on parameter estimates and predictions between models. Jacobian Information (JI) is quantified as the determinant of the weighted Jacobian (sensitivity) matrix. Fisher Information (FI) is quantified as the determinant of the weighted FI matrix. FI measures the relative disorder of a model (entropy) in a set of models. One‐dimensional models are used to demonstrate the relationship between JI and FI, and the resulting uncertainty on estimated parameter values and model predictions for increasing model complexity, different model structures, different boundary conditions, and over‐fitted models. Greater model complexity results in increased JI accompanied by an increase in parameter and prediction uncertainty. FI generally increases with increasing model complexity unless model error is large. Models with lower FI have a higher level of disorder (increase in entropy) which results in greater uncertainty of parameter estimates and model predictions. A constant‐head boundary constrains the heads in the area near the boundary, reducing sensitivity of simulated equivalents to estimated parameters. JI and FI are lower for this boundary condition as compared to a constant‐outflow boundary in which the heads in the area of the boundary can adjust freely. Complex, over‐fitted models, in which the structure of the model is not supported by the observation dataset, result in lower JI and FI because there is insufficient information to estimate all parameters in the model.     

Impact of the numbers of observations and calibration parameters on equifinality,model performance,and output and parameter uncertainty

The level of model complexity that can be effectively supported by available information has long been a subject of many studies in hydrologic modelling. In particular, distributed parameter models tend to be regarded as overparameterized because of numerous parameters used to describe spatially heterogeneous hydrologic processes. However, it is not clear how parameters and observations influence the degree of overparameterization, equifinality of parameter values, and uncertainty. This study investigated the impact of the numbers of observations and parameters on calibration quality including equifinality among calibrated parameter values, model performance, and output/parameter uncertainty using the Soil and Water Assessment Tool model. In the experiments, the number of observations was increased by expanding the calibration period or by including measurements made at inner points of a watershed. Similarly, additional calibration parameters were included in the order of their sensitivity. Then, unique sets of parameters were calibrated with the same objective function, optimization algorithm, and stopping criteria but different numbers of observations. The calibration quality was quantified with statistics calculated based on the ‘behavioural’ parameter sets, identified using 1% and 5% cut‐off thresholds in a generalized likelihood uncertainty estimation framework. The study demonstrated that equifinality, model performance, and output/parameter uncertainty were responsive to the numbers of observations and calibration parameters; however, the relationship between the numbers, equifinality, and uncertainty was not always conclusive. Model performance improved with increased numbers of calibration parameters and observations, and substantial equifinality did neither necessarily mean bad model performance nor large uncertainty in the model outputs and parameters. Copyright © 2015 John Wiley & Sons, Ltd.     

Assessing the relative importance of parameter and forcing uncertainty and their interactions in conceptual hydrological model simulations

Predictions of river flow dynamics provide vital information for many aspects of water management including water resource planning, climate adaptation, and flood and drought assessments. Many of the subjective choices that modellers make including model and criteria selection can have a significant impact on the magnitude and distribution of the output uncertainty. Hydrological modellers are tasked with understanding and minimising the uncertainty surrounding streamflow predictions before communicating the overall uncertainty to decision makers. Parameter uncertainty in conceptual rainfall-runoff models has been widely investigated, and model structural uncertainty and forcing data have been receiving increasing attention. This study aimed to assess uncertainties in streamflow predictions due to forcing data and the identification of behavioural parameter sets in 31 Irish catchments. By combining stochastic rainfall ensembles and multiple parameter sets for three conceptual rainfall-runoff models, an analysis of variance model was used to decompose the total uncertainty in streamflow simulations into contributions from (i) forcing data, (ii) identification of model parameters and (iii) interactions between the two. The analysis illustrates that, for our subjective choices, hydrological model selection had a greater contribution to overall uncertainty, while performance criteria selection influenced the relative intra-annual uncertainties in streamflow predictions. Uncertainties in streamflow predictions due to the method of determining parameters were relatively lower for wetter catchments, and more evenly distributed throughout the year when the Nash-Sutcliffe Efficiency of logarithmic values of flow (lnNSE) was the evaluation criterion.     

Using internal catchment information to reduce the uncertainty of discharge and baseflow predictions

The semi-distributed hydrological model TOPMODEL was tested with data from the Can Vila research basin (Vallcebre) in order to verify its adequacy for simulating runoff and the relative contributions from saturated overland flow and groundwater flow. After a test of the overall performance of the model, only data from a wet period were selected for this work. The test was performed using the GLUE method. The model was conditioned on continuous discharge and water table records. Furthermore, point measurements of recession flow simultaneous with water table depth and the extent of saturated areas were used to condition the distributions of the more relevant parameters, using new or updated evaluation measures. A wide range of parameter sets provided acceptable results for flow simulation when the model was conditioned on flow data alone, and the uncertainty of prediction of the contribution from groundwater was extremely large. However, conditioning on water table records and the distribution of parameters obtained from point observations strongly reduced the uncertainty of predictions for both stream flow and groundwater contribution.     

Extending Data Worth Analyses to Select Multiple Observations Targeting Multiple Forecasts

Hydrological models are often set up to provide specific forecasts of interest. Owing to the inherent uncertainty in data used to derive model structure and used to constrain parameter variations, the model forecasts will be uncertain. Additional data collection is often performed to minimize this forecast uncertainty. Given our common financial restrictions, it is critical that we identify data with maximal information content with respect to forecast of interest. In practice, this often devolves to qualitative decisions based on expert opinion. However, there is no assurance that this will lead to optimal design, especially for complex hydrogeological problems. Specifically, these complexities include considerations of multiple forecasts, shared information among potential observations, information content of existing data, and the assumptions and simplifications underlying model construction. In the present study, we extend previous data worth analyses to include: simultaneous selection of multiple new measurements and consideration of multiple forecasts of interest. We show how the suggested approach can be used to optimize data collection. This can be used in a manner that suggests specific measurement sets or that produces probability maps indicating areas likely to be informative for specific forecasts. Moreover, we provide examples documenting that sequential measurement election approaches often lead to suboptimal designs and that estimates of data covariance should be included when selecting future measurement sets.     

Influence of uncertain boundary conditions and model structure on flood inundation predictions

In this study, the GLUE methodology is applied to establish the sensitivity of flood inundation predictions to uncertainty of the upstream boundary condition and bridges within the modelled region. An understanding of such uncertainties is essential to improve flood forecasting and floodplain mapping. The model has been evaluated on a large data set. This paper shows uncertainty of the upstream boundary can have significant impact on the model results, exceeding the importance of model parameter uncertainty in some areas. However, this depends on the hydraulic conditions in the reach e.g. internal boundary conditions and, for example, the amount of backwater within the modelled region. The type of bridge implementation can have local effects, which is strongly influenced by the bridge geometry (in this case the area of the culvert). However, the type of bridge will not merely influence the model performance within the region of the structure, but also other evaluation criteria such as the travel time. This also highlights the difficulties in establishing which parameters have to be more closely examined in order to achieve better fits. In this study no parameter set or model implementation that fulfils all evaluation criteria could be established. We propose four different approaches to this problem: closer investigation of anomalies; introduction of local parameters; increasing the size of acceptable error bounds; and resorting to local model evaluation. Moreover, we show that it can be advantageous to decouple the classification into behavioural and non-behavioural model data/parameter sets from the calculation of uncertainty bounds.     

Bayesian updating of flood inundation likelihoods conditioned on flood extent data

Previously we have detailed an application of the generalized likelihood uncertainty estimation (GLUE) procedure to estimate spatially distributed uncertainty in models conditioned against binary pattern data contained in flood inundation maps. This method was applied to two sites where a single consistent synoptic image of inundation extent was available to test the simulation performance of the method. In this paper, we extend this to examine the predictive performance of the method for a reach of the River Severn, west‐central England. Uniquely for this reach, consistent inundation images of two major floods have been acquired from spaceborne synthetic aperture radars, as well as a high‐resolution digital elevation model derived using laser altimetry. These data thus allow rigorous split sample testing of the previous GLUE application. To achieve this, Monte Carlo analyses of parameter uncertainty within the GLUE framework are conducted for a typical hydraulic model applied to each flood event. The best 10% of parameter sets identified in each analysis are then used to map uncertainty in flood extent predictions using the method previously proposed for both an independent validation data set and a design flood. Finally, methods for combining the likelihood information derived from each Monte Carlo ensemble are examined to determine whether this has the potential to reduce uncertainty in spatially distributed measures of flood risk for a design flood. The results show that for this reach and these events, the method previously established is able to produce sharply defined flood risk maps that compare well with observed inundation extent. More generally, we show that even single, poor‐quality inundation extent images are useful in constraining hydraulic model calibrations and that values of effective friction parameters are broadly stationary between the two events simulated, most probably reflecting their similar hydraulics. Copyright © 2004 John Wiley & Sons, Ltd.     

Groundwater recharge and evapotranspiration for two natural ecosystems covered with oak and heather

The evapotranspiration and groundwater recharge from two natural areas with high (oak) and low (heath) vegetation were estimated by calibrating a semi-physical numerical soil water and heat model to fit 8 and 7 years of TDR-measurements of water content, respectively. The measurements were made between the surface and 7 m depth. For the oak stand, the estimated annual recharge for the years 1992–1999 is 390 mm, the evaporation from soil and interception is 205 mm, and the transpiration is 285 mm. For the heath area estimation was carried out for the years 1993–1999. However, the heath was struck by a heavy beetle attack in 1994, which strongly affected the vegetation and thus the water balance for the following 3 years. For years not affected, the estimated recharge is 733 mm (about 50% larger than for the oak stand for the same years), the evaporation is 316 mm, and the transpiration is 128 mm. The estimated recharge values compare fairly well to estimates obtained from bromide tracer experiments. However, the recharge estimates obtained from the tracer experiments are very uncertain. The uncertainty is mainly due to spatial heterogeneity making the three replicate samples taken here for each time and depth insufficient.

The analyses of TDR-measurements and tracer data showed that water front movement depends on the antecedent soil water content. Some layers are bypassed, especially at low water contents, and at high soil water contents preferential flow was observed at the heath site.     

Assessing the uncertainty in distributed model predictions using observed binary pattern information within GLUE

In this paper we extend the generalized likelihood uncertainty estimation (GLUE) technique to estimate spatially distributed uncertainty in models conditioned against binary pattern data contained in flood inundation maps. Untransformed binary pattern data already have been used within GLUE to estimate domain‐averaged (zero‐dimensional) likelihoods, yet the pattern information embedded within such sources has not been used to estimate distributed uncertainty. Where pattern information has been used to map distributed uncertainty it has been transformed into a continuous function prior to use, which may introduce additional errors. To solve this problem we use here ‘raw’ binary pattern data to define a zero‐dimensional global performance measure for each simulation in a Monte Carlo ensemble. Thereafter, for each pixel of the distributed model we evaluate the probability that this pixel was inundated. This probability is then weighted by the measure of global model performance, thus taking into account how well a given parameter set performs overall. The result is a distributed uncertainty measure mapped over real space. The advantage of the approach is that it both captures distributed uncertainty and contains information on global likelihood that can be used to condition predictions of further events for which observed data are not available. The technique is applied to the problem of flood inundation prediction at two test sites representing different hydrodynamic conditions. In both cases, the method reveals the spatial structure in simulation uncertainty and simultaneously enables mapping of flood probability predicted by the model. Spatially distributed uncertainty analysis is shown to contain information over and above that available from global performance measures. Overall, the paper highlights the different types of information that may be obtained from mappings of model uncertainty over real and n‐dimensional parameter spaces. Copyright © 2002 John Wiley & Sons, Ltd.     

An investigation of numerical grid effects in parameter estimation

Modern ground water characterization and remediation projects routinely require calibration and inverse analysis of large three-dimensional numerical models of complex hydrogeological systems. Hydrogeologic complexity can be prompted by various aquifer characteristics including complicated spatial hydrostratigraphy and aquifer recharge from infiltration through an unsaturated zone. To keep the numerical models computationally efficient, compromises are frequently made in the model development, particularly, about resolution of the computational grid and numerical representation of the governing flow equation. The compromise is required so that the model can be used in calibration, parameter estimation, performance assessment, and analysis of sensitivity and uncertainty in model predictions. However, grid properties and resolution as well as applied computational schemes can have large effects on forward-model predictions and on inverse parameter estimates. We investigate these effects for a series of one- and two-dimensional synthetic cases representing saturated and variably saturated flow problems. We show that "conformable" grids, despite neglecting terms in the numerical formulation, can lead to accurate solutions of problems with complex hydrostratigraphy. Our analysis also demonstrates that, despite slower computer run times and higher memory requirements for a given problem size, the control volume finite-element method showed an advantage over finite-difference techniques in accuracy of parameter estimation for a given grid resolution for most of the test problems.     

Practical Postcalibration Uncertainty Analysis: Yucca Mountain, Nevada

The values of parameters in a groundwater flow model govern the precision of predictions of future system behavior. Predictive precision, thus, typically depends on an ability to infer values of system properties from historical measurements through calibration. When such data are scarce, or when their information content with respect to parameters that are most relevant to predictions of interest is weak, predictive uncertainty may be high, even if the model is "calibrated." Recent advances help recognize this condition, quantitatively evaluate predictive uncertainty, and suggest a path toward improved predictive accuracy by identifying sources of predictive uncertainty and by determining what observations will most effectively reduce this uncertainty. We demonstrate linear and nonlinear predictive error/uncertainty analyses as applied to a groundwater flow model of Yucca Mountain, Nevada, the United States' proposed site for disposal of high-level radioactive waste. Linear and nonlinear uncertainty analyses are readily implemented as an adjunct to model calibration with medium to high parameterization density. Linear analysis yields contributions made by each parameter to a prediction's uncertainty and the worth of different observations, both existing and yet-to-be-gathered, toward reducing this uncertainty. Nonlinear analysis provides more accurate characterization of the uncertainty of model predictions while yielding their (approximate) probability distribution functions. This article applies the above methods to a prediction of specific discharge and confirms the uncertainty bounds on specific discharge supplied in the Yucca Mountain Project License Application.     

Prediction uncertainty of conceptual rainfall-runoff models caused by problems in identifying model parameters and structure

Abstract

The uncertainties arising from the problem of identifying a representative model structure and model parameters in a conceptual rainfall-runoff model were investigated. A conceptual model, the HBV model, was applied to the mountainous Brugga basin (39.9 km”) in the Black Forest, southwestern Germany. In a first step, a Monte Carlo procedure with randomly generated parameter sets was used for calibration. For a ten-year calibration period, different parameter sets resulted in an equally good correspondence between observed and simulated runoff. A few parameters were well defined (i.e. best parameter values were within small ranges), but for most parameters good simulations were found with values varying over wide ranges. In a second step, model variants with different numbers of elevation and landuse zones and various runoff generation conceptualizations were tested. In some cases, representation of more spatial variability gave better simulations in terms of discharge. However, good results could be obtained with different and even unrealistic concepts. The computation of design floods and low flow predictions illustrated that the parameter uncertainty and the uncertainty of identifying a unique best model variant have implications for model predictions. The flow predictions varied considerably. The peak discharge of a flood with a probability of 0.01 year^?1, for instance, varied from 40 to almost 60 mm day^?1. It was concluded that model predictions, particularly in applied studies, should be given as ranges rather than as single values.     

The assessment of high recharge areas using DO indicators and recharge potential analysis: a case study of Taiwan’s Pingtung plain

Identifying high groundwater recharge areas is important for the conservation of groundwater quality and quantity. A common practice used by previous studies is to estimate groundwater recharge potential (GRP) using recharge potential analysis (RPA) under different environments. These studies use the estimated GRP to identify the high potential groundwater recharge sites. However, the RPA parameters are subjectively defined for these previous studies. To remove the supposition, this study proposes a systematic approach that defines the RPA parameter values based on the theory of parameter identification. This study uses dissolved oxygen (DO) indicators to calibrate the RPA parameters. This calibration improves the correlation coefficient between the DO indicators and computed GRP values from 0.63 to 0.87. By comparing the initial values, these results indicate that the estimated RPA parameters better represent the field infiltration characteristic. This result also indicates that defining the RPA parameter values based on DO indicators is necessary and important for accuracy. These calibrated parameters are used to estimate the GRP distribution of Taiwan’s Pingtung Plain. The GRP values are delineated into five levels. High and excellent GRP areas are defined as high recharge areas, which compose about 26.74 % of the study area. Based on the proposed method, the estimated GRP distribution can accurately represent the study area’s field recharge characteristics. These study results can be a good reference for groundwater recharge analyses, specifically if well data is limited or difficult to obtain.     

Assessing the hydrological and geomorphic behaviour of a landscape evolution model within a limits-of-acceptability uncertainty analysis framework

Landscape evolution models (LEMs) have the capability to characterize key aspects of geomorphological and hydrological processes. However, their usefulness is hindered by model equifinality and paucity of available calibration data. Estimating uncertainty in the parameter space and resultant model predictions is rarely achieved as this is computationally intensive and the uncertainties inherent in the observed data are large. Therefore, a limits-of-acceptability (LoA) uncertainty analysis approach was adopted in this study to assess the value of uncertain hydrological and geomorphic data. These were used to constrain simulations of catchment responses and to explore the parameter uncertainty in model predictions. We applied this approach to the River Derwent and Cocker catchments in the UK using a LEM CAESAR-Lisflood. Results show that the model was generally able to produce behavioural simulations within the uncertainty limits of the streamflow. Reliability metrics ranged from 24.4% to 41.2% and captured the high-magnitude low-frequency sediment events. Since different sets of behavioural simulations were found across different parts of the catchment, evaluating LEM performance, in quantifying and assessing both at-a-point behaviour and spatial catchment response, remains a challenge. Our results show that evaluating LEMs within uncertainty analyses framework while taking into account the varying quality of different observations constrains behavioural simulations and parameter distributions and is a step towards a full-ensemble uncertainty evaluation of such models. We believe that this approach will have benefits for reflecting uncertainties in flooding events where channel morphological changes are occurring and various diverse (and yet often sparse) data have been collected over such events.     

Uncertainty assessment of a process-based integrated catchment model of phosphorus

Despite the many models developed for phosphorus concentration prediction at differing spatial and temporal scales, there has been little effort to quantify uncertainty in their predictions. Model prediction uncertainty quantification is desirable, for informed decision-making in river-systems management. An uncertainty analysis of the process-based model, integrated catchment model of phosphorus (INCA-P), within the generalised likelihood uncertainty estimation (GLUE) framework is presented. The framework is applied to the Lugg catchment (1,077 km²), a River Wye tributary, on the England–Wales border. Daily discharge and monthly phosphorus (total reactive and total), for a limited number of reaches, are used to initially assess uncertainty and sensitivity of 44 model parameters, identified as being most important for discharge and phosphorus predictions. This study demonstrates that parameter homogeneity assumptions (spatial heterogeneity is treated as land use type fractional areas) can achieve higher model fits, than a previous expertly calibrated parameter set. The model is capable of reproducing the hydrology, but a threshold Nash-Sutcliffe co-efficient of determination (E or R ²) of 0.3 is not achieved when simulating observed total phosphorus (TP) data in the upland reaches or total reactive phosphorus (TRP) in any reach. Despite this, the model reproduces the general dynamics of TP and TRP, in point source dominated lower reaches. This paper discusses why this application of INCA-P fails to find any parameter sets, which simultaneously describe all observed data acceptably. The discussion focuses on uncertainty of readily available input data, and whether such process-based models should be used when there isn’t sufficient data to support the many parameters.