Short-term optimal operation of water systems using ensemble forecasts

Short-term water system operation can be realized using Model Predictive Control (MPC). MPC is a method for operational management of complex dynamic systems. Applied to open water systems, MPC provides integrated, optimal, and proactive management, when forecasts are available. Notwithstanding these properties, if forecast uncertainty is not properly taken into account, the system performance can critically deteriorate.Ensemble forecast is a way to represent short-term forecast uncertainty. An ensemble forecast is a set of possible future trajectories of a meteorological or hydrological system. The growing ensemble forecasts’ availability and accuracy raises the question on how to use them for operational management.The theoretical innovation presented here is the use of ensemble forecasts for optimal operation. Specifically, we introduce a tree based approach. We called the new method Tree-Based Model Predictive Control (TB-MPC). In TB-MPC, a tree is used to set up a Multistage Stochastic Programming, which finds a different optimal strategy for each branch and enhances the adaptivity to forecast uncertainty. Adaptivity reduces the sensitivity to wrong forecasts and improves the operational performance.TB-MPC is applied to the operational management of Salto Grande reservoir, located at the border between Argentina and Uruguay, and compared to other methods.     

Use of daily precipitation uncertainties in streamflow simulation and forecast

Among other sources of uncertainties in hydrologic modeling, input uncertainty due to a sparse station network was tested. The authors tested impact of uncertainty in daily precipitation on streamflow forecasts. In order to test the impact, a distributed hydrologic model (PRMS, Precipitation Runoff Modeling System) was used in two hydrologically different basins (Animas basin at Durango, Colorado and Alapaha basin at Statenville, Georgia) to generate ensemble streamflows. The uncertainty in model inputs was characterized using ensembles of daily precipitation, which were designed to preserve spatial and temporal correlations in the precipitation observations. Generated ensemble flows in the two test basins clearly showed fundamental differences in the impact of input uncertainty. The flow ensemble showed wider range in Alapaha basin than the Animas basin. The wider range of streamflow ensembles in Alapaha basin was caused by both greater spatial variance in precipitation and shorter time lags between rainfall and runoff in this rainfall dominated basin. This ensemble streamflow generation framework was also applied to demonstrate example forecasts that could improve traditional ESP (Ensemble Streamflow Prediction) method.     

Hydrological Outlook UK: an operational streamflow and groundwater level forecasting system at monthly to seasonal time scales

This paper describes the development of the first operational seasonal hydrological forecasting service for the UK, the Hydrological Outlook UK (HOUK). Since June 2013, this service has delivered monthly forecasts of streamflow and groundwater levels, with an emphasis on forecasting hydrological conditions over the next three months, accompanied by outlooks over longer time horizons. This system is based on three complementary approaches combined to produce the outlooks: (i) national-scale modelling of streamflow and groundwater levels based on dynamic seasonal rainfall forecasts, (ii) catchment-scale modelling where streamflow and groundwater level models are driven by historical meteorological forcings (i.e. the Ensemble Streamflow Prediction, ESP, approach), and (iii) a catchment-scale statistical method based on persistence and historical analogues. This paper provides the background to the Hydrological Outlook, describes the various component methods in detail and then considers the impact and usefulness of the product. As an example of a multi-method, operational seasonal hydrological forecasting system, it is hoped that this overview provides useful information and context for other forecasting initiatives around the world.     

Post‐processing of medium‐range probabilistic hydrological forecasting: impact of forcing,initial conditions and model errors

The impact of errors in the forcing, errors in the model structure and parameters, and errors in the initial conditions is investigated in a simple hydrological ensemble prediction system. The hydrological model is based on an input nonlinearity connected with a linear transfer function and forced by precipitation forecasts from the European Centre for Medium‐Range Weather Forecast (ECMWF) Ensemble Prediction System (EPS). The post‐processing of the precipitation and/or the streamflow using information from the reforecasts performed by ECMWF is tested. For this purpose, hydrological reforecasts are obtained by forcing the hydrological model with the precipitation from the reforecast data. In the present case study, it is found that the post‐processing of the hydrological ensembles with a statistical model fitted on the hydrological reforecasts improves the verification scores better than the use of post‐processed precipitation ensembles. In the case of large biases in the precipitation, combining the post‐processing of both precipitation and streamflow allows for further improvements. During the winter, errors in the initial conditions have a larger impact on the scores than errors in the model structure as designed in the experiments. Errors in the parameter values are largely corrected with the post‐processing. Copyright © 2014 John Wiley & Sons, Ltd.     

Towards improved post‐processing of hydrologic forecast ensembles

Forecast ensembles of hydrological and hydrometeorologial variables are prone to various uncertainties arising from climatology, model structure and parameters, and initial conditions at the forecast date. Post‐processing methods are usually applied to adjust the mean and variance of the ensemble without any knowledge about the uncertainty sources. This study initially addresses the drawbacks of a commonly used statistical technique, quantile mapping (QM), in bias correction of hydrologic forecasts. Then, an auxiliary variable, the failure index (γ), is proposed to estimate the ineffectiveness of the post‐processing method based on the agreement of adjusted forecasts with corresponding observations during an analysis period prior to the forecast date. An alternative post‐processor based on copula functions is then introduced such that marginal distributions of observations and model simulations are combined to create a multivariate joint distribution. A set of 2500 hypothetical forecast ensembles with parametric marginal distributions of simulated and observed variables are post‐processed with both QM and the proposed multivariate post‐processor. Deterministic forecast skills show that the proposed copula‐based post‐processing is more effective than the QM method in improving the forecasts. It is found that the performance of QM is highly correlated with the failure index, unlike the multivariate post‐processor. In probabilistic metrics, the proposed multivariate post‐processor generally outperforms QM. Further evaluation of techniques is conducted for river flow forecast of Sprague River basin in southern Oregon. Results show that the multivariate post‐processor performs better than the QM technique; it reduces the ensemble spread and is a more reliable approach for improving the forecast. Copyright © 2012 John Wiley & Sons, Ltd.     

Post‐processing of ensemble forecasts in low‐flow period

For water supply, navigational, ecological protection or water quality control purposes, there is a great need in knowing the likelihood of the river level falling below a certain threshold. Ensemble streamflow prediction (ESP) based on simulations of deterministic hydrologic models is widely used to assess this likelihood. Raw ESP results can be biased in both the ensemble means and the spreads. In this study, we applied a modified general linear model post‐processor (GLMPP) to correct these biases. The modified GLMPP is built on the basis of regression of simulated and observed streamflow calculated on the basis of canonical events, instead of the daily values as is carried out in the original GLMPP. We conducted the probabilistic analysis of post‐processed ESP results falling below pre‐specified low‐flow levels at seasonal time scale. Raw ESP forecasts from the 1980 to 2006 periods by four different land surface models (LSMs) in eight large river basins in the continental USA are included in the analysis. The four LSMs are Noah, Mosaic, variable infiltration capacity and Sacramento models. The major results from this study are as follows: (1) a modified GLMPP was proposed on the basis of canonical events; (2) post‐processing can improve the accuracy and reduce the uncertainty of hydrologic forecasts; (3) post‐processing can help deal with the effect of human activity; and (4) raw simulation results from different models vary greatly in different basins. However, post‐processing can always remove model biases under different conditions. Copyright © 2014 John Wiley & Sons, Ltd.     

Inverse flood risk modelling under changing climatic conditions

One of the most significant anticipated consequences of global climate change is the increased frequency of hydrologic extremes. Predictions of climate change impacts on the regime of hydrologic extremes have traditionally been conducted using a top‐down approach. The top‐down approach involves a high degree of uncertainty associated with global circulation model (GCM) outputs and the choice of downscaling technique. This study attempts to explore an inverse approach to the modelling of hydrologic risk and vulnerability to changing climatic conditions. With a focus targeted at end‐users, the proposed approach first identifies critical hydrologic exposures that may lead to local failures of existing water resources systems. A hydrologic model is used to transform inversely the main hydrologic exposures, such as floods and droughts, into corresponding meteorological conditions. The frequency of critical meteorological situations is investigated under present and future climatic scenarios by means of a generic weather generator. The weather generator, linked with GCMs at the last step of the proposed methodology, allows the creation of an ensemble of different scenarios, as well as an easy updating, when new and improved GCM outputs become available. The technique has been applied in Ontario, Canada. The results show significant changes in the frequency of hydro‐climatic extremes under future climate scenarios in the study area. Copyright © 2007 John Wiley & Sons, Ltd.     

The value of updating ensemble streamflow prediction in reservoir operations

This study proposes a new monthly ensemble streamflow prediction (ESP) forecasting system that can update the ESP in the middle of a month to reflect the meteorological and hydrological variations during that month. The reservoir operating policies derived from a sampling stochastic dynamic programming model using ESP scenarios updated three times a month were applied to the Geum River basin to measure the value of updated ESP for 21 years with 100 initial storage combinations. The results clearly demonstrate that updating the ESP scenario improves the accuracy of the forecasts and consequently their operational benefit. This study also proves that the accuracy of the ESP scenario, particularly when high flows occur, has a considerable effect on the reservoir operations. Copyright © 2010 John Wiley & Sons, Ltd.     

Evaluation of monthly precipitation forecasting skill of the National Multi‐model Ensemble in the summer season

As a test bed, the National Multi‐model Ensemble (NMME) comprises seven climate models from different sources, including the National Oceanic and Atmospheric Administration, the National Aeronautics and Space Administration, the National Center for Atmospheric Research and the International Research Institute for Climate and Society. It provides 89 ensemble members of precipitation forecasts at different lead times. Precipitation forecasting from climate models has been applied to provide streamflow forecasts, and its utility in water resource system operation has been demonstrated in the literature. In this study, 1‐month‐ahead precipitation forecasts from NMME are evaluated for 945 grid points of 1°‐by‐1° resolution over the continental USA using mean square error and rank probability score. The temporal and spatial variabilities of the forecasting skill over different months of the summer season are discussed. The relation between forecasting uncertainty and observed precipitation is investigated. Such analyses have implications for monthly operational forecasts and water resource management at the watershed scale. Copyright © 2013 John Wiley & Sons, Ltd.     

PAI-OFF: A new proposal for online flood forecasting in flash flood prone catchments

The Process Modelling and Artificial Intelligence for Online Flood Forecasting (PAI-OFF) methodology combines the reliability of physically based, hydrologic/hydraulic modelling with the operational advantages of artificial intelligence. These operational advantages are extremely low computation times and straightforward operation. The basic principle of the methodology is to portray process models by means of ANN. We propose to train ANN flood forecasting models with synthetic data that reflects the possible range of storm events. To this end, establishing PAI-OFF requires first setting up a physically based hydrologic model of the considered catchment and – optionally, if backwater effects have a significant impact on the flow regime – a hydrodynamic flood routing model of the river reach in question. Both models are subsequently used for simulating all meaningful and flood relevant storm scenarios which are obtained from a catchment specific meteorological data analysis. This provides a database of corresponding input/output vectors which is then completed by generally available hydrological and meteorological data for characterizing the catchment state prior to each storm event. This database subsequently serves for training both a polynomial neural network (PoNN) – portraying the rainfall–runoff process – and a multilayer neural network (MLFN), which mirrors the hydrodynamic flood wave propagation in the river. These two ANN models replace the hydrological and hydrodynamic model in the operational mode. After presenting the theory, we apply PAI-OFF – essentially consisting of the coupled “hydrologic” PoNN and “hydrodynamic” MLFN – to the Freiberger Mulde catchment in the Erzgebirge (Ore-mountains) in East Germany (3000 km²). Both the demonstrated computational efficiency and the prediction reliability underline the potential of the new PAI-OFF methodology for online flood forecasting.     

Application of data assimilation with the Root Zone Water Quality Model for soil moisture profile estimation in the upper Cedar Creek,Indiana

Data assimilation techniques have been proven as an effective tool to improve model forecasts by combining information about observed variables in many areas. This article examines the potential of assimilating surface soil moisture observations into a field‐scale hydrological model, the Root Zone Water Quality Model, to improve soil moisture estimation. The Ensemble Kalman Filter (EnKF), a popular data assimilation technique for nonlinear systems, was applied and compared with a simple direct insertion method. In situ soil moisture data at four different depths (5, 20, 40, and 60 cm) from two agricultural fields (AS1 and AS2) in northeastern Indiana were used for assimilation and validation purposes. Through daily update, the EnKF improved soil moisture estimation compared with the direct insertion method and model results without assimilation, having more distinct improvement at the 5 and 20 cm depths than for deeper layers (40 and 60 cm). Local vertical soil property heterogeneity in AS1 deteriorated soil moisture estimates with the EnKF. Removal of systematic bias in the forecast model was found to be critical for more successful soil moisture data assimilation studies. This study also demonstrates that a more frequent update generally contributes in enhancing the open loop simulation; however, large forecasting error can prevent more frequent update from providing better results. In addition, results indicate that various ensemble sizes make little difference in the assimilation results. An ensemble of 100 members produced results that were comparable with results obtained from larger ensembles. Copyright © 2011 John Wiley & Sons, Ltd.     

Forecasting flash floods using data-based mechanistic models and NORA radar rainfall forecasts

Abstract

Data-based mechanistic (DBM) models can offer a parsimonious representation of catchment dynamics. They have been shown to provide reliable accurate flood forecasts in many hydrological situations. In this work, the DBM methodology is applied to forecast flash floods in a small Alpine catchment. Compared to previous DBM modelling studies, the catchment response is rapid. The use of novel radar-derived ensemble quantitative precipitation forecasts based on analogues to drive the DBM model allows the forecast horizon to be increased to a level useful for emergency response. The characterization of the predictive uncertainty in the resulting hydrological forecasts is discussed and a framework for its representation illustrated.

Editor Z.W. Kundzewicz; Guest editor R.J. Moore     

Ensemble prediction of regional droughts using climate inputs and the SVM–copula approach

In this study, the climate teleconnections with meteorological droughts are analysed and used to develop ensemble drought prediction models using a support vector machine (SVM)–copula approach over Western Rajasthan (India). The meteorological droughts are identified using the Standardized Precipitation Index (SPI). In the analysis of large‐scale climate forcing represented by climate indices such as El Niño Southern Oscillation, Indian Ocean Dipole Mode and Atlantic Multidecadal Oscillation on regional droughts, it is found that regional droughts exhibits interannual as well as interdecadal variability. On the basis of potential teleconnections between regional droughts and climate indices, SPI‐based drought forecasting models are developed with up to 3 months' lead time. As traditional statistical forecast models are unable to capture nonlinearity and nonstationarity associated with drought forecasts, a machine learning technique, namely, support vector regression (SVR), is adopted to forecast the drought index, and the copula method is used to model the joint distribution of observed and predicted drought index. The copula‐based conditional distribution of an observed drought index conditioned on predicted drought index is utilized to simulate ensembles of drought forecasts. Two variants of drought forecast models are developed, namely a single model for all the periods in a year and separate models for each of the four seasons in a year. The performance of developed models is validated for predicting drought time series for 10 years' data. Improvement in ensemble prediction of drought indices is observed for combined seasonal model over the single model without seasonal partitions. The results show that the proposed SVM–copula approach improves the drought prediction capability and provides estimation of uncertainty associated with drought predictions. Copyright © 2013 John Wiley & Sons, Ltd.     

Dual state–parameter estimation of hydrological models using ensemble Kalman filter

Hydrologic models are twofold: models for understanding physical processes and models for prediction. This study addresses the latter, which modelers use to predict, for example, streamflow at some future time given knowledge of the current state of the system and model parameters. In this respect, good estimates of the parameters and state variables are needed to enable the model to generate accurate forecasts. In this paper, a dual state–parameter estimation approach is presented based on the Ensemble Kalman Filter (EnKF) for sequential estimation of both parameters and state variables of a hydrologic model. A systematic approach for identification of the perturbation factors used for ensemble generation and for selection of ensemble size is discussed. The dual EnKF methodology introduces a number of novel features: (1) both model states and parameters can be estimated simultaneously; (2) the algorithm is recursive and therefore does not require storage of all past information, as is the case in the batch calibration procedures; and (3) the various sources of uncertainties can be properly addressed, including input, output, and parameter uncertainties. The applicability and usefulness of the dual EnKF approach for ensemble streamflow forecasting is demonstrated using a conceptual rainfall-runoff model.     

Exploratory analysis of statistical post‐processing methods for hydrological ensemble forecasts

Despite many recent improvements, ensemble forecast systems for streamflow often produce under‐dispersed predictive distributions. This situation is problematic for their operational use in water resources management. Many options exist for post‐processing of raw forecasts. However, most of these have been developed using meteorological variables such as temperature, which displays characteristics very different from streamflow. In addition, streamflow data series are often very short or contain numerous gaps, thus compromising the estimation of post‐processing statistical parameters. For operational use, a post‐processing method has to be effective while remaining as simple as possible. We compared existing post‐processing methods using normally distributed and gamma‐distributed synthetic datasets. To reflect situations encountered with ensemble forecasts of daily streamflow, four normal distribution parameterizations and six gamma distribution parameterizations were used. Three kernel‐based approaches were tested, namely, the ‘best member’ method and two improvements thereof, and one regression‐based approach. Additional tests were performed to assess the ability of post‐processing methods to cope with short calibration series, missing values or small numbers of ensemble members. We thus found that over‐dispersion is best corrected by the regression method, while under‐dispersion is best corrected by kernel‐based methods. This work also shows key limitations associated with short data series, missing values, asymmetry and bias. One of the improved best member methods required longer series for the estimation of post‐processing parameters, but if provided with adequate information, yielded the best improvement of the continuous ranked probability score. These results suggest guidelines for future studies involving real operational datasets. Copyright © 2014 John Wiley & Sons, Ltd.     

Evaluation of dynamically downscaled reanalysis precipitation data for hydrological application

Skilful and reliable precipitation data are essential for seasonal hydrologic forecasting and generation of hydrological data. Although output from dynamic downscaling methods is used for hydrological application, the existence of systematic errors in dynamically downscaled data adversely affects the skill of hydrologic forecasting. This study evaluates the precipitation data derived by dynamically downscaling the global atmospheric reanalysis data by propagating them through three hydrological models. Hydrological models are calibrated for 28 watersheds located across the southeastern United States that is minimally affected by human intervention. Calibrated hydrological models are forced with five different types of datasets: global atmospheric reanalysis (National Centers for Environmental Prediction/Department of Energy Global Reanalysis and European Centre for Medium‐Range Weather Forecasts 40‐year Reanalysis) at their native resolution; dynamically downscaled global atmospheric reanalysis at 10‐km grid resolution; stochastically generated data from weather generator; bias‐corrected dynamically downscaled; and bias‐corrected global reanalysis. The reanalysis products are considered as surrogates for large‐scale observations. Our study indicates that over the 28 watersheds in the southeastern United States, the simulated hydrological response to the bias‐corrected dynamically downscaled data is superior to the other four meteorological datasets. In comparison with synthetically generated meteorological forcing (from weather generator), the dynamically downscaled data from global atmospheric reanalysis result in more realistic hydrological simulations. Therefore, we conclude that dynamical downscaling of global reanalysis, which offers data for sufficient number of years (in this case 22 years), although resource intensive, is relatively more useful than other sources of meteorological data with comparable period in simulating realistic hydrological response at watershed scales. Copyright © 2013 John Wiley & Sons, Ltd.     

Forecast of Low Visibility and Fog from NCEP: Current Status and Efforts

Based on the visibility analysis data during November 2009 through April 2010 over North America from the Aviation Digital Database Service (ADDS), the performance of low visibility/fog predictions from the current operational 12?km-NAM, 13?km-RUC and 32?km-WRF-NMM models at the National Centers for Environmental Prediction (NCEP) was evaluated. The evaluation shows that the performance of the low visibility/fog forecasts from these models is still poor in comparison to those of precipitation forecasts from the same models. In order to improve the skill of the low visibility/fog prediction, three efforts have been made at NCEP, including application of a rule-based fog detection scheme, extension of the NCEP Short Range Ensemble Forecast System (SREF) to fog ensemble probabilistic forecasts, and a combination of these two applications. How to apply these techniques in fog prediction is described and evaluated with the same visibility analysis data over the same period of time. The evaluation results demonstrate that using the multi-rule-based fog detection scheme significantly improves the fog forecast skill for all three models relative to visibility-diagnosed fog prediction, and with a combination of both rule-based fog detection and the ensemble technique, the performance skill of fog forecasting can be further raised.     

Probabilistic radar rainfall nowcasts using empirical and theoretical uncertainty models

Weather radar has a potential to provide accurate short‐term (0–3 h) forecasts of rainfall (i.e. radar nowcasts), which are of great importance in warnings and risk management for hydro‐meteorological events. However, radar nowcasts are affected by large uncertainties, which are not only linked to limitations in the forecast method but also because of errors in the radar rainfall measurement. The probabilistic quantitative precipitation nowcasting approach attempts to quantify these uncertainties by delivering the forecasts in a probabilistic form. This study implements two forms of probabilistic quantitative precipitation nowcasting for a hilly area in the south of Manchester, namely, the theoretically based scheme [ensemble rainfall forecasts (ERF)‐TN] and the empirically based scheme (ERF‐EM), and explores which one exhibits higher predictive skill. The ERF‐TN scheme generates ensemble forecasts of rainfall in which each ensemble member is determined by the stochastic realisation of a theoretical noise component. The so‐called ERF‐EM scheme proposed and applied for the first time in this study, aims to use an empirically based error model to measure and quantify the combined effect of all the error sources in the radar rainfall forecasts. The essence of the error model is formulated into an empirical relation between the radar rainfall forecasts and the corresponding ‘ground truth’ represented by the rainfall field from rain gauges measurements. The ensemble members generated by the two schemes have been compared with the rain gauge rainfall. The hit rate and the false alarm rate statistics have been computed and combined into relative operating characteristic curves. The comparison of the performance scores for the two schemes shows that the ERF‐EM achieves better performance than the ERF‐TN at 1‐h lead time. The predictive skills of both schemes are almost identical when the lead time increases to 2 h. In addition, the relation between uncertainty in the radar rainfall forecasts and lead time is also investigated by computing the dispersion of the generated ensemble members. Copyright © 2013 John Wiley & Sons, Ltd.     

Hydrological data assimilation with the Ensemble Square-Root-Filter: Use of streamflow observations to update model states for real-time flash flood forecasting

The objective of the study is to evaluate the potential of a data assimilation system for real-time flash flood forecasting over small watersheds by updating model states. To this end, the Ensemble Square-Root-Filter (EnSRF) based on the Ensemble Kalman Filter (EnKF) technique was coupled to a widely used conceptual rainfall-runoff model called HyMOD. Two small watersheds susceptible to flash flooding from America and China were selected in this study. The modeling and observational errors were considered in the framework of data assimilation, followed by an ensemble size sensitivity experiment. Once the appropriate model error and ensemble size was determined, a simulation study focused on the performance of a data assimilation system, based on the correlation between streamflow observation and model states, was conducted. The EnSRF method was implemented within HyMOD and results for flash flood forecasting were analyzed, where the calibrated streamflow simulation without state updating was treated as the benchmark or nature run. Results for twenty-four flash-flood events in total from the two watersheds indicated that the data assimilation approach effectively improved the predictions of peak flows and the hydrographs in general. This study demonstrated the benefit and efficiency of implementing data assimilation into a hydrological model to improve flash flood forecasting over small, instrumented basins with potential application to real-time alert systems.