Group-based estimation of missing hydrological data: I. Approach and general methodology

Abstract

In this first paper in a set of two, the problem of estimating missing segments in streamflow records is described. The group approach, different from the traditional single-valued approach, is proposed and explained. The approach perceives the hydrological data as sequence of groups rather than single-valued observations. The techniques suggested to handle the group approach are regression, time series analysis, partitioning modelling, and artificial neural networks. Pertinent literature is reviewed and background material is used to support the group approach. Implementation and comparisons of models' performance are deferred to the second paper.     

Kernel distributed residual function in a revised multiple order autoregressive model and its applications in hydrology

ABSTRACT

This paper describes a new approach to fill missing data in hydrologic series. Based on a multiple-order autoregressive model, our algorithm represents the random term with an empirical distribution function that includes different parameters for the low, medium and high ranges of the modelled hydrologic variable. The algorithm involves a corrective mechanism that preserves the original statistical distribution of the series that are filled, while also eliminating the possibility of obtaining negative values for low flows. The algorithm requires multiple correlated hydrologic time series with sufficient data to permit accurate calculation of their statistical properties. It ensures that both the original statistical dependence among the data series and the statistical distribution functions will be preserved after the missing data had been filled. The model has been tested using 15 streamflow series in the Upper Bow River watershed in Alberta, Canada.     

Analysis of cross-correlated chaotic streamflows

Abstract

A trial is made to explore the applicability of chaos analysis outside the commonly reported analysis of a single chaotic time series. Two cross-correlated streamflows, the Little River and the Reed Creek, Virginia, USA, are analysed with regard to the chaotic behaviour. Segments of missing data are assumed in one of the time series and estimated using the other complete time series. Linear regression and artificial neural network models are employed. Two experiments are conducted in the analysis: (a) fitting one global model and (b) fitting multiple local models. Each local model is in the direct vicinity of the missing data. A nonlinear noise reduction method is used to reduce the noise in both time series and the two experiments are repeated. It is found that using multiple local models to estimate the missing data is superior to fitting one global model with regard to the mean squared error and the mean relative error of the estimated values. This result is attributed to the chaotic behaviour of the streamflows under consideration.     

Spatial interpolation using nonlinear mathematical programming models for estimation of missing precipitation records

Abstract

New mathematical programming models are proposed, developed and evaluated in this study for estimating missing precipitation data. These models use nonlinear and mixed integer nonlinear mathematical programming (MINLP) formulations with binary variables. They overcome the limitations associated with spatial interpolation methods relevant to the arbitrary selection of weighting parameters, the number of control points within a neighbourhood, and the size of the neighbourhood itself. The formulations are solved using genetic algorithms. Daily precipitation data obtained from 15 rain gauging stations in a temperate climatic region are used to test and derive conclusions about the efficacy of these methods. The developed methods are compared with some naïve approaches, multiple linear regression, nonlinear least-square optimization, kriging, and global and local trend surface and thin-plate spline models. The results suggest that the proposed new mathematical programming formulations are superior to those obtained from all the other spatial interpolation methods tested in this study.

Editor D. Koutsoyiannis; Associate editor S. Grimaldi

Citation Teegavarapu, R.S.V., 2012. Spatial interpolation using nonlinear mathematical programming models for estimation of missing precipitation records. Hydrological Sciences Journal, 57 (3), 383–406.     

Suspended sediment estimation using neuro-fuzzy and neural network approaches/Estimation des matières en suspension par des approches neurofloues et à base de réseau de neurones

Abstract

The abilities of neuro-fuzzy (NF) and neural network (NN) approaches to model the streamflow–suspended sediment relationship are investigated. The NF and NN models are established for estimating current suspended sediment values using the streamflow and antecedent sediment data. The sediment rating curve and multi-linear regression are also applied to the same data. Statistic measures were used to evaluate the performance of the models. The daily streamflow and suspended sediment data for two stations—Quebrada Blanca station and Rio Valenciano station—operated by the US Geological Survey were used as case studies. Based on comparison of the results, it is found that the NF model gives better estimates than the other techniques.     

Incorporating structural uncertainty of hydrological models in likelihood functions via an ensemble range approach

ABSTRACT

Model ensembles are possibly the most powerful tool to assess uncertainties in runoff predictions stemming from inadequacies in model structure. But in many applications little knowledge is gained about the specific weaknesses of the individual models. Here we introduce the ensemble range approach (ERA). Compared to other ensemble techniques, ERA is primarily intended to facilitate hydrological reasoning about model structural uncertainty. This is attempted by separate modelling of data uncertainty and structural uncertainty with two different error density functions that are combined in one likelihood function. The width of the structural error density is in accordance with the range of runoff predictions calculated by a small model ensemble at each individual time step. Albeit not the only choice, this study is restricted on the use of a modified beta density to represent structural uncertainty. The performance of ERA is assessed in some synthetic and real data case studies. Ensembles of two structurally identical models are applied, made possible by estimating the parameters of ERA and both models simultaneously.

Editor D. Koutsoyiannis; GUEST editor S. Weijs     

Evaluation of efficiency of different estimation methods for missing climatological data

Reliable estimation of missing data is an important task for meteorologists, hydrologists and environment protection workers all over the world. In recent years, artificial intelligence techniques have gained enormous interest of many researchers in estimating of missing values. In the current study, we evaluated 11 artificial intelligence and classical techniques to determine the most suitable model for estimating of climatological data in three different climate conditions of Iran. In this case, 5 years (2001–2005) of observed data at target and neighborhood stations were used to estimate missing data of monthly minimum temperature, maximum temperature, mean air temperature, relative humidity, wind speed and precipitation variables. The comparison includes both visual and parametric approaches using such statistic as mean absolute errors, coefficient of efficiency and skill score. In general, it was found that although the artificial intelligence techniques are more complex and time-consuming models in identifying their best structures for optimum estimation, but they outperform the classical methods in estimating missing data in three distinct climate conditions. Moreover, the in-filling done by artificial neural network rivals that by genetic programming and sometimes becomes more satisfactory, especially for precipitation data. The results also indicated that multiple regression analysis method is the suitable method among the classical methods. The results of this research proved the high importance of choosing the best and most precise method in estimating different climatological data in Iran and other arid and semi-arid regions.     

Reconstruction of groundwater levels to impute missing values using singular and multichannel spectrum analysis: application to the Ardabil Plain,Iran

ABSTRACT

Groundwater-level time series often have a substantial number of missing values which should be taken into consideration before using them for further analysis, particularly for numerical groundwater flow modelling applications. This study aims to comprehensively compare two data-driven models, singular spectrum analysis (SSA) and multichannel spectrum analysis (MSSA), to reconstruct groundwater-level time series and impute the missing values for 25 piezometric stations in Ardabil Plain, northwest Iran. The reconstructed groundwater-level time series are assessed against the complete observed groundwater time series, while the imputed values are appraised against the artificially created gap values. The results show that both SSA and MSSA demonstrate a solid competency in imputation and reconstruction of groundwater-level data. However, depending on the spatial correlation between the piezometers, and the most suitable probability distribution function (pdf) fitted to the time series of each piezometer, the performance may vary from piezometer to piezometer.     

Improving the performance of data-driven techniques through data pre-processing for modelling daily reservoir inflow

Abstract

The effect of data pre-processing while developing artificial intelligence (AI) -based data-driven techniques, such as artificial neural networks (ANN), model trees (MT) and linear genetic programming (LGP), is studied for Pawana Reservoir in Maharashtra, India. The daily one-step-ahead inflow forecasts are compared with flows generated from a univariate autoregressive integrated moving average (ARIMA) model. For the full-year data series, a large error is found mainly due to the occurrence of zero values, since the reservoir is located in an intermittent river. Hence, all the techniques are evaluated using two data series: 18 years of daily full-year inflow data (from 1 January to 31 December); and 18 years of daily monsoon season inflow data (from 1 June to 31 October) to take into account the intermittent nature of the data. The relevant range of inputs for each category is selected based on autocorrelation and partial autocorrelation analyses of the inflow series. Conventional pre-processing methods, such as transformation and/or normalization of data, do not perform well because of the large variation in magnitudes, as well as the many zero values (65% of the full-year data series). Therefore, the input data are pre-processed into un-weighted moving average (MA) series of 3 days, 5 days and 7 days. The 3-day MA series performs better, maintaining the peak inflow pattern as in the actual data series, while the coarser-scale (5-day and 7-day) MA series reduce the peak inflow pattern, leading to more errors in peak inflow prediction. The results indicate that AI methods are powerful tools for modelling the daily flow time series with appropriate data pre-processing, in spite of the presence of many zero values. The time-lagged recurrent network (TLRN) ANN modelling technique applied in this study maps the inflow forecasting in a better way than the standard multilayer perceptron (MLP) neural networks, especially in the case of the seasonal data series. The MT technique performs equally well for low and medium inflows, but fails to predict the peak inflows. However, LGP outperforms the other AI models, and also the ARIMA model, for all inflow magnitudes. In the LGP model, the daily full-year data series with more zero inflow values performs better than the daily seasonal models.

Citation Jothiprakash, V. & Kote, A. S. (2011) Improving the performance of data-driven techniques through data pre-processing for modelling daily reservoir inflow. Hydrol. Sci. J. 56(1), 168–186.     

Data time-step dependency of conceptual rainfall—streamflow model parameters: an empirical study with implications for regionalisation

Abstract

Employing 1-, 2-, 4-, 6-, 12- and 24-hourly data sets for two catchments (10.6 and 298 km²) in Wales, the calibrated parameters of a unit hydrograph-based model are shown to change substantially over that range of data time steps. For the smaller basin, each model parameter reaches, or approaches, a stable value as the data time step decreases, providing a straightforward method of estimating time-step independent model parameter values. For the larger basin, the model parameters also reach, or approach, stable values using hourly data, but, for reasons given in the paper, interpretation of the results is more difficult. Model parameter sensitivity analyses are presented that give insights into the relative precision on the parameters for both catchments. The paper discusses the importance of accounting for model parameter data time-step dependency in pursuit of a reduction in the uncertainty associated with estimates of flow in ungauged basins, and suggests that further work along these lines be undertaken using different catchments and models.     

Transfer function noise modelling of groundwater level fluctuation using threshold rainfall-based binary-weighted parameter estimation approach

Considerable uncertainty occurs in the parameter estimates of traditional rainfall–water level transfer function noise (TFN) models, especially with the models built using monthly time step datasets. This is due to the equal weights assigned for rainfall occurring during both water level rise and water level drop events while estimating the TFN model parameters using the least square technique. As an alternative to this approach, a threshold rainfall-based binary-weighted least square method was adopted to estimate the TFN model parameters. The efficacy of this binary-weighted approach in estimating the TFN model parameters was tested on 26 observation wells distributed across the Adyar River basin in Southern India. Model performance indices such as mean absolute error and coefficient of determination values showed that the proposed binary-weighted approach of fitting independent threshold-based TFN models for water level rise and water level drop scenarios considerably improves the model accuracy over other traditional TFN models.

EDITOR D. Koutsoyiannis

ASSOCIATE EDITOR A. Fiori     

Event-based estimation of water budget components using a network of multi-sensor capacitance probes

Abstract

A new approach was developed for estimating vertical soil water fluxes using soil water content time series data. Instead of a traditional fixed time interval, this approach utilizes the time interval between two sequential minima of the soil water storage time series to identify groundwater recharge events and calculate components of the soil water budget. We calculated water budget components: surface-water excess (Sw), infiltration less evapotranspiration (I – ET) and groundwater recharge (R) from May 2001 to January 2003 at eight locations at the USDA Agricultural Research Center, Beltsville, Maryland, USA. High uncertainty was observed for all budget components. This uncertainty was attributed to spatial and temporal variation in Sw, I – ET and R, and was caused by nonuniform rainfall distributions during recharge events, variability in the profile water content, and spatial variability in soil hydraulic properties. The proposed event-based approach allows estimating water budget components when profile water content monitoring data are available.

Citation Guber, A., Gish, T., Pachepsky, Y., McKee, L., Nicholson, T. & Cady, R. (2011) Event-based estimation of water budget components using a network of multi-sensor capacitance probes. Hydrol. Sci. J. 56(7), 1227–1241.     

BIBLIOGRAPHIE

Abstract

Time series modelling approaches are useful tools for simulating and forecasting hydrological variables and their change through time. Although linear time series models are common in hydrology, the nonlinear time series model, the Generalized Autoregressive Conditional Heteroscedasticity (GARCH) model, has rarely been used in hydrology and water resources engineering. The GARCH model considers the conditional variance remaining in the residuals of the linear time series models, such as an ARMA or an ARIMA model. In the present study, the advantages of a GARCH model against a linear ARIMA model are investigated using three classes of the GARCH approach, namely Power GARCH, Threshold GARCH and Exponential GARCH models. A daily streamflow time series of the Matapedia River, Quebec, Canada, is selected for this study. It is shown that the ARIMA (13,1,4) model is adequate for modelling streamflow time series of Matapedia River, but the Engle test shows the existence of heteroscedasticity in the residuals of the ARIMA model. Therefore, an ARIMA (13,1,4)-GARCH (3,1) error model is fitted to the data. The residuals of this model are examined for the existence of heteroscedasticity. The Engle test indicates that the GARCH model has considerably reduced the heteroscedasticity of the residuals. However, the Exponential GARCH model seems to completely remove the heteroscedasticity from the residuals. The multi-criteria evaluation for model performance also proves that the Exponential GARCH model is the best model among ARIMA and GARCH models. Therefore, the application of a GARCH model is strongly suggested for hydrological time series modelling as the conditional variance of the residuals of the linear models can be removed and the efficiency of the model will be improved.

Editor D. Koutsoyiannis; Associate editor C. Onof

Citation Modarres, R. and Ouarda, T.B.M.J., 2013. Modelling heteroscedasticty of streamflow times series. Hydrological Sciences Journal, 58 (1), 1–11.     

Application of adaptive fuzzy rule-based models for reconstruction of missing precipitation events

Abstract

This paper describes a fuzzy rule-based approach applied for reconstruction of missing precipitation events. The working rules are formulated from a set of past observations using an adaptive algorithm. A case study is carried out using the data from three precipitation stations in northern Italy. The study evaluates the performance of this approach compared with an artificial neural network and a traditional statistical approach. The results indicate that, within the parameter sub-space where its rules are trained, the fuzzy rule-based model provided solutions with low mean square error between observations and predictions. The problems that have yet to be addressed are overfitting and applicability outside the range of training data.     

Rainfall downscaling in time: theoretical and empirical comparison between multifractal and Hurst-Kolmogorov discrete random cascades

Abstract

During recent decades, intensive research has focused on techniques capable of generating rainfall time series at a fine time scale that are (fully or partially) consistent with a given series at a coarser time scale. Here we theoretically investigate the consequences on the ensemble statistical behaviour caused by the structure of a simple and widely-used approach of stochastic downscaling for rainfall time series, the discrete Multiplicative Random Cascade. We show that synthetic rainfall time series generated by these cascade models correspond to a stochastic process which is non-stationary, because its temporal autocorrelation structure depends on the position in time in an undesirable manner. Then, we propose and theoretically analyse an alternative downscaling approach based on the Hurst-Kolmogorov process, which is equally simple but is stationary. Finally, we provide Monte Carlo experiments which validate our theoretical results.

Editor Z.W. Kundzewicz

Citation Lombardo, F., Volpi, E., and Koutsoyiannis, D., 2012. Rainfall downscaling in time: theoretical and empirical comparison between multifractal and Hurst-Kolmogorov discrete random cascades. Hydrological Sciences Journal, 57 (6), 1052–1066.     

Multi-model averaging for continuous streamflow prediction in ungauged basins

ABSTRACT

This paper assesses the possibility of using multi-model averaging techniques for continuous streamflow prediction in ungauged basins. Three hydrological models were calibrated on the Nash-Sutcliffe Efficiency metric and were used as members of four multi-model averaging schemes. Model weights were estimated through optimization on the donor catchments. The averaging methods were tested on 267 catchments in the province of Québec, Canada, in a leave-one-out cross-validation approach. It was found that the best hydrological model was practically always better than the others used individually or in a multi-model framework, thus no averaging scheme performed statistically better than the best single member. It was also found that the robustness and adaptability of the models were highly influential on the models’ performance in cross-verification. The results show that multi-model averaging techniques are not necessarily suited for regionalization applications, and that models selected in such studies must be chosen carefully to be as robust as possible on the study site.

Editor M.C. Acreman; Associate editor S. Grimaldi     

Probabilistic flood forecasting for a mountainous headwater catchment using a nonparametric stochastic dynamic approach

Abstract

Hydrological models are commonly used to perform real-time runoff forecasting for flood warning. Their application requires catchment characteristics and precipitation series that are not always available. An alternative approach is nonparametric modelling based only on runoff series. However, the following questions arise: Can nonparametric models show reliable forecasting? Can they perform as reliably as hydrological models? We performed probabilistic forecasting one, two and three hours ahead for a runoff series, with the aim of ascribing a probability density function to predicted discharge using time series analysis based on stochastic dynamics theory. The derived dynamic terms were compared to a hydrological model, LARSIM. Our procedure was able to forecast within 95% confidence interval 1-, 2- and 3-h ahead discharge probability functions with about 1.40 m³/s of range and relative errors (%) in the range [–30; 30]. The LARSIM model and the best nonparametric approaches gave similar results, but the range of relative errors was larger for the nonparametric approaches.

Editor D. Koutsoyiannis; Associate editor K. Hamed

Citation Costa, A.C., Bronstert, A. and Kneis, D., 2012. Probabilistic flood forecasting for a mountainous headwater catchment using a nonparametric stochastic dynamic approach. Hydrological Sciences Journal, 57 (1), 10–25.     

Estimation of the generalized logistic distribution of extreme events using partial L-moments

Abstract

Statistical analysis of extreme events is often carried out to predict large return period events. In this paper, the use of partial L-moments (PL-moments) for estimating hydrological extremes from censored data is compared to that of simple L-moments. Expressions of parameter estimation are derived to fit the generalized logistic (GLO) distribution based on the PL-moments approach. Monte Carlo analysis is used to examine the sampling properties of PL-moments in fitting the GLO distribution to both GLO and non-GLO samples. Finally, both PL-moments and L-moments are used to fit the GLO distribution to 37 annual maximum rainfall series of raingauge station Kampung Lui (3118102) in Selangor, Malaysia, and it is found that analysis of censored rainfall samples of PL-moments would improve the estimation of large return period events.

Editor D. Koutsoyiannis; Associate editor K. Hamed

Citation Zakaria, Z.A., Shabri, A. and Ahmad, U.N., 2012. Estimation of the generalized logistic distribution of extreme events using partial L-moments. Hydrological Sciences Journal, 57 (3), 424–432.     

A revisit of NRCS-CN inspired models coupled with RS and GIS for runoff estimation

The Natural Resource Conservation Service – Curve Number (NRCS-CN) methodology is a widely used tool for estimating surface runoff, which is of prime importance in hydrological engineering, agricultural planning and management, environmental impact assessment, flood forecasting, and others fields. This article reviews the methodology and associated hydrological models used for runoff estimation along with their advantages and limitations. Furthermore, discussion focuses on the potential applications of Remote Sensing (RS) and Geographical Information System (GIS) techniques for estimating hydrological variables, such as rainfall, soil moisture and CN required for the NRCS-CN methodology, as well as future research and opportunities for improved runoff estimation at the macro scale.

EDITOR D. Koutsoyiannis

ASSOCIATE EDITOR A. Efstratiadis     

György Kovacs 1925–1988

Abstract

There has been a trend in recent years towards the development and popularity of physically-based deterministic models. However, the application of such models is not without difficulties. This paper investigates the usefulness of a conceptual single-event model for simulating floods from catchments covering a wide variety of climatic and physiographic areas. The model has been calibrated on a group of catchments and the calibrated parameter values related to physical catchment indices. The resulting quantitative relationships are assessed with respect to their value for estimating the parameter values of the model when calibration is not possible. The results indicate that the technique is likely to provide flood estimations for medium sized catchments (5–150 km²) that are more reliable than several flood estimation methods currently in use in South Africa.