Comparison of data-driven modelling techniques for river flow forecasting

Abstract

Accurate forecasting of streamflow is essential for the efficient operation of water resources systems. The streamflow process is complex and highly nonlinear. Therefore, researchers try to devise alterative techniques to forecast streamflow with relative ease and reasonable accuracy, although traditional deterministic and conceptual models are available. The present work uses three data-driven techniques, namely artificial neural networks (ANN), genetic programming (GP) and model trees (MT) to forecast river flow one day in advance at two stations in the Narmada catchment of India, and the results are compared. All the models performed reasonably well as far as accuracy of prediction is concerned. It was found that the ANN and MT techniques performed almost equally well, but GP performed better than both these techniques, although only marginally in terms of prediction accuracy in normal and extreme events.

Citation Londhe, S. & Charhate, S. (2010) Comparison of data-driven modelling techniques for river flow forecasting. Hydrol. Sci. J. 55(7), 1163–1174.     

Genetic programming model for forecast of short and noisy data

Though forecasting of river flow has received a great deal of attention from engineers and researchers throughout the world, this still continues to be a challenging task owing to the complexity of the process. In the last decade or so, artificial neural networks (ANNs) have been widely applied, and their ability to model complex phenomena has been clearly demonstrated. However, the success of ANNs depends very crucially on having representative records of sufficient length. Further, the forecast accuracy decreases rapidly with an increase in the forecast horizon. In this study, the use of the Darwinian theory‐based recent evolutionary technique of genetic programming (GP) is suggested to forecast fortnightly flow up to 4‐lead. It is demonstrated that short lead predictions can be significantly improved from a short and noisy time series if the stochastic (noise) component is appropriately filtered out. The deterministic component can then be easily modelled. Further, only the immediate antecedent exogenous and/or non‐exogenous inputs can be assumed to control the process. With an increase in the forecast horizon, the stochastic components also play an important role in the forecast, besides the inherent difficulty in ascertaining the appropriate input variables which can be assumed to govern the underlying process. GP is found to be an efficient tool to identify the most appropriate input variables to achieve reasonable prediction accuracy for higher lead‐period forecasts. A comparison with ANNs suggests that though there is no significant difference in the prediction accuracy, GP does offer some unique advantages. Copyright © 2006 John Wiley & Sons, Ltd.     

Comparative study of monthly inflow prediction methods for the Three Gorges Reservoir

Due to the complexity of influencing factors and the limitation of existing scientific knowledge, current monthly inflow prediction accuracy is unable to meet the requirements of various water users yet. A flow time series is usually considered as a combination of quasi-periodic signals contaminated by noise, so prediction accuracy can be improved by data preprocess. Singular spectrum analysis (SSA), as an efficient preprocessing method, is used to decompose the original inflow series into filtered series and noises. Current application of SSA only selects filtered series as model input without considering noises. This paper attempts to prove that noise may contain hydrological information and it cannot be ignored, a new method that considerers both filtered and noises series is proposed. Support vector machine (SVM), genetic programming (GP), and seasonal autoregressive (SAR) are chosen as the prediction models. Four criteria are selected to evaluate the prediction model performance: Nash–Sutcliffe efficiency, Water Balance efficiency, relative error of annual average maximum (RE_max) monthly flow and relative error of annual average minimum (RE_min) monthly flow. The monthly inflow data of Three Gorges Reservoir is analyzed as a case study. Main results are as following: (1) coupling with the SSA, the performance of the SVM and GP models experience a significant increase in predicting the inflow series. However, there is no significant positive change in the performance of SAR (1) models. (2) After considering noises, both modified SSA-SVM and modified SSA-GP models perform better than SSA-SVM and SSA-GP models. Results of this study indicated that the data preprocess method SSA can significantly improve prediction precision of SVM and GP models, and also proved that noises series still contains some information and has an important influence on model performance.     

An improved genetic programming to SSM/I estimation typhoon precipitation over ocean

This article proposes an improved multi‐run genetic programming (GP) and applies it to estimate the typhoon rainfall over ocean using multi‐variable meteorological satellite data. GP is a well‐known evolutionary programming and data mining method used to automatically discover the complex relationships among nonlinear systems. The main advantage of GP is to optimize appropriate types of function and their associated coefficients simultaneously. However, the searching efficiency of traditional GP can be decreased by the complex structure of parse tree to represent the multiple input variables. This study processed an improvement to enhance escape ability from local optimums during the optimization procedure. We continuously run GP several times by replacing the terminal nodes at the next run with the best solution at the current run. The current method improves GP, obtaining a highly nonlinear meaningful equation to estimate the rainfall. In the case study, this improved GP (IGP) described above combined with special sensor microwave imager (SSM/I) seven channels was employed. These results are then verified with the data from four offshore rainfall stations located on islands around Taiwan. The results show that the IGP generates sophisticated and accurate multi‐variable equation through two runs. The performance of IGP outperforms the traditional multiple linear regression, back‐propagated network (BPN) and three empirical equations. Because the extremely high values of precipitation rate are quite few and the number of zero values (no rain) is very large, the underestimations of heavy rainfall are obvious. A simple genetic algorithm was therefore used to search for the optimal threshold value of SSM/I channels, detecting the data of no rain. The IGP with two runs, used to construct an appropriate mathematical function to estimate the precipitation, can obtain more favourable results from estimating extremely high values. Copyright © 2011 John Wiley & Sons, Ltd.     

国内外隧道超前预报技术评析与推介

国内外的隧道地质超前预报技术正在发展之中,目前应用的负视速度法,水平剖面法,TSP、TGP等技术,由于观测方式和资料处理方法比较简单,不能准确确定掌子面前方围岩的波速,从而致使地质界面定位不准确,围岩工程类别划分缺乏依据,亟待改进.TRT与TST技术采用空间观测方式和偏移成像方法,技术比较先进,能解决了掌子面前方围岩速度结构可靠分析的问题,实现了隧道围岩地质结构的精确成像,适合复杂地质条件下的地质超前预报应用.目前的超前预报过于依赖地震方法,实际上地震方法对于解决构造问题比较有效,解决含水性问题不如电磁方法,高密度电法和瞬变电磁方法在超前预报领域会有广阔的应用前景.实际预报工作中要取得好的预报结果,必须提倡综合方法,提倡地质分析与工程物探相结合,地震方法与地磁方法相结合,共同解决不良地质构造、含水构造、含瓦斯气构造的超前预报问题.     

Performance assessment of genetic programming (GP) and minimax probability machine regression (MPMR) for prediction of seismic ultrasonic attenuation

The determination of seismic attenuation (s) (dB/cm) is a challenging task in earthquake science. This article employs genetic programming (GP) and minimax probability machine regression (MPMR) for prediction of s. GP is developed based on genetic algorithm. MPMR maximizes the minimum probability of future predictions being within some bound of the true regression function. Porosity (n) (%), permeability (k) (millidarcy), grain size (d) (μm), and clay content (c) (%) have been considered as inputs of GP and MPMR. The output of GP and MPMR is s. The developed GP gives an equation for prediction of s. The results of GP and MPMR have been compared with the artificial neural network. This article gives robust models based on GP and MPMR for prediction of s.     

Short-term prediction of influent flow in wastewater treatment plant

Predicting influent flow is important in the management of a wastewater treatment plant (WWTP). Because influent flow includes municipal sewage and rainfall runoff, it exhibits nonlinear spatial and temporal behavior and therefore makes it difficult to model. In this paper, a neural network approach is used to predict influent flow in the WWTP. The model inputs include historical influent data collected at a local WWTP, rainfall data and radar reflectivity data collected by the local weather station. A static multi-layer perceptron neural network performs well for the current time prediction but a time lag occurs and increases with the time horizon. A dynamic neural network with an online corrector is proposed to solve the time lag problem and increase the prediction accuracy for longer time horizons. The computational results show that the proposed neural network accurately predicts the influent flow for time horizons up to 300 min.     

Prediction of hourly actual evapotranspiration using neural networks,genetic programming,and statistical models

The complexity of the evapotranspiration process and its variability in time and space have imposed some limitations on previously developed evapotranspiration models. In this study, two data‐driven models: genetic programming (GP) and artificial neural networks (ANNs), and statistical regression models were developed and compared for estimating the hourly eddy covariance (EC)‐measured actual evapotranspiration (AET) using meteorological variables. The utility of the investigated data‐driven models was also compared with that of HYDRUS‐1D model, which makes use of conventional Penman–Monteith (PM) model for the prediction of AET. The latent heat (LE), which is measured using the EC method, is modelled as a function of five climatic variables: net radiation, ground temperature, air temperature, relative humidity, and wind speed in a reconstructed landscape located in Northern Alberta, Canada. Several ANN models were evaluated using two training algorithms of Levenberg–Marquardt and Bayesian regularization. The GP technique was used to generate mathematical equations correlating AET to the five climatic variables. Furthermore, the climatic variables, as well as their two‐factor interactions, were statistically analysed to obtain a regression equation and to indicate the climatic factors having significant effect on the evapotranspiration process. HYDRUS‐1D model as an available physically based model was examined for estimating AET using climatic variables, leaf area index (LAI), and soil moisture information. The results indicated that all three proposed data‐driven models were able to approximate the AET reasonably well; however, GP and regression models had better generalization ability than the ANN model. The results of HYDRUS‐1D model exhibited that a physically based model, such as HYDRUS‐1D, might be comparable or even inferior to the data‐driven models in terms of the overall prediction accuracy. Based on the developed GP and regression models, net radiation and ground temperature had larger contribution to the AET process than other variables. Copyright © 2010 John Wiley & Sons, Ltd.     

Dynamics of hourly sea level at Hillarys Boat Harbour,Western Australia: a chaos theory perspective

Water level forecasting using recorded time series can provide a local modelling capability to facilitate local proactive management practices. To this end, hourly sea water level time series are investigated. The records collected at the Hillarys Boat Harbour, Western Australia, are investigated over the period of 2000 and 2002. Two modelling techniques are employed: low-dimensional dynamic model, known as the deterministic chaos theory, and genetic programming, GP. The phase space, which describes the evolution of the behaviour of a nonlinear system in time, was reconstructed using the delay-embedding theorem suggested by Takens. The presence of chaotic signals in the data was identified by the phase space reconstruction and correlation dimension methods, and also the predictability into the future was calculated by the largest Lyapunov exponent to be 437 h or 18 days into the future. The intercomparison of results of the local prediction and GP models shows that for this site-specific dataset, the local prediction model has a slight edge over GP. However, rather than recommending one technique over another, the paper promotes a pluralistic modelling culture, whereby different techniques should be tested to gain a specific insight from each of the models. This would enable a consensus to be drawn from a set of results rather than ignoring the individual insights provided by each model.     

Dairy of forthcoming events

Abstract

Due to its great importance, the availability of long flow records, contemporary as well as older, and the additional historical information of its behaviour, the Nile is an ideal test case for identifying and understanding hydrological behaviours, and for model development. Such behaviours include the long-term persistence, which historically has motivated the discovery of the Hurst phenomenon and has put into question classical statistical results and typical stochastic models. Based on the empirical evidence from the exploration of the Nile flows and on the theoretical insights provided by the principle of maximum entropy, a concept newly employed in hydrological stochastic modelling, an advanced yet simple stochastic methodology is developed. The approach is focused on the prediction of the Nile flow a month ahead, but the methodology is general and can be applied to any type of stochastic prediction. The stochastic methodology is also compared with deterministic approaches, specifically an analogue (local nonlinear chaotic) model and a connectionist (artificial neural network) model based on the same flow record. All models have good performance with the stochastic model outperforming in prediction skills and the analogue model in simplicity. In addition, the stochastic model has other elements of superiority such as the ability to provide long-term simulations and to improve understanding of natural behaviours.     

Experimental study of extreme shear stress for shallow flow under simulated rainfall

The raindrop impact and overland flow are two major factors causing soil detachment and particle transportation. In this study, the turbulent characteristics of the shallow rain‐impacted water flow were investigated using a 2‐D fibre‐optic laser Doppler velocimetry (FLDV) and an artificial rainfall simulator. The fluctuating turbulent shear stress was computed using digital data processing techniques. The experimental data showed that the Reynolds shear stress follows a probability distribution with heavy tails. The tail probability increases with an increase of rainfall intensity or raindrop diameter, and it decreases with an increase of Reynolds number. A modified empirical equation was derived using both the raindrop diameter and rainfall intensity as independent variables to provide a better prediction of the Darcy‐Weisbach friction coefficient f under rainfall conditions. Copyright © 2009 John Wiley & Sons, Ltd.     

A multiscale method for subsurface inverse modeling: Single-phase transient flow

High-resolution geologic models that incorporate observed state data are expected to effectively enhance the reliability of reservoir performance prediction. One of the major challenges faced is how to solve the large-scale inverse modeling problem, i.e., to infer high-resolution models from the given observations of state variables that are related to the model parameters according to some known physical rules, e.g., the flow and transport partial differential equations. There are typically two difficulties, one is the high-dimensional problem and the other is the inverse problem. A multiscale inverse method is presented in this work to attack these problems with the aid of a gradient-based optimization algorithm. In this method, the model responses (i.e., the simulated state data) can be efficiently computed from the high-resolution model using the multiscale finite-volume method. The mismatch between the observations and the multiscale solutions is then used to define a proper objective function, and the fine-scale sensitivity coefficients (i.e., the derivatives of the objective function with respect to each node’s attribute) are computed by a multiscale adjoint method for subsequent optimization. The difficult high-dimensional optimization problem is reduced to a one-dimensional one using the gradient-based gradual deformation method. A synthetic single-phase transient flow example problem is employed to illustrate the proposed method. Results demonstrate that the multiscale framework presented is not only computationally efficient but also can generate geologically consistent models. By preserving spatial structure for inverse modeling, the method presented overcomes the artifacts introduced by the multiscale simulation and may enhance the prediction ability of the inverse-conditional realizations generated.     

Geostatistical radar–raingauge merging: A novel method for the quantification of rain estimation accuracy

Compared to other estimation techniques, one advantage of geostatistical techniques is that they provide an index of the estimation accuracy of the variable of interest with the kriging estimation standard deviation (ESD). In the context of radar–raingauge quantitative precipitation estimation (QPE), we address in this article the question of how the kriging ESD can be transformed into a local spread of error by using the dependency of radar errors to the rain amount analyzed in previous work. The proposed approach is implemented for the most significant rain events observed in 2008 in the Cévennes-Vivarais region, France, by considering both the kriging with external drift (KED) and the ordinary kriging (OK) methods. A two-step procedure is implemented for estimating the rain estimation accuracy: (i) first kriging normalized ESDs are computed by using normalized variograms (sill equal to 1) to account for the observation system configuration and the spatial structure of the variable of interest (rainfall amount, residuals to the drift); (ii) based on the assumption of a linear relationship between the standard deviation and the mean of the variable of interest, a denormalization of the kriging ESDs is performed globally for a given rain event by using a cross-validation procedure. Despite the fact that the KED normalized ESDs are usually greater than the OK ones (due to an additional constraint in the kriging system and a weaker spatial structure of the residuals to the drift), the KED denormalized ESDs are generally smaller the OK ones, a result consistent with the better performance observed for the KED technique. The evolution of the mean and the standard deviation of the rainfall-scaled ESDs over a range of spatial (5–300 km²) and temporal (1–6 h) scales demonstrates that there is clear added value of the radar with respect to the raingauge network for the shortest scales, which are those of interest for flash-flood prediction in the considered region.     

The influence of rainfall on PR radar measurement of ocean surface wind speed and its calibration

Rain can significantly degrade the wind vector retrieval from Precipitation Radar(PR) by three mechanisms, namely, two-way rain attenuation, rain volume-backscattering, and ocean surface roughening from the rain splash effect. Here we first derive the radar equation for PR in rainy conditions. Then we use the rain attenuation model for Ku band, volume backscatter model for spherical raindrops and PR–TMI(TRMM Microwave Imager, TMI) matchup datasets from June to August in 2010 to solve the radar equation, and quantitatively analyze the influence of rainfall on PR radar measurement of ocean surface wind speed. Our results show that the significant effect of rain on radar signal is dominated by two-way rain attenuation and rain splash effect, and the effect of rain volume-backscattering is relatively the weakest, which can even be neglected in rain-weak conditions. Moreover, both the two-way rain attenuation and rain splash effect increase with the increasing of integration rain rate and incident angle. Last, we combine volume-backscattering effect and splash effect into a simple phenomenological model for rain calibration and select three typhoon cases from June to August in 2012 to verify the accuracy of this model. Before calibration, the mean difference and mean square error(MSE) between PR-observed ? 0 and wind-induced ? 0 are about 2.95 dB and 3.10 dB respectively. However, after calibration, the mean difference and MSE are reduced to 0.64 dB and 1.61 dB respectively. The model yields an accurate calibration for PR near-nadir normalized radar cross section(NRCS) in rainy conditions.     

A simple technique for measuring rainfall interception by small shrub: “interception flow collection box”

In this paper a simple technique for field measurement of rain water loss arising from interception and water flows associated with species of small Mediterranean shrub is described: the ‘interception flow collection box’. This technique solves the problem of installing devices to control stemflow in species with a multiple trunk and demonstrates its efficiency through the results obtained from the data observed for three species of semi-arid Mediterranean shrub: Juniperus oxycedrus, Rosmarinus officinalis and Thymus vulgaris. Finally, the empirical equations for the prediction of throughfall, stemflow and rain water loss through interception are presented for the three selected species and the validity of the technique employed is established. © 1998 John Wiley & Sons, Ltd.     

Integrated cave drip monitoring for epikarst recharge estimation in a dry Mediterranean area,Sif Cave,Israel

Understanding recharge mechanisms and controls in karst regions is extremely important for managing water resources because of the dynamic nature of the system. The objective of this study was to evaluate water percolation through epikarst by monitoring water flow into a cave and conducting artificial irrigation and tracer experiments, at Sif Cave in Wadi Sussi, Israel from 2005 through 2007. The research is based on continuous high‐resolution direct measurements of both rainfall and water percolation in the cave chamber collected by three large PVC sheets which integrate drips from three different areas (17, 46, and 52 m²). Barrels equipped with pressure transducers record drip rate and volume for each of the three areas. The combined measured rainfall and cave data enables estimation of recharge into the epikarst and to better understand the relationship of rainfall‐recharge. Three distinct types of flow regimes were identified: (1) ‘Quick flow’ through preferential flow paths (large fractures and conduits); (2) ‘Intermediate flow’ through a secondary crack system; and (3) ‘Slow flow’ through the matrix. A threshold of ～100 mm of rain at the beginning of the rainy season is required to increase soil water content allowing later rainfall events to percolate deeper through the soil and to initiate dripping in the cave. During winter, as the soil water content rises, the lag time between a rain event and cave drip response decreases. Annual recharge (140–160 mm in different areas in the cave) measured represents 30–35% of annual rainfall (460 mm). Copyright © 2011 John Wiley & Sons, Ltd.     

Simulation of dam-break flow with grid adaptation

The unsteady free surface flow caused by sudden collapse of a dam produces discontinuities in the flow variables. As the flow surges downstream, it forms a moving bore front with steep gradients of water height and velocity. In the numerical simulation of this flow, proper grid distribution can play a crucial part in the prediction and resolution of the solutions. The use of presently available numerical schemes to solve this problem on a uniform course grid system fails to resolve the characteristic flow features and hence do a poor job in simulating this flow. In this paper, an adaptive grid which adjusts itself as the solution evolves is used for a better resolution of the flow properties. Rai and Anderson's¹² method is used to determine the grid speed; however, a different partial differential equation based on the conservative principle of grid arc lengths for clustering grids in one-dimensional flow is used along with the St. Venant equations to numerically simulate the flow. Both the subcritical and the supercritical flows under extreme boundary conditions are solved using this technique. With a specified number of grid points, this provides better quality solutions as compared to those obtained with uniformly distributed grids.     

Collected Rain Water as Cost-Efficient Source for Aquifer Tracer Testing

Locally collected precipitation water can be actively used as a groundwater tracer solution based on four inherent tracer signals: electrical conductivity, stable isotopic signatures of deuterium [δ²H], oxygen-18 [δ¹⁸O], and heat, which all may strongly differ from the corresponding background values in the tested groundwater. In hydrogeological practice, a tracer test is one of the most important methods for determining subsurface connections or field parameters, such as porosity, dispersivity, diffusion coefficient, groundwater flow velocity, or flow direction. A common problem is the choice of tracer and the corresponding permission by the appropriate authorities. This problem intensifies where tracer tests are conducted in vulnerable conservation or water protection areas (e.g., around drinking water wells). The use of (if required treated) precipitation as an elemental groundwater tracer is a practical solution for this problem, as it does not introduce foreign matters into the aquifer system, which may contribute positively to the permission delivery. Before tracer application, the natural variations of the participating end members' tracer signals have to be evaluated locally. To obtain a sufficient volume of tracer solution, precipitation can be collected as rain using a detached, large-scale rain collector, which will be independent from possibly existing surfaces like roofs or drained areas. The collected precipitation is then stored prior to a tracer experiment.     

Rangeland hydrology and erosion model (RHEM) enhancements for applications on disturbed rangelands

The rangeland hydrology and erosion model (RHEM) is a new process‐based model developed by the USDA Agricultural Research Service. RHEM was initially developed for functionally intact rangelands where concentrated flow erosion is minimal and most soil loss occurs by rain splash and sheet flow erosion processes. Disturbance such as fire or woody plant encroachment can amplify overland flow erosion by increasing the likelihood of concentrated flow formation. In this study, we enhanced RHEM applications on disturbed rangelands by using a new approach for the prediction and parameterization of concentrated flow erosion. The new approach was conceptualized based on observations and results of experimental studies on rangelands disturbed by fire and/or by tree encroachment. The sediment detachment rate for concentrated flow was calculated using soil erodibility and hydraulic (flow width and stream power) parameters. Concentrated flow width was calculated based on flow discharge and slope using an equation developed specifically for disturbed rangelands. Soil detachment was assumed to begin with concentrated flow initiation. A dynamic erodibility concept was applied where concentrated flow erodibility was set to decrease exponentially during a run‐off event because of declining sediment availability. Erodibility was estimated using an empirical parameterization equation as a function of vegetation cover and surface soil texture. A dynamic partial differential sediment continuity equation was used to model the total detachment rate of concentrated flow and rain splash and sheet flow. The enhanced version of the model was evaluated against rainfall simulation data for three different sites that exhibit some degree of disturbance by fire and/or by tree encroachment. The coefficient of determination (R²) and Nash–Sutcliffe efficiency were 0.78 and 0.71, respectively, which indicates the capability of the model using the new approach for predicting soil loss on disturbed rangeland. By using the new concentrated flow modelling approach, the model was enhanced to be a practical tool that utilizes readily available vegetation and soil data for quantifying erosion and assessing erosion risk following rangeland disturbance. Copyright © 2014 John Wiley & Sons, Ltd.