Missing precipitation data estimation using optimal proximity metric-based imputation,nearest-neighbour classification and cluster-based interpolation methods

Abstract

New optimal proximity-based imputation, K-nearest neighbour (K-NN) classification and K-means clustering methods are proposed and developed for estimation of missing daily precipitation records. Mathematical programming formulations are developed to optimize the weighting, classification and clustering schemes used in these methods. Ten different binary and real-valued distance metrics are used as proximity measures. Two climatic regions, Kentucky and Florida, (temperate and tropical) in the USA, with different gauge density and network structure, are used as case studies to evaluate the new methods. A comprehensive exercise is undertaken to compare the performances of the new methods with those of several deterministic and stochastic spatial interpolation methods. The results from these comparisons indicate that the proposed methods performed better than existing methods. Use of optimal proximity metrics as weights, spatial clustering of observation sites and classification of precipitation data resulted in improvement of missing data estimates.

Editor D. Koutsoyiannis; Associate editor C. Onof     

Statistical corrections of spatially interpolated missing precipitation data estimates

Spatial interpolation methods used for estimation of missing precipitation data generally under and overestimate the high and low extremes, respectively. This is a major limitation that plagues all spatial interpolation methods as observations from different sites are used in local or global variants of these methods for estimation of missing data. This study proposes bias‐correction methods similar to those used in climate change studies for correcting missing precipitation estimates provided by an optimal spatial interpolation method. The methods are applied to post‐interpolation estimates using quantile mapping, a variant of equi‐distant quantile matching and a new optimal single best estimator (SBE) scheme. The SBE is developed using a mixed‐integer nonlinear programming formulation. K‐fold cross validation of estimation and correction methods is carried out using 15 rain gauges in a temperate climatic region of the U.S. Exhaustive evaluation of bias‐corrected estimates is carried out using several statistical, error, performance and skill score measures. The differences among the bias‐correction methods, the effectiveness of the methods and their limitations are examined. The bias‐correction method based on a variant of equi‐distant quantile matching is recommended. Post‐interpolation bias corrections have preserved the site‐specific summary statistics with minor changes in the magnitudes of error and performance measures. The changes were found to be statistically insignificant based on parametric and nonparametric hypothesis tests. The correction methods provided improved skill scores with minimal changes in magnitudes of several extreme precipitation indices. The bias corrections of estimated data also brought site‐specific serial autocorrelations at different lags and transition states (dry‐to‐dry, dry‐to‐wet, wet‐to‐wet and wet‐to‐dry) close to those from the observed series. Bias corrections of missing data estimates provide better serially complete precipitation time series useful for climate change and variability studies in comparison to uncorrected filled data series. Copyright © 2013 John Wiley & Sons, Ltd.     

Simulated precipitation fields with variance-consistent interpolation

Abstract

Gridded meteorological data are available for all of Norway as time series dating from 1961. A new way of interpolating precipitation in space from observed values is proposed. Based on the criteria that interpolated precipitation fields in space should be consistent with observed spatial statistics, such as spatial mean, variance and intermittency, spatial fields of precipitation are simulated from a gamma distribution with parameters determined from observed data, adjusted for intermittency. The simulated data are distributed in space, using the spatial pattern derived from kriging. The proposed method is compared to indicator kriging and to the current methodology used for producing gridded precipitation data. Cross-validation gave similar results for the three methods with respect to RMSE, temporal mean and standard deviation, whereas a comparison on estimated spatial variance showed that the new method has a near perfect agreement with observations. Indicator kriging underestimated the spatial variance by 60–80% and the current method produced a significant scatter in its estimates.

Citation Skaugen, T. & Andersen, J. (2010) Simulated precipitation fields with variance-consistent interpolation. Hydrol. Sci. J. 55(5), 676–686.     

Bilinear surface smoothing for spatial interpolation with optional incorporation of an explanatory variable. Part 2: Application to synthesized and rainfall data

ABSTRACT

The non-parametric mathematical framework of bilinear surface smoothing (BSS) methodology provides flexible means for spatial (two dimensional) interpolation of variables. As presented in a companion paper, interpolation is accomplished by means of fitting consecutive bilinear surface into a regression model with known break points and adjustable smoothing terms defined by means of angles formed by those bilinear surface. Additionally, the second version of the methodology (BSSE) incorporates, in an objective manner, the influence of an explanatory variable available at a considerably denser dataset. In the present study, both versions are explored and illustrated using both synthesized and real world (hydrological) data, and practical aspects of their application are discussed. Also, comparison and validation against the results of commonly used spatial interpolation methods (inverse distance weighted, spline, ordinary kriging and ordinary cokriging) are performed in the context of the real world application. In every case, the method’s efficiency to perform interpolation between data points that are interrelated in a complicated manner was confirmed. Especially during the validation procedure presented in the real world case study, BSSE yielded very good results, outperforming those of the other interpolation methods. Given the simplicity of the approach, the proposed mathematical framework’s overall performance is quite satisfactory, indicating its applicability for diverse tasks of scientific and engineering hydrology and beyond.

Editor Z. W. Kundzewicz; Associate editor A. Carsteanu     

Field survey and modelling of irrigation water quality indices in a Mediterranean island catchment: a comparison between spatial interpolation methods

ABSTRACT

A biannual survey of physico-chemical quality indices of 104 irrigation-water wells located in a cultivated plain of a Mediterranean island catchment was conducted using a multi-parameter probe. The campaign was planned so as to differentiate between the dry and wet seasons. The acquired data constituted the test bed for evaluating the results and the features of four spatial interpolation methods, i.e. ordinary kriging, universal kriging, inverse distance weighted and nearest neighbours, against those of the recently introduced bilinear surface smoothing (BSS). In several cases, BSS outperformed the other interpolation methods, especially during the two-fold cross-validation procedure. The study emphasizes the fact that both in situ measurements and good mathematical techniques for studying the spatial distribution of water quality indices are pivotal to agricultural practice management. In the specific case studied, the spatio-temporal variability of water quality parameters and the need for monitoring were evident, as low irrigation water quality was encountered throughout the study area.     

A spatial disaggregation procedure for precipitation

Abstract

A disaggregation procedure is presented to render forecast values of precipitation from an atmospheric model with spatial resolution of 11 × 11 km suitable as input for a distributed hydrological model with spatial resolution of 1.1 × 1.1 km. Statistical and morphological properties of the input field, such as spatial mean, variance, correlation structure and intermittency, are respected in the disaggregated field. The adopted approach is a combination of interpolation and simulation. The four nodal points of the atmospheric model grid cell are used both for determining the parameters of the exponential distribution for simulating precipitation values, and in a simple interpolation procedure to determine the spatial location of the precipitation values. A shifted distribution with two parameters is used in the case of full coverage of the grid cell, and a one-parameter distribution with a theoretically derived intermittency parameter is used if intermittency is present. The results are promising with respect to the statistical and morphological properties of the disaggregated field.     

Development of a fuzzy logic-based rainfall-runoff model

Abstract

Rainfall-runoff models are used to describe the hydrological behaviour of a river catchment. Many different models exist to simulate the physical processes of the relationship between precipitation and runoff. Some of them are based on simple and easy-to-handle concepts, others on highly sophisticated physical and mathematical approaches that require extreme effort in data input and handling. Recently, mathematical methods using linguistic variables, rather than conventional numerical variables applied extensively in other disciplines, are encroaching in hydrological studies. Among these is the application of a fuzzy rule-based modelling. In this paper an attempt was made to develop fuzzy rule-based routines to simulate the different processes involved in the generation of runoff from precipitation. These routines were implemented within a conceptual, modular, and semi-distributed model-the HBV model. The investigation involved determining which modules of this model could be replaced by the new approach and the necessary input data were identified. A fuzzy rule-based routine was then developed for each of the modules selected, and application and validation of the model was done on a rainfall-runoff analysis of the Neckar River catchment, in southwest Germany.     

Determining Spatially-Distributed Annual Water Balances for Ungauged Locations on Shikoku Island,Japan: A Comparison of Two Interpolators/Détermination de Bilans Hydriques Spatialisés pour des Sites Non-Jaugés de L'Īle de Shikoku,au Japon: Comparaison de Deux Interpolateurs

Abstract

Methods were evaluated for interpolating precipitation (P), evapotranspiration (ET), and runoff (RO) at ungauged points on Shikoku Island, Japan, using data gathered from gauged stations on the same island. Two methods were examined: a “local” cubic spline interpolator, which, for a given point, fitted the function exactly to nearby gauged data points; and a “global” multivariate regression interpolator, which fitted the function to all gauged data points based on their topographic positions (i.e. latitude, longitude, altitude). Local and global interpolators did not generate similar results for P and temperature (T). The spatial density of gauged data points used in the interpolation affected the performance of the interpolators. With any given density of gauged data points included in the interpolation, the local interpolator outperformed the global interpolator. The findings indicate that local interpolators are more accurate predictors of the spatial distribution of water balance components in mountainous regions such as Shikoku Island.     

A) A.I.H.S.—I.A.S.H.

Abstract

Mathematical models are the means to characterize variables quantitatively in many groundwater problems. Recent advances in applied mathematics have perfected what is now called Adomian's decomposition method (ADM), a simple modelling procedure for practical applications. Decomposition exhibits the benefits of analytical solutions (i.e. stability, analytic derivation of heads, gradients, fluxes and simple programming). It also offers the advantages of traditional numerical methods (i.e. consideration of heterogeneity, irregular domain shapes and multiple dimensions). In addition, decomposition is one of the few systematic procedures for solving nonlinear equations. By far its greatest advantage is its simplicity of application. It may produce simple results for preliminary simulations, or in cases with scarce information. The method is described with simple applications to regional groundwater flow. Many applications in groundwater flow and contaminant transport are available in the literature.

Editor D. Koutsoyiannis; Associate editor Xi Chen

Citation Serrano, S.E., 2013. A simple approach to groundwater modelling with decomposition. Hydrological Sciences Journal, 58 (1), 1–9.     

Spatio-temporal precipitation modelling in rural watersheds

Abstract

The natural variability of precipitation in agricultural regions both in time and space is modelled using extensions of Box & Jenkins (1976) methodology based on the ARMA procedure. This broad class of aggregate regional models belongs to the general family of Space-Time Autoregressive Moving Average (STARMA) processes. The paper develops a three-stage iterative procedure for building a ST ARMA model of multiple precipitation series. The identified model is STMA (1₃). The emphasis is placed on the three stages of the model building procedure, namely identification, parameter estimation and diagnostic checking. In the parameter estimation stage the polytope (or simplex) method and three further classical nonlinear optimization algorithms are used, namely two conjugate gradient methods and a quasi-Newton method. The polytope method has been adopted and the developed model performed well in describing the spatio-temporal characteristics of the multiple precipitation series. Application has been attempted in a rural watershed in southern Canada.     

Construction of a high-resolution gridded rainfall dataset for Peru from 1981 to the present day

ABSTRACT

A new gridded rainfall dataset available for Peru is introduced, called PISCOp V2.1 (Peruvian Interpolated data of SENAMHI’s Climatological and Hydrological Observations). PISCOp has been developed for the period 1981 to the present, with an average latency of eight weeks at 0.1° spatial resolution. The merging algorithm is based on geostatistical and deterministic interpolation methods including three different rainfall sources: (i) the national quality-controlled and infilled raingauge dataset, (ii) radar-gauge merged precipitation climatologies and (iii) the Climate Hazards Group Infrared Precipitation (CHIRP) estimates. The validation results suggest that precipitation estimates are acceptable showing the highest performance for the Pacific coast and the western flank of the Andes. Furthermore, a meticulous quality-control and gap-infilling procedure allowed us to reduce the formation of inhomogeneities (non-climatic breaks). The dataset is publicly available at https://piscoprec.github.io/ and is intended to support hydrological studies and water management practices.     

Comparison of data-driven modelling techniques for river flow forecasting

Abstract

Accurate forecasting of streamflow is essential for the efficient operation of water resources systems. The streamflow process is complex and highly nonlinear. Therefore, researchers try to devise alterative techniques to forecast streamflow with relative ease and reasonable accuracy, although traditional deterministic and conceptual models are available. The present work uses three data-driven techniques, namely artificial neural networks (ANN), genetic programming (GP) and model trees (MT) to forecast river flow one day in advance at two stations in the Narmada catchment of India, and the results are compared. All the models performed reasonably well as far as accuracy of prediction is concerned. It was found that the ANN and MT techniques performed almost equally well, but GP performed better than both these techniques, although only marginally in terms of prediction accuracy in normal and extreme events.

Citation Londhe, S. & Charhate, S. (2010) Comparison of data-driven modelling techniques for river flow forecasting. Hydrol. Sci. J. 55(7), 1163–1174.     

Improving kriging of groundwater level data using nonlinear normalizing transformations—a field application

Abstract

The present research study investigates the application of nonlinear normalizing data transformations in conjunction with ordinary kriging (OK) for the accurate prediction of groundwater level spatial variability in a sparsely-gauged basin. We investigate three established normalizing methods, Gaussian anamorphosis, trans-Gaussian kriging and the Box-Cox method to improve the estimation accuracy. The first two are applied for the first time to groundwater level data. All three methods improve the mean absolute prediction error compared to the application of OK to the non-transformed data. In addition, a modified Box-Cox transformation is proposed and applied to normalize the hydraulic heads. The modified Box-Cox transformation in conjunction with OK is found to be the optimal spatial model based on leave-one-out cross-validation. The recently established Spartan semivariogram family provides the optimal model fit to the transformed data. Finally, we present maps of the groundwater level and the kriging variance based on the optimal spatial model.

Editor D. Koutsoyiannis; Associate editor A. Montanari

Citation Varouchakis, E.A., Hristopoulos, D.T., and Karatzas, G.P., 2012. Improving kriging of groundwater level data using nonlinear normalizing transformations—a field application. Hydrological Sciences Journal, 57 (7), 1404–1419.     

Nonlinear response of runoff to atmospheric freezing level height variation based on hybrid prediction models

ABSTRACT

We use data on the freezing level height (FLH) and summer runoff in the Hotan River, China, from 1960 to 2013, to analyse the nonlinear relationships of atmospheric and hydrological factors at different time scales, by employing three nonlinear decomposition methods. Six hybrid prediction models are established by combining linear regression and back-propagation artificial neural network (BPANN) models. The decomposition results by three nonlinear methods are compared, indicating that the extreme-point symmetric mode decomposition (ESMD) method ensures the best prediction capacity. The runoff and FLH have periods of 3 and 6 years, respectively, at the inter-annual scale, which pass the significance test of 0.05 (P < 0.05) by using the Monte Carlo method, although there were slight differences in the periods at the inter-decadal scale. Among the six models, ESMD-BPANN exhibits the highest accuracy, with good reliability and resolution, according to several performance indicators. The ESMD-BPANN model is thus selected for the simulation and prediction of runoff.     

Evaluation of satellite-based rainfall products for hydrological modelling in Morocco

ABSTRACT

Several satellite-based precipitation estimates are becoming available at a global scale, providing new possibilities for water resources modelling, particularly in data-sparse regions and developing countries. This work provides a first validation of five different satellite-based precipitation products (TRMM-3B42 v6 and v7, RFE 2.0, PERSIANN-CDR, CMORPH1.0 version 0.x) in the 1785 km² Makhazine catchment (Morocco). Precipitation products are first compared against ground observations. Ten raingauges and four different interpolation methods (inverse distance, nearest neighbour, ordinary kriging and residual kriging with altitude) were used to compute a set of interpolated precipitation reference fields. Second, a parsimonious conceptual hydrological model is considered, with a simulation approach based on the random generation of model parameters drawn from existing parameter set libraries, to compare the different precipitation inputs. The results indicate that (1) all four interpolation methods, except the nearest neighbour approach, give similar and valid precipitation estimates at the catchment scale; (2) among the different satellite-based precipitation estimates verified, the TRMM-3B42 v7 product is the closest to observed precipitation, and (3) despite poor performance at the daily time step when used in the hydrological model, TRMM-3B42 v7 estimates are found adequate to reproduce monthly dynamics of discharge in the catchment. The results provide valuable perspectives for water resources modelling of data-scarce catchments with satellite-based rainfall data in this region.

Editor M.C. Acreman; Associate editor N. Verhoest     

Nonlinearities and Linearities in Internal Gravity Waves of the Atmosphere and Oceans

The spectrum of internal gravity waves in the atmosphere and oceans is sufficiently intense that nonlinear interactions must occur, if these waves are analyzed in Eulerian coordinates as is usually done. As it happens, however, if these waves are analyzed in Lagrangian coordinates the most important nonlinearity can be entirely avoided: it is an Eulerian mathematical construct only, not a physical process. The mathematical basis for this assertion is developed here, and some of its consequences are discussed. Among the latter is a questioning of the validity of standard Eulerian eikonal methods of calculating ray paths and related functions in a multiwave environment, discussed in an appendix.     

Group-based estimation of missing hydrological data: II. Application to streamflows

Abstract

The group approach that treats hydrological data as groups rather than as single-valued observations was proposed in a companion paper. Various models representing four techniques are briefly presented and applied to single series and bi-series cases, respectively, in this paper. The techniques represented by these models are regression, time series analysis, partitioning modelling, and artificial neural networks. The utility of the models for estimating missing streamflow data using the group approach is investigated. It turns out that the group approach is valid for estimating missing values, and possibly other applications, when data are significantly auto-correlated.     

Identifying hydrograph parameters and their relationships to urbanization variables

Abstract

This study examines relationships between model parameters and urbanization variables for evaluating urbanization effects in a watershed. Rainfall–runoff simulation using the Nash model is the main basis of the study. Mean rainfall and excesses resulting from time-variant losses were completed using the kriging and nonlinear programming methods, respectively. Calibrated parameters of 47 events were related to urbanized variables, change of shape parameter responds more sensitively than that of scale parameter based on comparisons between annual average and optimal interval methods. Regression equations were used to obtain four continuous correlations for linking shape parameter with urbanization variables. Verification of 10 events demonstrates that shape parameter responds more strongly to imperviousness than to population, and a power relationship is suitable. Therefore, an imperviousness variable is a major reference for analysing urbanization changes to a watershed. This study found that time to peak of IUH was reduced from 11.76 to 3.97 h, whereas peak discharge increased from 44.79 to 74.92 m³/s.

Editor D. Koutsoyiannis; Associate editor S. Grimaldi

Citation Huang, S.-Y., Cheng, S.-J., Wen, J.-C. and Lee, J.-H., 2012. Identifying hydrograph parameters and their relationships to urbanization variables. Hydrological Sciences Journal, 57 (1), 144–161.     

Artificial neural networks for daily rainfall—runoff modelling

Abstract

The application of artificial neural network (ANN) methodology for modelling daily flows during monsoon flood events for a large size catchment of the Narmada River in Madhya Pradesh (India) is presented. The spatial variation of rainfall is accounted for by subdividing the catchment and treating the average rainfall of each subcatchment as a parallel and separate lumped input to the model. A linear multiple-input single-output (MISO) model coupled with the ANN is shown to provide a better representation of the rainfall-runoff relationship in such large size catchments compared with linear and nonlinear MISO models. The present model provides a systematic approach for runoff estimation and represents improvement in prediction accuracy over the other models studied herein.