Unbiased estimation of probability weighted moments and partial probability weighted moments from systematic and historical flood information and their application to estimating the GEV distribution

Unbiased estimators of probability weighted moments (PWM) and partial probability weighted moments (PPWM) from systematic and historical flood information are derived. Applications are made to estimating parameters and quantiles of the generalized extreme value (GEV) distribution. The effect of lower bound censoring, which might be deliberately introduced in practice, is also considered.     

Further research on application of probability weighted moments in estimating parameters of the Pearson type three distribution

This paper contains further research results on the recently developed method for estimating parameters of P-III distribution by using probability weighted moments (PWM). The computing procedure can be largely simplified and the PWM method is appropriate and functional for various conditions of the skewness coefficient. Statistical experiments made based on the new procedure have shown that PWM estimators are almost unbiased and compare favorably with those by the conventional moment method.     

Estimation of asymptotic variances of quantiles for the generalized logistic distribution

In this study, the parameter estimations for the 3-parameter generalized logistic (GL) distribution are presented based on the methods of moments (MOM), maximum likelihood (ML), and probability weighted moments (PWM). The asymptotic variances of the MOM, ML, and PWM quantile estimators for the GL distribution are expressed as functions of the sample size, return period, and parameters. A Monte Carlo simulation was performed to verify the derived expressions for variances and covariances between parameters and to evaluate the applicability of the derived asymptotic variances of quantiles for the MOM, ML and PWM methods. The simulation results generally show good agreement with the analytical results estimated from the asymptotic variances of parameters and quantiles when the shape parameter (β) of the GL distribution is between −0.10 and 0.10 for the MOM method and between −0.25 and 0.45 for the ML and PWM methods, respectively. In addition, the actual sample variances and the root mean square error (RMSE) of asymptotic variances of quantiles for various sample sizes, return periods, and shape parameters were presented. In order to evaluate the applicability of the estimation methods to real data and to compare the values of estimated parameter, quantiles, and confidence intervals based on each parameter estimation method, the GL distribution was fitted to the 24-h annual maximum rainfall data at Pohang, Korea.     

Using regional regression within index flood procedures and an empirical Bayesian estimator

Studies have illustrated the performance of at-site and regional flood quantile estimators. For realistic generalized extreme value (GEV) distributions and short records, a simple index-flood quantile estimator performs better than two-parameter (2P) GEV quantile estimators with probability weighted moment (PWM) estimation using a regional shape parameter and at-site mean and L-coefficient of variation (L-CV), and full three-parameter at-site GEV/PWM quantile estimators. However, as regional heterogeneity or record lengths increase, the 2P-estimator quickly dominates. This paper generalizes the index flood procedure by employing regression with physiographic information to refine a normalized T-year flood estimator. A linear empirical Bayes estimator uses the normalized quantile regression estimator to define a prior distribution which is employed with the normalized 2P-quantile estimator. Monte Carlo simulations indicate that this empirical Bayes estimator does essentially as well as or better than the simpler normalized quantile regression estimator at sites with short records, and performs as well as or better than the 2P-estimator at sites with longer records or smaller L-CV.     

Estimation of confidence intervals of quantiles for the Weibull distribution

 Estimation of confidence limits and intervals for the two- and three-parameter Weibull distributions are presented based on the methods of moment (MOM), probability weighted moments (PWM), and maximum likelihood (ML). The asymptotic variances of the MOM, PWM, and ML quantile estimators are derived as a function of the sample size, return period, and parameters. Such variances can be used for estimating the confidence limits and confidence intervals of the population quantiles. Except for the two-parameter Weibull model, the formulas obtained do not have simple forms but can be evaluated numerically. Simulation experiments were performed to verify the applicability of the derived confidence intervals of quantiles. The results show that overall, the ML method for estimating the confidence limits performs better than the other two methods in terms of bias and mean square error. This is specially so for γ≥0.5 even for small sample sizes (e.g. N=10). However, the drawback of the ML method for determining the confidence limits is that it requires that the shape parameter be bigger than 2. The Weibull model based on the MOM, ML, and PWM estimation methods was applied to fit the distribution of annual 7-day low flows and 6-h maximum annual rainfall data. The results showed that the differences in the estimated quantiles based on the three methods are not large, generally are less than 10%. However, the differences between the confidence limits and confidence intervals obtained by the three estimation methods may be more significant. For instance, for the 7-day low flows the ratio between the estimated confidence interval to the estimated quantile based on ML is about 17% for T≥2 while it is about 30% for estimation based on MOM and PWM methods. In addition, the analysis of the rainfall data using the three-parameter Weibull showed that while ML parameters can be estimated, the corresponding confidence limits and intervals could not be found because the shape parameter was smaller than 2.     

Alternative PWM-estimators of the Gumbel distribution

Probability weighted moments (PWM) are widely used in hydrology for estimating parameters of statistical distributions, including the Gumbel distribution. The classical PWM-approach considers the moments β_i=E[XFⁱ] with i=0,1 for estimation of the Gumbel scale and location parameters. However, there is no reason why these probability weights (F⁰ and F¹) should provide the most efficient PWM-estimators of Gumbel parameters and quantiles. We explore an extended class of PWMs that does not impose arbitrary restrictions on the values of i. Estimation based on the extended class of PWMs is called the generalized method of probability weighted moments (GPWM) to distinguish it from the classical procedure. In fact, our investigation demonstrates that it may be advantage to use weight functions that are not of the form Fⁱ. We propose an alternative PWM-estimator of the Gumbel distribution that maintains the computational simplicity of the classical PWM method, but provides slightly more accurate quantile estimates in terms of mean square error of estimation. A simple empirical formula for the standard error of the proposed quantile estimator is presented.     

A review of methods of parameter estimation for the extreme value type-1 distribution

Four important methods of estimating the parameters of the extreme value type-1 (Gumbel) distribution, namely: (1) moments (MMM); (2) maximum likelihood (MML); (3) maximum entropy (MME); and (4) probability weighted moments (PWM), are considered, suitable solution procedures are recommended and related problems discussed. Approximate formulae for computing the variances-covariances of the estimators by each method are given in terms of the population value of the scale parameter, and these are shown to indicate very well the performance of the related method.

A simulation study to evaluate the performance of these methods in terms of commonly used criteria, i.e. the bias, root mean square error and goodness-of-fit statistic is described. It was found that the MMM is not as good as the remaining three methods, of which the PWM is best in terms of the bias, and the MML is best in terms of the root mean square error and efficiency. The MME, by all the criteria, ranks second best and follows the MML more closely than the PWM.     

Frequency analyses of annual extreme rainfall series from 5 min to 24 h

The parameter estimation methods of (1) moments, (2) maximum‐likelihood, (3) probability‐weighted moments (PWM) and (4) self‐determined PWM are applied to the probability distributions of Gumbel, general extreme values, three‐parameter log‐normal (LN3), Pearson‐3 and log‐Pearson‐3. The special method of computing parameters so as to make the sample skewness coefficient zero is also applied to LN3, and hence, altogether 21 candidate distributions resulted. The parameters of these distributions are computed first by original sample series of 14 successive‐duration annual extreme rainfalls recorded at a rain‐gauging station. Next, the parameters are scaled by first‐degree semi‐log or log‐log polynomial regressions versus rainfall durations from 5 to 1440 min (24 h). Those distributions satisfying the divergence criterion for frequency curves are selected as potential distributions, whose better‐fit ones are determined by a conjunctive evaluation of three goodness‐of‐fit tests. Frequency tables, frequency curves and intensity–duration–frequency curves are the outcome. Copyright © 2010 John Wiley & Sons, Ltd.     

Emulation of simulated earthquake catalogues

In earthquake occurrence studies, the so-called q value can be considered both as one of the parameters describing the distribution of interevent times and as an index of non-extensivity. Using simulated datasets, we compare four kinds of estimators, based on principle of maximum entropy (POME), method of moments (MOM), maximum likelihood (MLE), and probability weighted moments (PWM) of the parameters (q and τ ₀) of the distribution of inter-events times, assumed to be a generalized Pareto distribution (GPD), as defined by Tsallis (1988) in the frame of non-extensive statistical physics. We then propose to use the unbiased version of PWM estimators to compute the q value for the distribution of inter-event times in a realistic earthquake catalogue simulated according to the epidemic type aftershock sequence (ETAS) model. Finally, we use these findings to build a statistical emulator of the q values of ETAS model. We employ treed Gaussian processes to obtain partitions of the parameter space so that the resulting model respects sharp changes in physical behaviour. The emulator is used to understand the joint effects of input parameters on the q value, exploring the relationship between ETAS model formulation and distribution of inter-event times.     

Parameter estimation for 2-parameter generalized pareto distribution by POME

The principle of maximum entropy (POME) was employed to derive a new method of parameter estimation for the 2-parameter generalized Pareto (GP2) distribution. Monte Carlo simulated data were used to evaluate this method and compare it with the methods of moments (MOM), probability weighted moments (PWM), and maximum likelihood estimation (MLE). The parameter estimates yielded by POME were comparable or better within certain ranges of sample size and coefficient of variation.     

Estimation of the GEV distribution from censored samples by method of partial probability weighted moments

The concept of partial probability weighted moments (PPWM), which can be used to estimate a distribution from censored samples, is introduced. Unbiased estimators of PPWM are derived. An application is made to estimating parameters and quantiles of the generalized extreme value (GEV) distribution from censored samples. Censored samples yield high quantile estimates which are almost as efficient as those obtained from uncensored samples. This could be a very useful technique for dealing with the undesirable effects of low outliers which occur in semiarid and arid zones.     

On the use of partial probability weighted moments in the analysis of hydrological extremes

The use of partial probability weighted moments (PPWM) for estimating hydrological extremes is compared to that of probability weighted moments (PWM). Firstly, estimates from at‐site data are considered. Two Monte Carlo analyses, conducted using continuous and empirical parent distributions (of peak discharge and daily rainfall annual maxima) and applying four different distributions (Gumbel, Fréchet, GEV and generalized Pareto), show that the estimates obtained from PPWMs are better than those obtained from PWMs if the parent distribution is unknown, as happens in practice. Secondly, the use of partial L‐moments (obtained from PPWMs) as diagnostic tools is considered. The theoretical partial L‐diagrams are compared with the experimental data. Five different distributions (exponential, Pareto, Gumbel, GEV and generalized Pareto) and 297 samples of peak discharge annual maxima are considered. Finally, the use of PPWMs with regional data is investigated. Three different kinds of regional analyses are considered. The first kind is the regression of quantile estimates on basin area. The study is conducted applying the GEV distribution to peak discharge annual maxima. The regressions obtained with PPWMs are slightly better than those obtained with PWMs. The second kind of regional analysis is the parametric one, of which four different models are considered. The congruence between local and regional estimates is examined, using peak discharge annual maxima. The congruence degree is sometimes higher for PPWMs, sometimes for PWMs. The third kind of regional analysis uses the index flood method. The study, conducted applying the GEV distribution to synthetic data from a lognormal joint distribution, shows that better estimates are obtained sometimes from PPWMs, sometimes from PWMs. All the results seem to indicate that using PPWMs can constitute a valid tool, provided that the influence of ouliers, of course higher with censored samples, is kept under control. Copyright © 2007 John Wiley & Sons, Ltd.     

Unbiased estimation of flood risk with the GEV distribution

Conventional flood frequency analysis is concerned with providing an unbiased estimate of the magnitude of the design flow exceeded with the probabilityp, but sampling uncertainties imply that such estimates will, on average, be exceeded more frequently. An alternative approach is therefore, to derive an estimator which gives an unbiased estimate of flow risk: the difference between the two magnitudes reflects uncertainties in parameter estimation. An empirical procedure has been developed to estimate the mean true exceedance probabilities of conventional estimates made using a GEV distribution fitted by probability weighted moments, and adjustment factors have been determined to enable the estimation of flood magnitudes exceeded with, on average, the desired probability.     

Entropy-based parameter estimation for extended Burr XII distribution

Two entropy-based methods, called ordinary entropy (ENT) method and parameter space expansion method (PSEM), both based on the principle of maximum entropy, are applied for estimating parameters of the extended Burr XII distribution. With the parameters so estimated, the Burr XII distribution is applied to six peak flow datasets and quantiles (discharges) corresponding to different return periods are computed. These two entropy methods are compared with the methods of moments (MOM), probability weighted moments (PWM) and maximum likelihood estimation (MLE). It is shown that PSEM yields the same quantiles as does MLE for discrete cases, while ENT is found comparable to the MOM and PWM. For shorter return periods (<10–30 years), quantiles (discharges) estimated by these four methods are in close agreement, but the differences amongst them grow as the return period increases. The error in quantiles computed using the four methods becomes larger for return periods greater than 10–30 years.     

On the geostatistical approach to the inverse problem

The geostatistical approach to the inverse problem is discussed with emphasis on the importance of structural analysis. Although the geostatistical approach is occasionally misconstrued as mere cokriging, in fact it consists of two steps: estimation of statistical parameters (“structural analysis”) followed by estimation of the distributed parameter conditional on the observations (“cokriging” or “weighted least squares”). It is argued that in inverse problems, which are algebraically undetermined, the challenge is not so much to reproduce the data as to select an algorithm with the prospect of giving good estimates where there are no observations. The essence of the geostatistical approach is that instead of adjusting a grid-dependent and potentially large number of block conductivities (or other distributed parameters), a small number of structural parameters are fitted to the data. Once this fitting is accomplished, the estimation of block conductivities ensues in a predetermined fashion without fitting of additional parameters. Also, the methodology is compared with a straightforward maximum a posteriori probability estimation method. It is shown that the fundamental differences between the two approaches are: (a) they use different principles to separate the estimation of covariance parameters from the estimation of the spatial variable; (b) the method for covariance parameter estimation in the geostatistical approach produces statistically unbiased estimates of the parameters that are not strongly dependent on the discretization, while the other method is biased and its bias becomes worse by refining the discretization into zones with different conductivity.     

Single station and regional analysis of daily rainfall extremes

A peaks over threshold (POT) method of analysing daily rainfall values is developed using a Poisson process of occurrences and a generalised Pareto distribution (GPD) for the exceedances. The parameters of the GPD are estimated by the method of probability weighted moments (PWM) and a method of combining the individual estimates to define a regional curve is proposed.     

Appraisal of regional and index flood quantile estimators

Our results illustrate the performance of at-site and regional GEV/PWM flood quantile estimators in regions with different coefficients of variation, degrees of regional heterogeneity, record lengths, and number of sites. Analytic approximations of bias and variance are employed. For realistic GEV distributions and short records, the index-flood quantile estimator performs better than a 2-parameter GEV/PWM quantile estimator with a regional shape parameter, or a 3-parameter at-site GEV/PWM quantile estimator, in both humid and especially in arid regions, as long as the degree of regional heterogeneity is moderate. As regional heterogeneity or record lengths increases, 2-parameter estimators quickly dominate. Flood frequency models that assign probabilities larger than 2% to negative flows are unrealistic; experiments employing such distributions provide questionable results. This appraisal generally demonstrates the value of regionalizing estimators of the shape of a flood distribution, and sometimes the coefficient of variation.     

Bias compensation in flood frequency analysis

Abstract

Flood frequency analysis (FFA) is essential for water resources management. Long flow records improve the precision of estimated quantiles; however, in some cases, sample size in one location is not sufficient to achieve a reliable estimate of the statistical parameters and thus, regional FFA is commonly used to decrease the uncertainty in the prediction. In this paper, the bias of several commonly used parameter estimators, including L-moment, probability weighted moment and maximum likelihood estimation, applied to the general extreme value (GEV) distribution is evaluated using a Monte Carlo simulation. Two bias compensation approaches: compensation based on the shape parameter, and compensation using three GEV parameters, are proposed based on the analysis and the models are then applied to streamflow records in southern Alberta. Compensation efficiency varies among estimators and between compensation approaches. The results overall suggest that compensation of the bias due to the estimator and short sample size would significantly improve the accuracy of the quantile estimation. In addition, at-site FFA is able to provide reliable estimation based on short data, when accounting for the bias in the estimator appropriately.

Editor D. Koutsoyiannis; Associate editor Sheng Yue     

Appraisal of regional and index flood quantile estimators

Our results illustrate the performance of at-site and regional GEV/PWM flood quantile estimators in regions with different coefficients of variation, degrees of regional heterogeneity, record lengths, and number of sites. Analytic approximations of bias and variance are employed. For realistic GEV distributions and short records, the index-flood quantile estimator performs better than a 2-parameter GEV/PWM quantile estimator with a regional shape parameter, or a 3-parameter at-site GEV/PWM quantile estimator, in both humid and especially in arid regions, as long as the degree of regional heterogeneity is moderate. As regional heterogeneity or record lengths increases, 2-parameter estimators quickly dominate. Flood frequency models that assign probabilities larger than 2% to negative flows are unrealistic; experiments employing such distributions provide questionable results. This appraisal generally demonstrates the value of regionalizing estimators of the shape of a flood distribution, and sometimes the coefficient of variation.     

Parameter and quantile estimation of the 2-parameter kappa distribution by maximum likelihood

Asymptotic properties of maximum likelihood parameter and quantile estimators of the 2-parameter kappa distribution are studied. Eight methods for obtaining large sample confidence intervals for the shape parameter and for quantiles of this distribution are proposed and compared by using Monte Carlo simulation. The best method is highlighted on the basis of the coverage probability of the confidence intervals that it produces for sample sizes commonly found in practice. For such sample sizes, confidence intervals for quantiles and for the shape parameter are shown to be more accurate if the quantile estimators are assumed to be log normally distributed rather than normally distributed (same for the shape parameter estimator). Also, confidence intervals based on the observed Fisher information matrix perform slightly better than those based on the expected value of this matrix. A hydrological example is provided in which the obtained theoretical results are applied.