On the informative value of the largest sample element of log-Gumbel distribution

Extremes of stream flow and precipitation are commonly modeled by heavytailed distributions. While scrutinizing annual flow maxima or the peaks over threshold, the largest sample elements are quite often suspected to be low quality data, outliers or values corresponding to much longer return periods than the observation period. Since the interest is primarily in the estimation of the right tail (in the case of floods or heavy rainfalls), sensitivity of upper quantiles to largest elements of a series constitutes a problem of special concern. This study investigated the sensitivity problem using the log-Gumbel distribution by generating samples of different sizes (n) and different values of the coefficient of variation by Monte Carlo experiments. Parameters of the log-Gumbel distribution were estimated by the probability weighted moments (PWMs) method, method of moments (MOMs) and maximum likelihood method (MLM), both for complete samples and the samples deprived of their largest elements. In the latter case, the distribution censored by the non-exceedance probability threshold, F _T , was considered. Using F _T instead of the censored threshold T creates possibility of controlling estimator property. The effect of the F _T value on the performance of the quantile estimates was then examined. It is shown that right censoring of data need not reduce an accuracy of large quantile estimates if the method of PWMs or MOMs is employed. Moreover allowing bias of estimates one can get the gain in variance and in mean square error of large quantiles even if ML method is used.     

On robustness of large quantile estimates of log-Gumbel and log-logistic distributions to largest element of the observation series: Monte Carlo results vs. first order approximation.

Robustness of large quantile estimates to the largest element in a sample of methods of moments (MOM) and L-moments (LMM) was evaluated and compared. Quantiles were estimated by log-logistic and log-Gumbel distributions. Both are lower bounded and two-parameter distributions, with the coefficient of variation (CV) serving as the shape parameter. In addition, the results of these two methods were compared with those of the maximum likelihood method (MLM). Since identification and elimination of the outliers in a single sample require the knowledge of the samples parent distribution which is unknown, one estimates it by using the parameter estimation method which is relatively robust to the largest element in the sample. In practice this means that the method should be robust to extreme elements (including outliers) in a sample.The effect of dropping the largest element of the series on the large quantile values was assessed for various coefficient of variation (CV) / sample size (N) combinations. To that end, Monte-Carlo sampling experiments were applied. The results were compared with those obtained from the single representative sample, (the first order approximation), i.e., consisting of both the average values (Ex_i) for every position (i) of an ordered sample and the theoretical quantiles based on the plotting formula (PP).The ML-estimates of large quantiles were found to be most robust to largest element of samples except for a small sample where MOM-estimates were more robust. Comparing the performance of two other methods in respect to the large quantiles estimation, MOM was found to be more robust for small and moderate samples drawn from distributions with zero lower bound as shown for log-Gumbel and log-logistic distributions. The results from representative samples were fairly compatible with the M-C simulation results. The Ex-sample results were closer to the M-C results for smaller CV-values, and to the PP-sample results for greater CV values.     

A note on the applicability of log-Gumbel and log-logistic probability distributions in hydrological analyses: II. Assumed pdf

Abstract

Applicability of log-Gumbel (LG) and log-logistic (LL) probability distributions in hydrological studies is critically examined under real conditions, where the assumed distribution differs from the true one. The set of alternative distributions consists of five two-parameter distributions with zero lower bound, including LG and LL as well as lognormal (LN), linear diffusion analogy (LD) and gamma (Ga) distributions. The log-Gumbel distribution is considered as both a false and a true distribution. The model error of upper quantiles and of the first two moments is analytically derived for three estimation methods: the method of moments (MOM), the linear moments method (LMM) and the maximum likelihood method (MLM). These estimation methods are used as methods of approximation of one distribution by another distribution. As recommended in the first of this two-part series of papers, MLM turns out to be the worst method, if the assumed LG or LL distribution is not the true one. It produces a huge bias of upper quantiles, which is at least one order higher than that of the other two methods. However, the reverse case, i.e. acceptance of LN, LD or Ga as a hypothetical distribution, while the LG or LL distribution is the true one, gives the MLM bias of reasonable magnitude in upper quantiles. Therefore, one should avoid choosing the LG and LL distributions in flood frequency analysis, especially if MLM is to be applied.     

The PWM large quantile estimates of heavy tailed distributions from samples deprived of their largest element / Estimation des grands quantiles de distributions à queue à décroissance lente par la méthode des moments pondérés par les probabilités à partir d'échantillons amputés de leur plus grande valeur

Abstract

Extremes of streamflow are usually modelled using heavy tailed distributions. While scrutinising annual flow maxima or the peaks over threshold, the largest elements in a sample are often suspected to be low quality data, outliers or values corresponding to much longer return periods than the observation period. In the case of floods, since the interest is focused mainly on the estimation of the right-hand tail of a distribution function, sensitivity of large quantiles to extreme elements of a series becomes the problem of special concern. This study investigated the sensitivity problem using the log-Gumbel distribution by generating samples of different sizes and different values of the coefficient of L-variation by means of Monte Carlo experiments. Parameters of the log-Gumbel distribution were estimated by the probability weighted moments (PWM) method, both for complete samples and the samples deprived of their largest element. In the latter case Hosking's concept of the “A” type PWM with Type II censoring was employed. The largest value was censored above the random threshold T corresponding to the non-exceedence probability F _T. The effect of the F _T value on the performance of the quantile estimates was then examined. Experimental results show that omission of the largest sample element need not result in a decrease in the accuracy of large quantile estimates obtained from the log-Gumbel model by the PWM method.     

Estimation of confidence intervals of quantiles for the Weibull distribution

 Estimation of confidence limits and intervals for the two- and three-parameter Weibull distributions are presented based on the methods of moment (MOM), probability weighted moments (PWM), and maximum likelihood (ML). The asymptotic variances of the MOM, PWM, and ML quantile estimators are derived as a function of the sample size, return period, and parameters. Such variances can be used for estimating the confidence limits and confidence intervals of the population quantiles. Except for the two-parameter Weibull model, the formulas obtained do not have simple forms but can be evaluated numerically. Simulation experiments were performed to verify the applicability of the derived confidence intervals of quantiles. The results show that overall, the ML method for estimating the confidence limits performs better than the other two methods in terms of bias and mean square error. This is specially so for γ≥0.5 even for small sample sizes (e.g. N=10). However, the drawback of the ML method for determining the confidence limits is that it requires that the shape parameter be bigger than 2. The Weibull model based on the MOM, ML, and PWM estimation methods was applied to fit the distribution of annual 7-day low flows and 6-h maximum annual rainfall data. The results showed that the differences in the estimated quantiles based on the three methods are not large, generally are less than 10%. However, the differences between the confidence limits and confidence intervals obtained by the three estimation methods may be more significant. For instance, for the 7-day low flows the ratio between the estimated confidence interval to the estimated quantile based on ML is about 17% for T≥2 while it is about 30% for estimation based on MOM and PWM methods. In addition, the analysis of the rainfall data using the three-parameter Weibull showed that while ML parameters can be estimated, the corresponding confidence limits and intervals could not be found because the shape parameter was smaller than 2.     

Estimation of asymptotic variances of quantiles for the generalized logistic distribution

In this study, the parameter estimations for the 3-parameter generalized logistic (GL) distribution are presented based on the methods of moments (MOM), maximum likelihood (ML), and probability weighted moments (PWM). The asymptotic variances of the MOM, ML, and PWM quantile estimators for the GL distribution are expressed as functions of the sample size, return period, and parameters. A Monte Carlo simulation was performed to verify the derived expressions for variances and covariances between parameters and to evaluate the applicability of the derived asymptotic variances of quantiles for the MOM, ML and PWM methods. The simulation results generally show good agreement with the analytical results estimated from the asymptotic variances of parameters and quantiles when the shape parameter (β) of the GL distribution is between −0.10 and 0.10 for the MOM method and between −0.25 and 0.45 for the ML and PWM methods, respectively. In addition, the actual sample variances and the root mean square error (RMSE) of asymptotic variances of quantiles for various sample sizes, return periods, and shape parameters were presented. In order to evaluate the applicability of the estimation methods to real data and to compare the values of estimated parameter, quantiles, and confidence intervals based on each parameter estimation method, the GL distribution was fitted to the 24-h annual maximum rainfall data at Pohang, Korea.     

Impulse response of the kinematic diffusion model as a probability distribution of hydrologic samples with zero values

It is hypothesized that the unit impulse response of a linearized kinematic diffusion (KD) model is a probability distribution suitable for frequency analysis of hydrologic samples with zero values. Such samples may include data on monthly precipitation in dry seasons, annual low flow, and annual maximum peak discharge observed in arid and semiarid regions. The hypothesized probability distribution has two parameters, which are estimated using the methods of moments (MOM) and maximum likelihood (MLM). Also estimated are errors in quantiles for MOM and MLM. The distribution shows an equivalency of MOM and MLM with respect to the mean value—an important property for ML-estimation in the case of the unknown true distribution function. The hypothesized KD distribution is tested on 44 discharge data series and compared with the Muskingum-like (M-like) probability distribution function. A comparison of empirical distribution with KD and M-like distributions shows that MOM better reproduces the upper tail of the distribution, while MLM is more robust for higher sample values and more conditioned on the value of the probability of the zero value event. The KD-model is suitable for frequency analysis of short samples with zero values and it is more universal than the M-like model as its modal value cannot be only equaled to zero value but also to any positive value.     

Estimation of quantiles and confidence intervals for the log-Gumbel distribution

The log-Gumbel distribution is one of the extreme value distributions which has been widely used in flood frequency analysis. This distribution has been examined in this paper regarding quantile estimation and confidence intervals of quantiles. Specific estimation algorithms based on the methods of moments (MOM), probability weighted moments (PWM) and maximum likelihood (ML) are presented. The applicability of the estimation procedures and comparison among the methods have been illustrated based on an application example considering the flood data of the St. Mary's River.     

On seasonal approach to flood frequency modelling. Part II: flood frequency analysis of Polish rivers

The annual peak flow series of Polish rivers are mixtures of summer and winter flows. As Part II of a sequence of two papers, practical aspects of applicability of seasonal approach to flood frequency analysis (FFA) of Polish rivers are discussed. Taking A Two‐Component Extreme Value (TCEV1) model as an example it was shown in the first part that regardless of estimation method, the seasonal approach can give profit in terms of upper quantile estimation accuracy that rises with the return period of the quantile and is the greatest for no seasonal variation. In this part, an assessment of annual maxima (AM) versus seasonal maxima (SM) approach to FFA was carried out with respect to seasonal and annual peak flow series of 38 Polish gauging stations. First, the assumption of mutual independence of the seasonal maxima has been tested. The smoothness of SM and AM empirical probability distribution functions was analysed and compared. The TCEV1 model with seasonally estimated parameters was found to be not appropriate for most Polish data as it considerably underrates the skewness of AM distributions and upper quantile values as well. Consequently, the discrepancies between the SM and AM estimates of TCEV1 are observed. Taking SM and TCEV1 distribution, the dominating season in AM series was confronted with predominant season for extreme floods. The key argument for presumptive superiority of SM approach that SM samples are more statistically homogeneous than AM samples has not been confirmed by the data. An analysis of fitness to SM and AM of Polish datasets made for seven distributions pointed to Pearson (3) distribution as the best for AM and Summer Maxima, whereas it was impossible to select a single best model for winter samples. In the multi‐model approach to FFA, the tree functions, i.e., Pe(3), CD3 and LN3, should be involved for both SM and AM. As the case study, Warsaw gauge on the Vistula River was selected. While most of AM elements are here from winter season, the prevailing majority of extreme annual floods are the summer maxima. The upper quantile estimates got by means of classical annual and two‐season methods happen to be fairly close; what's more they are nearly equal to the quantiles calculated just for the season of dominating extreme floods. Copyright © 2011 John Wiley & Sons, Ltd.     

On seasonal approach to flood frequency modelling. Part I: Two‐component distribution revisited

The annual peak flow series of the Polish rivers are mixtures of summer and winter flows. In the Part I of a sequence of two papers, theoretical aspects of applicability of seasonal approach to flood frequency analysis (FFA) in Poland are discussed. A testing procedure is introduced for the seasonal model and the data overall fitness. Conditions for objective comparative assessment of accuracy of annual maxima (AM) and seasonal maxima (SM) approaches to FFA are formulated and finally Gumbel (EV1) distribution is chosen as seasonal distribution for detailed investigation. Sampling properties of AM quantile x(F) estimates are analysed and compared for the SM and AM models for equal seasonal variances. For this purpose, four estimation methods were used, employing both asymptotic approach and sampling experiments. Superiority of the SM over AM approach is stated evident in the upper quantile range, particularly for the case of no seasonal variation in the parameters of Gumbel distribution. In order to learn whether the standard two‐ and three‐parameter flood frequency distributions can be used to model the samples generated from the Two‐Component Extreme Value 1 (TCEV1) distribution, the shape of TCEV1 probability density function (PDF) has been tested in terms of bi‐modality. Then the use of upper quantile estimate obtained from the dominant season of extreme floods (DEFS) as AM upper quantile estimate is studied and respective systematic error is assessed. The second part of the paper deals with advantages and disadvantages of SM and AM approach when applied to real flow data of Polish rivers. Copyright © 2011 John Wiley & Sons, Ltd.     

Non-crossing nonlinear regression quantiles by monotone composite quantile regression neural network,with application to rainfall extremes

The goal of quantile regression is to estimate conditional quantiles for specified values of quantile probability using linear or nonlinear regression equations. These estimates are prone to “quantile crossing”, where regression predictions for different quantile probabilities do not increase as probability increases. In the context of the environmental sciences, this could, for example, lead to estimates of the magnitude of a 10-year return period rainstorm that exceed the 20-year storm, or similar nonphysical results. This problem, as well as the potential for overfitting, is exacerbated for small to moderate sample sizes and for nonlinear quantile regression models. As a remedy, this study introduces a novel nonlinear quantile regression model, the monotone composite quantile regression neural network (MCQRNN), that (1) simultaneously estimates multiple non-crossing, nonlinear conditional quantile functions; (2) allows for optional monotonicity, positivity/non-negativity, and generalized additive model constraints; and (3) can be adapted to estimate standard least-squares regression and non-crossing expectile regression functions. First, the MCQRNN model is evaluated on synthetic data from multiple functions and error distributions using Monte Carlo simulations. MCQRNN outperforms the benchmark models, especially for non-normal error distributions. Next, the MCQRNN model is applied to real-world climate data by estimating rainfall Intensity–Duration–Frequency (IDF) curves at locations in Canada. IDF curves summarize the relationship between the intensity and occurrence frequency of extreme rainfall over storm durations ranging from minutes to a day. Because annual maximum rainfall intensity is a non-negative quantity that should increase monotonically as the occurrence frequency and storm duration decrease, monotonicity and non-negativity constraints are key constraints in IDF curve estimation. In comparison to standard QRNN models, the ability of the MCQRNN model to incorporate these constraints, in addition to non-crossing, leads to more robust and realistic estimates of extreme rainfall.     

Flood frequency analysis of Turkey using L‐moments method

The index flood procedure coupled with the L‐moments method is applied to the annual flood peaks data taken at all stream‐gauging stations in Turkey having at least 15‐year‐long records. First, screening of the data is done based on the discordancy measure (D_i) in terms of the L‐moments. Homogeneity of the total geographical area of Turkey is tested using the L‐moments based heterogeneity measure, H, computed on 500 simulations generated using the four parameter Kappa distribution. The L‐moments analysis of the recorded annual flood peaks data at 543 gauged sites indicates that Turkey as a whole is hydrologically heterogeneous, and 45 of 543 gauged sites are discordant which are discarded from further analyses. The catchment areas of these 543 sites vary from 9·9 to 75121 km² and their mean annual peak floods vary from 1·72 to 3739·5 m³ s^?1. The probability distributions used in the analyses, whose parameters are computed by the L‐moments method are the general extreme values (GEV), generalized logistic (GLO), generalized normal (GNO), Pearson type III (PE3), generalized Pareto (GPA), and five‐parameter Wakeby (WAK). Based on the L‐moment ratio diagrams and the |Z_dist|‐statistic criteria, the GEV distribution is identified as the robust distribution for the study area (498 gauged sites). Hence, for estimation of flood magnitudes of various return periods in Turkey, a regional flood frequency relationship is developed using the GEV distribution. Next, the quantiles computed at all of 543 gauged sites by the GEV and the Wakeby distributions are compared with the observed values of the same probability based on two criteria, mean absolute relative error and determination coefficient. Results of these comparisons indicate that both distributions of GEV and Wakeby, whose parameters are computed by the L‐moments method, are adequate in predicting quantile estimates. Copyright © 2011 John Wiley & Sons, Ltd.     

Selection of the best fit flood frequency distribution and parameter estimation procedure: a case study for Tasmania in Australia

Selection of a flood frequency distribution and associated parameter estimation procedure is an important step in flood frequency analysis. This is however a difficult task due to problems in selecting the best fit distribution from a large number of candidate distributions and parameter estimation procedures available in the literature. This paper presents a case study with flood data from Tasmania in Australia, which examines four model selection criteria: Akaike Information Criterion (AIC), Akaike Information Criterion—second order variant (AIC_c), Bayesian Information Criterion (BIC) and a modified Anderson–Darling Criterion (ADC). It has been found from the Monte Carlo simulation that ADC is more successful in recognizing the parent distribution correctly than the AIC and BIC when the parent is a three-parameter distribution. On the other hand, AIC and BIC are better in recognizing the parent distribution correctly when the parent is a two-parameter distribution. From the seven different probability distributions examined for Tasmania, it has been found that two-parameter distributions are preferable to three-parameter ones for Tasmania, with Log Normal appears to be the best selection. The paper also evaluates three most widely used parameter estimation procedures for the Log Normal distribution: method of moments (MOM), method of maximum likelihood (MLE) and Bayesian Markov Chain Monte Carlo method (BAY). It has been found that the BAY procedure provides better parameter estimates for the Log Normal distribution, which results in flood quantile estimates with smaller bias and standard error as compared to the MOM and MLE. The findings from this study would be useful in flood frequency analyses in other Australian states and other countries in particular, when selecting an appropriate probability distribution from a number of alternatives.     

Parameter estimation for 2-parameter generalized pareto distribution by POME

The principle of maximum entropy (POME) was employed to derive a new method of parameter estimation for the 2-parameter generalized Pareto (GP2) distribution. Monte Carlo simulated data were used to evaluate this method and compare it with the methods of moments (MOM), probability weighted moments (PWM), and maximum likelihood estimation (MLE). The parameter estimates yielded by POME were comparable or better within certain ranges of sample size and coefficient of variation.     

Hydraulic model calibration for extreme floods in bedrock‐confined channels: case study from northern Thailand

Palaeoflood reconstructions based on stage evidence are typically conducted in data‐poor field settings. Few opportunities exist to calibrate the hydraulic models used to estimate discharge from this evidence. Consequently, an important hydraulic model parameter, the roughness coefficient (e.g. Manning's n), is typically estimated by a range of approximate techniques, such as ‘visual estimation’ and semi‐empirical equations. These techniques contribute uncertainty to resulting discharge estimates, especially where the study reach exhibits sensitivity in the discharge–Manning's n relation. We study this uncertainty within a hydraulic model for a large flood of known discharge on the Mae Chaem River, northern Thailand. Comparison of the ‘calibrated’ Manning's n with that obtained from semi‐empirical equations indicates that these underestimate roughness. Substantial roughness elements in the extra‐channel zone, inundated during large events, contribute significant additional sources of flow resistance that are captured neither by the semi‐empirical equations, nor by existing models predicting stage–roughness variations. This bedrock channel exhibits a complex discharge–Manning's n relation, and reliable estimates of the former are dependent upon realistic assignment of the latter. Our study demonstrates that a large recent flood can provide a valuable opportunity to constrain this parameter, and this is illustrated when we model a palaeoflood event in the same reach, and subsequently examine the magnitude–return period consequences of discharge uncertainty within a flood frequency analysis, which contributes its own source of uncertainty. Copyright © 2005 John Wiley & Sons, Ltd.     

Complementary aspects of linear flood routing modelling and flood frequency analysis

Similarity and differences between linear flood routing modelling (LFRM) and flood frequency analysis (FFA) techniques are presented. The moment matching used in LFRM to approximate the impulse response function (IRF) was applied in FFA to derive the asymptotic bias caused by the false distribution assumption. Proceeding in this way, other estimation methods were used as approximation methods in FFA to derive the asymptotic bias. Using simulation experiments, the above investigation was extended to evaluate the sampling bias. As a feedback, the maximum likelihood method (MLM) can be used for approximating linear channel response (LCR) by the IRFs of conceptual models. Impulse responses of the convective diffusion and kinematic diffusion models were applied and developed as FFA models. Based on kinematic diffusion LFRM, the equivalence of estimation problems of discrete‐continuous distribution and single‐censored sample are shown both for the method of moments (MOM) and the MLM. Hence, the applicability of MOM is extended for the case of censored samples. Owing to the complexity and non‐linearity of hydrological systems and resulting processes, the use of simple models is often questionable. The rationale of simple models is discussed. The problems of model choice and overparameterization are common in mathematical modelling and FF modelling. Some results for the use of simple models in the stationary FFA are presented. The problems of model discrimination are then discussed. Finally, a conjunction of linear stochastic processes and LFRM is presented. The influence of river courses on stochastic properties of the runoff process is shown by combining Gaussian input with the LCR of the simplified Saint Venant model. It is shown that, from the classification of the ways of their development, both LFRM and FFA can benefit. Copyright © 2006 John Wiley & Sons, Ltd.     

Assessment of regional estimation error by canonical residual kriging

Various regional flood frequency analysis procedures are used in hydrology to estimate hydrological variables at ungauged or partially gauged sites. Relatively few studies have been conducted to evaluate the accuracy of these procedures and estimate the error induced in regional flood frequency estimation models. The objective of this paper is to assess the overall error induced in the residual kriging (RK) regional flood frequency estimation model. The two main error sources in specific flood quantile estimation using RK are the error induced in the quantiles local estimation procedure and the error resulting from the regional quantile estimation process. Therefore, for an overall error assessment, the corresponding errors associated with these two steps must be quantified. Results show that the main source of error in RK is the error induced into the regional quantile estimation method. Results also indicate that the accuracy of the regional estimates increases with decreasing return periods. Copyright © 2010 John Wiley & Sons, Ltd.     

Robust flood statistics: comparison of peak over threshold approaches based on monthly maxima and TL-moments

ABSTRACT

Flood quantile estimation based on partial duration series (peak over threshold, POT) represents a noteworthy alternative to the classical annual maximum approach since it enlarges the available information spectrum. Here the POT approach is discussed with reference to its benefits in increasing the robustness of flood quantile estimations. The classical POT approach is based on a Poisson distribution for the annual number of exceedences, although this can be questionable in some cases. Therefore, the Poisson distribution is compared with two other distributions (binomial and Gumbel-Schelling). The results show that only rarely is there a difference from the Poisson distribution. In the second part we investigate the robustness of flood quantiles derived from different approaches in the sense of their temporal stability against the occurrence of extreme events. Besides the classical approach using annual maxima series (AMS) with the generalized extreme value distribution and different parameter estimation methods, two different applications of POT are tested. Both are based on monthly maxima above a threshold, but one also uses trimmed L-moments (TL-moments). It is shown how quantile estimations based on this “robust” POT approach (rPOT) become more robust than AMS-based methods, even in the case of occasional extraordinary extreme events.

Editor M.C. Acreman Associate editor A. Viglione     

Probability of correct selection from lognormal and convective diffusion models based on the likelihood ratio

The objective of the paper is to show that the use of a discrimination procedure for selecting a flood frequency model without the knowledge of its performance for the considered underlying distributions may lead to erroneous conclusions. The problem considered is one of choosing between lognormal (LN) and convective diffusion (CD) distributions for a given random sample of flood observations. The probability density functions of these distributions are similarly shaped in the range of the main probability mass and the discrepancies grow with the increase in the value of the coefficient of variation (C _V ). This problem was addressed using the likelihood ratio (LR) procedure. Simulation experiments were performed to determine the probability of correct selection (PCS) for the LR method. Pseudo-random samples were generated for several combinations of sample sizes and the coefficient of variation values from each of the two distributions. Surprisingly, the PCS of the LN model was twice smaller than that of the CD model, rarely exceeding 50%. The results obtained from simulation were analyzed and compared both with those obtained using real data and with the results obtained from another selection procedure known as the QK method. The results from the QK are just the opposite to that of the LR procedure.