Evaluating model performance: towards a non-parametric variant of the Kling-Gupta efficiency

ABSTRACT

Goodness-of-fit measures are important for an objective evaluation of runoff model performance. The Kling-Gupta efficiency (R_KG), which has been introduced as an improvement of the widely used Nash-Sutcliffe efficiency, considers different types of model errors, namely the error in the mean, the variability, and the dynamics. The calculation of R_KG is implicitly based on the assumptions of data linearity, data normality, and the absence of outliers. In this study, we propose a modification of R_KG as an efficiency measure comprising non-parametric components, i.e. the Spearman rank correlation and the normalized flow–duration curve. The performances of model simulations for 100 catchments using the new measure were compared to those obtained using R_KG based on a number of statistical metrics and hydrological signatures. The new measure resulted overall in better or comparable model performances, and thus it was concluded that efficiency measures with non-parametric components provide a suitable alternative to commonly used measures.     

京津冀地区井水位的数字化观测及其地震前兆监测效能评估

在较为系统地调查北京、 天津、 河北地区井水位数字化观测现状的基础上, 分析了各井水位的年、 月、 日动态特征, 并根据以往的震例评估了地震前兆监测的效能。 研究结果表明, 约90%井的水位年动态有一定规律性, 可在地震中期前兆监测中发挥效能, 约50%井的水位日、 月动态有一定规律性, 无阶变与超3倍均方差的脉冲, 可在地震短期和短临前兆监测中发挥一定效能。 研究结果还表明, 数字化井水位观测中目前较为突出的问题是, 约一半的井水位动态中存在较大幅度的脉冲状"异常"变化, 严重影响地震前兆监测效能。     

京津冀地区氡的数字化观测及其地震前兆监测效能评估

京津冀地区已建成氡的数字化(气氡)观测井(泉)共12口, 目前正在运行8口, 其中有1口井观测的数值低于仪器检出限, 故实际有效观测井(泉)为7口。 文中系统分析2004年7口井(泉)气氡年、 月、 日动态特征, 根据以往的水氡震例评估其地震中期、 短期与短临前兆异常的可能性, 认为多数井中有可能识别出以破坏年变规律为标志的中期前兆异常, 但很难识别出以阶变或脉冲为特征的短期和短临前兆异常。 文中还对比分析了同一口井(泉)气氡与水氡动态的差异性, 发现气氡的动态稳定性明显不如水氡, 这不利于气氡在地震短期或短临前兆异常监测中发挥作用。     

京津冀地区井水温度的数字化观测及其地震前兆监测效能的评估

较为系统的分析了北京、 天津、 河北地区井水温度观测现状及年、 月、 日不同层次上的动态特征, 及其在相关的地震前兆监测中发挥效能的可能性做了宏观的评估。 评估结果表明, 74.7%井的水温观测正常运行, 58.6%的井在地震中期或中短期的前兆监测发挥效能, 67.8%的井在地震短期前兆监测发挥效能, 41.2%的井在地震短临前兆监测中有可能发挥一定效能。 文中还提出了进一步发挥井水温度观测监测效能的若干建议。     

Evaluating sampling efficiency when estimating sediment source contributions to suspended sediment in rivers by fingerprinting

A general method is proposed which measures the increase in uncertainty when sampling effort is reduced in sediment fingerprinting. The method gives quantitative measures of how reduced sampling of material in one of the source areas, and/or of suspended sediment in streams, increases the uncertainties in the proportions of sediment contributed from the sources. Because the proportions of sediment contributed by the source areas must add to one, standard errors of the estimated proportions cannot be used as the usual measures of uncertainty: the paper uses instead the volume of the joint 95% confidence region for the estimated proportions. The paper shows how the uncertainty in this volume changes as numbers of suspended sediment samples, and the numbers of samples collected from cropped fields, are reduced by successive steps from 24 (20, in the case of cropped fields) to 16, 12, 8, 4 and 2 samples. As expected, uncertainty increases rapidly as the number of samples – whether of suspended sediment or from cropped fields – is reduced drastically. The pattern of increasing uncertainty is similar both for reductions in suspended sediment sampling, and for reduced sampling from cropped areas. When the number of suspended sediment samples, and the number of samples from cropped fields, are reduced to the same values, the increase in uncertainty from fewer suspended sediment samples was always slightly greater than the increased uncertainty from the reduced sampling of cropped areas, although this finding took no account of differences in the costs of field sampling and laboratory analysis. Copyright © 2016 John Wiley & Sons, Ltd.     

青海地区井水温度(地温)的数字化观测及地震前兆监测效能评估

较为系统的分析了青海地区"十五"期间架设的12个数字化水温(地温)井的观测现状,及年、月、日不同层次上的动态特征,对其在相关的地震前兆观测中发挥效能的可能性作了宏观的评估。结果表明,有10个井孔可在地震中期前兆监测中发挥效能,有7个井孔可在地震短期的前兆监测中发挥效能,有7个井孔可在地震短临前兆监测中发挥效能。文中还介绍了主要影响数字化水温(地温)资料的干扰因素。     

邢台地震台数字化水位监测效能分析

对邢台地震台地下流体数字化水位观测资料的内在质量及年、月、日动态特征等进行分析.结果表明:所辖流体观测井水位资料连续、可靠,内在质量较高,4个测点均是较理想的井水位观测点,其中,衡水井、辛集井水位对远场大震的同震效应明显;宁晋井因受井孔条件所限,同震效应不明显.井水位年变均为趋势性下降变化,但下降幅度均大于5 m,不利...     

Evaluating preprocessing methods of digital elevation models for hydrological modelling

With the introduction of high‐resolution digital elevation models, it is possible to use digital terrain analysis to extract small streams. In order to map streams correctly, it is necessary to remove errors and artificial sinks in the digital elevation models. This step is known as preprocessing and will allow water to move across a digital landscape. However, new challenges are introduced with increasing resolution because the effect of anthropogenic artefacts such as road embankments and bridges increases with increased resolution. These are problematic during the preprocessing step because they are elevated above the surrounding landscape and act as artificial dams. The aims of this study were to evaluate the effect of different preprocessing methods such as breaching and filling on digital elevation models with different resolutions (2, 4, 8, and 16 m) and to evaluate which preprocessing methods most accurately route water across road impoundments at actual culvert locations. A unique dataset with over 30,000 field‐mapped road culverts was used to assess the accuracy of stream networks derived from digital elevation models using different preprocessing methods. Our results showed that the accuracy of stream networks increases with increasing resolution. Breaching created the most accurate stream networks on all resolutions, whereas filling was the least accurate. Burning streams from the topographic map across roads from the topographic map increased the accuracy for all methods and resolutions. In addition, the impact in terms of change in area and absolute volume between original and preprocessed digital elevation models was smaller for breaching than for filling. With the appropriate methods, it is possible to extract accurate stream networks from high‐resolution digital elevation models with extensive road networks, thus providing forest managers with stream networks that can be used when planning operations in wet areas or areas near streams to prevent rutting, sediment transport, and mercury export.     

A new measure for assessing the efficiency of hydrological data-driven forecasting models

Abstract

There is a lack of consistency and generality in assessing the performance of hydrological data-driven forecasting models, and this paper presents a new measure for evaluating that performance. Despite the fact that the objectives of hydrological data-driven forecasting models differ from those of the conventional hydrological simulation models, criteria designed to evaluate the latter models have been used until now to assess the performance of the former. Thus, the objectives of this paper are, firstly, to examine the limitations in applying conventional methods for evaluating the data-driven forecasting model performance, and, secondly, to present new performance evaluation methods that can be used to evaluate hydrological data-driven forecasting models with consistency and objectivity. The relative correlation coefficient (RCC) is used to estimate the forecasting efficiency relative to the naïve model (unchanged situation) in data-driven forecasting. A case study with 12 artificial data sets was performed to assess the evaluation measures of Persistence Index (PI), Nash-Sutcliffe coefficient of efficiency (NSC) and RCC. In particular, for six of the data sets with strong persistence and autocorrelation coefficients of 0.966–0.713 at correlation coefficients of 0.977–0.989, the PIs varied markedly from 0.368 to 0.930 and the NSCs were almost constant in the range 0.943–0.972, irrespective of the autocorrelation coefficients and correlation coefficients. However, the RCCs represented an increase of forecasting efficiency from 2.1% to 37.8% according to the persistence. The study results show that RCC is more useful than conventional evaluation methods as the latter do not provide a metric rating of model improvement relative to naïve models in data-driven forecasting.

Editor D. Koutsoyiannis, Associate editor D. Yang

Citation Hwang, S.H., Ham, D.H., and Kim, J.H., 2012. A new measure for assessing the efficiency of hydrological data-driven forecasting models. Hydrological Sciences Journal, 57 (7), 1257–1274.     

A comparison of three methods for downscaling daily precipitation in the Punjab region

Many downscaling techniques have been developed in the past few years for projection of station‐scale hydrological variables from large‐scale atmospheric variables simulated by general circulation models (GCMs) to assess the hydrological impacts of climate change. This article compares the performances of three downscaling methods, viz. conditional random field (CRF), K‐nearest neighbour (KNN) and support vector machine (SVM) methods in downscaling precipitation in the Punjab region of India, belonging to the monsoon regime. The CRF model is a recently developed method for downscaling hydrological variables in a probabilistic framework, while the SVM model is a popular machine learning tool useful in terms of its ability to generalize and capture nonlinear relationships between predictors and predictand. The KNN model is an analogue‐type method that queries days similar to a given feature vector from the training data and classifies future days by random sampling from a weighted set of K closest training examples. The models are applied for downscaling monsoon (June to September) daily precipitation at six locations in Punjab. Model performances with respect to reproduction of various statistics such as dry and wet spell length distributions, daily rainfall distribution, and intersite correlations are examined. It is found that the CRF and KNN models perform slightly better than the SVM model in reproducing most daily rainfall statistics. These models are then used to project future precipitation at the six locations. Output from the Canadian global climate model (CGCM3) GCM for three scenarios, viz. A1B, A2, and B1 is used for projection of future precipitation. The projections show a change in probability density functions of daily rainfall amount and changes in the wet and dry spell distributions of daily precipitation. Copyright © 2011 John Wiley & Sons, Ltd.     

Comparison of ground-based and UAV a-UHF artificial tracer mobility monitoring methods on a braided river

Radio frequency identification (RFID) technologies, which allow wireless detection of individual buried or immersed tracers, represent a step forward in sediment tracking, especially passive integrated transponders (PIT tags) that have been widely used. Despite their widespread adoption in the scientific community, they typically have low efficiency when deployed in river systems with active bedload transport or deep wet channels, attributed to their technical specifications. A recent evaluation of active ultra-high frequency transponders (a-UHF tags) assessed their larger detection range and provided a methodology for their geopositioning. In this study, we test five different survey methods (one including an unmanned aerial vehicle [UAV]) in a sediment tracking study, and compare them in terms of recovery rate, field effort, geopositioning error, and efficiency. We then tested the method on a larger reach following a Q₅ flood and performed cross-comparisons between active and passive RFIDs. The results confirmed that the a-UHF RFID technology allowed rapid (1.5 h ha⁻¹) survey of a large area (<34 ha) of emerged bars and shallow water channels with recovery of a high percentage of tracers (72%) that had travelled large distances (mean ≈ 1000 m; max ≈ 3400 m). Moreover, the tracers were identified with low geopositioning error (mean ≈ 7.1 m, ≤1% of their travel distance). We also showed that a UAV-based survey was fast (0.38 h ha⁻¹), efficient (recovery rate = 84%), and low error (mean ≈ 4.2 m). Thus, a-UHF RFID technology permits the development of a variety of survey methods, depending on the study objectives and the human and financial resources available. This allows field efforts to be optimized by determining an appropriate balance between the high equipment cost of a-UHF tracers and the resulting reduced survey costs. © 2019 John Wiley & Sons, Ltd.     

景洪电站水库诱发地震监测台网地震监测能力评估

基于近震震级公式,得出景洪电站水库诱发地震监测台网理论监测能力在坝区为M_L 0.5,在坝区周边,理论监测能力为M_L 0.7。基于G-R震级频度关系及Wiemer Wyss最小完整性震级估计方法求取景洪电站水库诱发地震监测台网实际监测能力,结果证明,其实际监测能力能够满足理论监测能力的要求。在0.5的理论监测区范围内,最小完整性震级为M_L 0.0,而在0.7的理论监测范围内,最小完整性震级为M_L 0.4。     

Reproductive strategies of three invasive Gobiidae co-occurring in the Lower Rhine (Germany)

At the Lower Rhine, three congeneric, invasive Gobiidae constitute the major part of local fish assemblages. Reproduction strategies play an important role in biological invasion processes and help to predict future population development. Up to now, studies on the reproductive traits of the three species have led to contradictory findings in invaded areas, and studies on native populations are scarce, especially when regarding a co-occurrence of the three species. This study provides an overview over already existing knowledge of reproductive traits in native as well as invaded regions and describes the reproduction modes of all three species at the Lower Rhine. Based on intensive beach seine sampling in nursery habitats for three consecutive years, detailed information about the spawning habits of the bighead goby Ponticola kessleri, the round goby Neogobius melanostomus, and the monkey goby N. fluviatilis is provided. Spawning onset as well as length and intensity of the spawning season were determined by estimating growth rates and subsequent back-calculations of hatching and spawning events. Gonadosomatic index (GSI) analyses of adult gobies were used to cross-validate results and to assess validity of the used method. Growth rates for the first three months of all three species were highly variable and ranged between 0.3 mm and 0.95 mm total length per day. All three Gobiidae displayed different reproductive modes with bighead gobies spawning first in March/April, but probably only once a season. In contrast, round and monkey gobies started reproducing slightly later, but then continued spawning throughout the season until September, exhibiting temporal separation with varying intensities. High variation in spawning onset, length and reproductive output for all three species was observed over the years and no relationship between spawning onset and temperature could be detected. For the first time, highly resolvent information on spawning onset and spawning season length is obtained and the fine-tuned differences between the three species are highlighted.     

区域测震台网监测效能评估方法研究及应用

如何通过评价区域测震台网方式,改善与提高我国区域测震台网监测效能,是当前我们面临的一个新的课题。本文提出的一个省级区域测震台网监测效能评价体系基本思路是,通过建立台网监测效能评价子系统,引入动态统计,指标分析,目标评价,公众评判的综合分析与关联评价方法,探讨我国省级测震台网系统性能与台网运行,维护能力及资料产出、数据服务之间的相互关联。以安徽区域测震台网为例,检验区域台网监测效能评价体系的可行性与实效性。     

淡水生态系统温室气体通量监测方法综述

淡水系统是全球温室气体排放清单中最具不确定性的组成部分。淡水生态系统温室气体通量监测的方法多种多样,不同方法之间观测结果存在偏差,在一定程度上限制了全球或区域间的相互比较,是造成目前全球估算不确定性的重要原因。然而,目前关于淡水系统温室气体排放通量监测方法的系统梳理、比较及影响因素分析等鲜见报道。本研究在总结淡水系统温室气体排放特征的基础上,整理当前主要监测方法（箱法、边界层模型法、微气象法、倒置漏斗法）的基本原理、通用模型与算法、适用条件以及优缺点等,评价不同方法在实际应用中的适应性;进一步分析不同方法观测偏差的主要影响因素,提出当前箱法与边界层模型法存在方案众多、缺乏统一规范等问题,为研究方法的优化提供科学支撑;同时,综述了不同方法的对比研究,大部分研究发现箱法估算结果高于边界层模型法,而微气象法与箱法、边界层模型法观测的结果的一致性仍存在争议;最后,本研究提出未来应重点关注淡水系统温室气体通量监测方法的偏差来源及潜在影响,建议开展更多相关的方法学研究,进一步优化相关的估算模型及参数,研发具有普适性的监测规范,为提高监测结果的精度提供支撑。     

湖泊沉积物碳酸盐含量3种测定方法的比较

湖泊沉积物中碳酸盐含量是指示气候和环境变化的常用指标之一,其测定方法多样,但缺乏不同测定方法结果之间的比较,尤其是对同一沉积序列样品的比较.选择新疆天山大龙池DLC12孔110 cm岩芯沉积物,按1 cm间隔取样,利用烧失量法、酸碱滴定法和气量法分别进行碳酸盐含量分析,并比较3种分析方法在表达湖泊沉积物碳酸盐含量时的差异性.结果表明:3种方法测定的碳酸盐含量随岩芯深度的变化趋势并无太大差别,其中烧失量法测定的结果较滴定法和气量法测定的结果平均分别偏高5%和3%,气量法结果与滴定法结果较为接近,平均差值不到2%,表明烧失量测定过程中有其他矿物分解导致碳酸盐含量被高估.将3种方法测定的结果与岩芯中20个样品的X衍射(XRD)得到的碳酸盐含量对比,发现气量法的测定结果与XRD测定结果的相关性最高.通过比较各种方法的优缺点,考虑到分析精度、操作技巧和分析费用,认为气量法比较适合湖泊沉积物的碳酸盐含量分析.     

Evaluating simulated daily discharge for operational hydrological drought monitoring in the Global Drought Observatory (GDO)

ABSTRACT

Hydrological drought is currently underrepresented in global monitoring systems, mainly due the shortage of near real-time estimates of river discharge at the global scale. In this study, the outputs of the Lisflood model are used to define a low-flow drought index, which shows a good correspondence with long-term records of the Global Runoff Data Centre in the period 1980–2014, as well as with verified information from the literature on six major drought events (covering different regions and watershed sizes). In contrast, the near real-time simulation (from 2015 onward) provides temporally inconsistent estimates over about 20% of the modelled cells (mostly over South America and Central Africa), even if reasonable results are obtained over other regions, as confirmed by intercomparison with the operational outcomes of the European Drought Observatory for the 2018 drought. In spite of the highlighted limitations, valuable information for operational drought monitoring can be retrieved from these simulations.     

Evaluating the efficiency of supplementary rebar system in improving hysteretic damping of self-centering rocking walls

Bulletin of Earthquake Engineering - Although self-centering rocking walls have shown acceptable performance in decreasing downtime, repair cost, and continuous serviceability, their energy...