A Monte Carlo study of rainfall forecasting with a stochastic model

A procedure for short-term rainfall forecasting in real-time is developed and a study of the role of sampling on forecast ability is conducted. Ground level rainfall fields are forecasted using a stochastic space-time rainfall model in state-space form. Updating of the rainfall field in real-time is accomplished using a distributed parameter Kalman filter to optimally combine measurement information and forecast model estimates. The influence of sampling density on forecast accuracy is evaluated using a series of a simulated rainfall events generated with the same stochastic rainfall model. Sampling was conducted at five different network spatial densities. The results quantify the influence of sampling network density on real-time rainfall field forecasting. Statistical analyses of the rainfall field residuals illustrate improvement in one hour lead time forecasts at higher measurement densities.     

An approach to handling non-Gaussianity of parameters and state variables in ensemble Kalman filtering

The ensemble Kalman filter (EnKF) is a commonly used real-time data assimilation algorithm in various disciplines. Here, the EnKF is applied, in a hydrogeological context, to condition log-conductivity realizations on log-conductivity and transient piezometric head data. In this case, the state vector is made up of log-conductivities and piezometric heads over a discretized aquifer domain, the forecast model is a groundwater flow numerical model, and the transient piezometric head data are sequentially assimilated to update the state vector. It is well known that all Kalman filters perform optimally for linear forecast models and a multiGaussian-distributed state vector. Of the different Kalman filters, the EnKF provides a robust solution to address non-linearities; however, it does not handle well non-Gaussian state-vector distributions. In the standard EnKF, as time passes and more state observations are assimilated, the distributions become closer to Gaussian, even if the initial ones are clearly non-Gaussian. A new method is proposed that transforms the original state vector into a new vector that is univariate Gaussian at all times. Back transforming the vector after the filtering ensures that the initial non-Gaussian univariate distributions of the state-vector components are preserved throughout. The proposed method is based in normal-score transforming each variable for all locations and all time steps. This new method, termed the normal-score ensemble Kalman filter (NS-EnKF), is demonstrated in a synthetic bimodal aquifer resembling a fluvial deposit, and it is compared to the standard EnKF. The proposed method performs better than the standard EnKF in all aspects analyzed (log-conductivity characterization and flow and transport predictions).     

Integration of transfer function model and back propagation neural network for forecasting storm sewer flow in Taipei metropolis

Typhoons and storms have often brought heavy rainfalls and induced floods that have frequently caused severe damage and loss of life in Taiwan. Our ability to predict sewer discharge and forecast floods in advance during storm seasons plays an important role in flood warning and flood hazard mitigation. In this paper, we develop an integrated model (TFMBPN) for forecasting sewer discharge that combines two traditional models: a transfer function model and a back propagation neural network. We evaluated the integrated model and the two traditional models by applying them to a sewer system of Taipei metropolis during three past typhoon events (NARI, SINLAKU, and NAKR). The performances of the models were evaluated by using predictions of a total of 6 h of sewer flow stages, and six different evaluation indices of the predictions. Finally, an overall performance index was determined to assess the overall performance of each model. Based on these evaluation indices, our analysis shows that TFMBNP yields accurate results that surpass the two traditional models. Thus, TFMBNP appears to be a promising tool for flood forecasting for the Taipei metropolis sewer system. For publication in Stochastic Environmental Research and Risk Analysis.     

Subsurface characterization with localized ensemble Kalman filter employing adaptive thresholding

Ensemble Kalman filter, EnKF, as a Monte Carlo sequential data assimilation method has emerged promisingly for subsurface media characterization during past decade. Due to high computational cost of large ensemble size, EnKF is limited to small ensemble set in practice. This results in appearance of spurious correlation in covariance structure leading to incorrect or probable divergence of updated realizations. In this paper, a universal/adaptive thresholding method is presented to remove and/or mitigate spurious correlation problem in the forecast covariance matrix. This method is, then, extended to regularize Kalman gain directly. Four different thresholding functions have been considered to threshold forecast covariance and gain matrices. These include hard, soft, lasso and Smoothly Clipped Absolute Deviation (SCAD) functions. Three benchmarks are used to evaluate the performances of these methods. These benchmarks include a small 1D linear model and two 2D water flooding (in petroleum reservoirs) cases whose levels of heterogeneity/nonlinearity are different. It should be noted that beside the adaptive thresholding, the standard distance dependant localization and bootstrap Kalman gain are also implemented for comparison purposes. We assessed each setup with different ensemble sets to investigate the sensitivity of each method on ensemble size. The results indicate that thresholding of forecast covariance yields more reliable performance than Kalman gain. Among thresholding function, SCAD is more robust for both covariance and gain estimation. Our analyses emphasize that not all assimilation cycles do require thresholding and it should be performed wisely during the early assimilation cycles. The proposed scheme of adaptive thresholding outperforms other methods for subsurface characterization of underlying benchmarks.     

基于GPS技术实时监测2009年7月22日日全食期间长三角区域电离层TEC变化

2009年7月22日上午发生的日全食是21世纪持续时间最长的日全食,其全食带覆盖了中国中部的K江流域,为研究日全食对电离层的影响提供了一次难得的机会,为此本文通过卡尔曼滤波算法实现了实时求解TEC和GPS系统硬件延迟,为实时监测日全食期间电离层变化提供了绝对的电离层TEC.采用上海和浙江区域内GPS网的观测数据,建立了...     

Monitoring ephemeral headwater streams: a paired‐sensor approach

This paper introduces a paired‐sensor approach to monitoring ephemeral streamflow. Part of this approach includes the design of a new flow detection sensor. This flow detection sensor addresses the limitation of previous electronic resistance sensors that use water presence as a proxy for flow for assessing hydrological connectivity, by explicitly measuring flow presence. Using paired electronic resistance and flow detection sensors, this paper evaluates the performance of each sensor individually, and as a pair. Individually, the sensors were tested for the amount of noise they contain and the types of errors they were prone to committing. As a paired set, the sensors were analysed by the percent of time they were in valid states versus invalid states. Valid states included when water was present but flow was absent, when water and flow were both present and when water and flow were both absent during a storm. One invalid case existed, where the sensors recorded flow presence but not water presence. These valid and invalid cases were assessed using data collected from sensor networks established at two study sites in southern Ontario. This analysis was completed for the overall corroboration at each site, for each storm at each site and based on the relative position of the sensors in the channel at each site. The sensors were in valid states 83% and 94% of the time at each respective study site. Differences in local site conditions were found to affect the performance of the sensor network; however, no significant correlation was found between storm characteristics and sensor performance. Particularly, bed roughness was found to be a factor as it restricted the placement of the sensors. Despite this, the paired‐sensor network helps to increase the understanding of the flow dynamics within headwater streams by explicitly separating the two hydrological characteristics. A discussion of the challenges, limitations and opportunities of monitoring ephemeral flow is presented, and insights into how to address those limitations are provided. Copyright © 2015 John Wiley & Sons, Ltd.     

Monthly stream flow forecasting via dynamic spatio-temporal models

In this research, a dynamic linear spatio-temporal model (DLSTM) was developed and evaluated for monthly streamflow forecasting. For parameter estimation, coupled expectation–maximization (EM) algorithm and Kalman filter was adopted. This combination enables the model to estimate the state vector and parameters concurrently. Different forecast scenarios including various combinations of upstream stations were considered for downstream station streamflow forecasting. Several statistical criteria, nonparametric and visual tests were used for model evaluation. Results indicated that the spatio-temporal model performed acceptably in almost all scenarios. The dynamic model was able to capitalize on coupled spatial and temporal information provided that there is spatial connectivity in the studied hydrometric stations network. Moreover, threshold level method was used for model evaluation in drought and wet periods. Results indicated that, in validation phase, the model was able to forecast the drought duration and volume deficit/over threshold, although volume deficit/over threshold could not be accurately simulated.     

Evaluating the utility of the ensemble transform Kalman filter for adaptive sampling when updating a hydrodynamic model

This paper compares two Monte Carlo sequential data assimilation methods based on the Kalman filter, for estimating the effect of measurements on simulations of state error variance made by a one-dimensional hydrodynamic model. The first method used an ensemble Kalman filter (EnKF) to update state estimates, which were then used as initial conditions for further simulations. The second method used an ensemble transform Kalman filter (ETKF) to quickly estimate the effect of measurement error covariance on forecast error covariance without the need to re-run the simulation model. The ETKF gave an unbiased estimate of EnKF analysed error variance, although differences in the treatment of measurement errors meant the results were not identical. Estimates of forecast error variance could also be made, but their accuracy deteriorated as the time from measurements increased due in part to model non-linearity and the decreasing signal variance. The motivation behind the study was to assess the ability of the ETKF to target possible measurements, as part of an adaptive sampling framework, before they are assimilated by an EnKF-based forecasting model on the River Crouch, Essex, UK. The ETKF was found to be a useful tool for quickly estimating the error covariance expected after assimilating measurements into the hydrodynamic model. It, thus, provided a means of quantifying the ‘usefulness’ (in terms of error variance) of possible sampling schemes.     

Real-time correction of water stage forecast during rainstorm events using combination of forecast errors

This study proposes a real-time error correction method for the forecasted water stage using a combination of forecast errors estimated by the time series models, AR(1), AR(2), MA(1) and MA(2), and the average deviation model to update the water stage forecast during rainstorm events. During flood forecasting and warning operations, the proposed real-time error correction method takes advantage of being individually and continuously implemented and the results not being updated to the hydrological model and hydraulic routings so as to save computational time by recalibrating the parameters of the proposed methods with real-time observation. For model validation, the current study adopts the observed and forecasted data on a severe typhoon, Morakot, collected at eight water level gauges in Southern Taiwan and provided by the flood forecast system FEWS_Taiwan, which is linked with the reliable quantitative precipitation forecast (QPF) at 3 h of lead time provided by the Center Weather Bureau in Taiwan, as the model validation. The results of numerical experiments indicate that the proposed real-time error correction method can effectively reduce the errors of forecasted water stages at the 1-, 2-, and 3-h lead time and so enhance the reliability of forecast information issued by the FEWS_Taiwan. By means of real-time estimating potential forecast error, the uncertainties in hydrology, modules as well as associated parameters, and physiographical features of the river can be reduced.     

On the oceanic impact of a data-assimilation method in a coupled ocean-land-atmosphere model

 Based on the Kalman filter theory, a new data-assimilation method has been used to improve the 3-D oceanic temperature field of the Center for Ocean-Land-Atmosphere Studies (COLA) coupled general circulation model. This method is applied to assimilate surface and subsurface temperature of in situ measurements from the Pilot Research Moored Array in the Tropical Atlantic project (PIRATA). The assimilation of the PIRATA data produces an improved representation of the thermal state of the ocean and allows a better estimation of other oceanographic quantities, like meridional heat fluxes and zonal currents. The present paper focuses on the tropical Atlantic and, in particular, it contains new reconstructed temperature profiles. One-month forecast experiments during 1999 were performed and the impact of the assimilation is discussed. Received: 24 April 2001 / Accepted: 8 March 2002     

Determination of subcatchment and watershed boundaries in a complex and highly urbanized landscape

Urban development significantly alters the landscape by introducing widespread impervious surfaces, which quickly convey surface run‐off to streams via stormwater sewer networks, resulting in “flashy” hydrological responses. Here, we present the inadequacies of using raster‐based digital elevation models and flow‐direction algorithms to delineate large and highly urbanized watersheds and propose an alternative approach that accounts for the influence of anthropogenically modified land cover. We use a semi‐automated approach that incorporates conventional drainage networks into overland flow paths and define the maximal run‐off contributing area. In this approach, stormwater pipes are clustered according to their slope attributes, which define flow direction. Land areas drained by each cluster and contributing (or exporting) flow to a topographically delineated catchment were determined. These land masses were subsequently added or removed from the catchment, modifying both the shape and the size. Our results in a highly urbanized Toronto, Canada, area watershed indicate a moderate net increase in the directly connected watershed area by 3% relative to a topographically forced method; however, differences across three smaller scale subcatchments are greater. Compared to topographic delineation, the directly connected watershed areas of both the upper and middle subcatchments decrease by 5% and 8%, respectively, whereas the lower subcatchment area increases by 15%. This is directly related to subsurface storm sewer pipes that cross topographic boundaries. When directly connected subcatchment area is plotted against total streamflow and flashiness indices using this method, the coefficients of variation are greater (0.93 to 0.97) compared to the use of digital elevation model‐derived subcatchment areas (0.78 to 0.85). The accurate identification of watershed and subcatchment boundaries should incorporate ancillary data such as stormwater sewer networks and retention basin drainage areas to reduce water budget errors in urban systems.     

Neural networks in auroral data assimilation

Data assimilation is an essential step for improving space weather forecasting by means of a weighted combination between observational data and data from a mathematical model. In the present work data assimilation methods based on Kalman filter (KF) and artificial neural networks are applied to a three-wave model of auroral radio emissions. A novel data assimilation method is presented, whereby a multilayer perceptron neural network is trained to emulate a KF for data assimilation by using cross-validation. The results obtained render support for the use of neural networks as an assimilation technique for space weather prediction.     

Realizing User-Relevant Conceptual Model for the Ski Jump Venue of the Vancouver 2010 Winter Olympics

As was the case for most other Olympic competitions, providing weather guidance for the ski jump and Nordic combined events involved its own set of unique challenges. The extent of these challenges was brought to light before the Vancouver 2010 Winter Olympics during a series of outflow wind events in the 2008/2009 winter season. The interactions with the race officials during the difficult race conditions brought on by the outflows provided a new perspective on the service delivery requirements for the upcoming Olympic Games. In particular, the turbulent nature of the winds and its impact on the ski jump practice events that season highlighted the need of race officials for nowcasting advice at very short time scales (from 2 min to 1 h) and forecast products tailored to their decision-making process. These realizations resulted in last minute modifications to the monitoring strategy leading up to the Olympic Games and required forecasters’ conceptual models for flow within the Callaghan Valley to be downscaled further to reflect the evolution of turbulence at the ski jump site. The SNOW-V10 (Science of Nowcasting Olympic Weather for Vancouver 2010) team provided support for these efforts by supplying diagnostic case analyses of important events using numerical weather data and by enhancing the real-time monitoring capabilities at the ski jump venue.     

Uncertainty in sewer sediment deposit modelling: Detailed vs simplified modelling approaches

The paper presents the results of a study in which the uncertainty levels associated with a detailed and a simplified/parsimonious sewer sediment modelling approach have been compared. The detailed approach used an Infoworks CS sewer network model combined with a user developed sediment transport code and the simplified approach used a conceptual sewer flow and quality model. The two approaches have been applied to a single case study sewer network and the simulation results compared. The case study was selected as moderate storm events had occurred during a 2 year rainfall and sewer flow monitoring period. Flooding had been observed and this was thought to be caused by significant solids accumulation in the sewer network. As a result sediment deposit measurements were carried out over a 6 month period. Model simulations were made of this period and predictions obtained of sediment deposit location and depth. The uncertainty analysis of both modelling approaches was carried out using Monte Carlo based computational methods. This was a limitation for the detailed approach with regards to computational time. Use of the simplified model was not constrained by this issue and so a more conventional assessment of the uncertainty was possible. The simplified approach, due to its structure, only provided a temporal estimate of uncertainty at the final section of the catchment. The detailed approach enabled an assessment of uncertainty at an individual pipe scale but only at the end of the simulation period. A comparison of the uncertainty estimations from both methods at the final section of the catchment and the end of the simulation period indicated comparable values of predicted uncertainty. Therefore a complementary use of both approaches would allow reasonably comparable estimations of levels of uncertainty at both a spatial and temporal scale. The use of such modelling approaches may provide a useful decision-making tool for sewer system management.     

Calibration framework for a Kalman filter applied to a groundwater model

The paper presents a novel approach to the setup of a Kalman filter by using an automatic calibration framework for estimation of the covariance matrices. The calibration consists of two sequential steps: (1) Automatic calibration of a set of covariance parameters to optimize the performance of the system and (2) adjustment of the model and observation variance to provide an uncertainty analysis relying on the data instead of ad-hoc covariance values. The method is applied to a twin-test experiment with a groundwater model and a colored noise Kalman filter. The filter is implemented in an ensemble framework. It is demonstrated that lattice sampling is preferable to the usual Monte Carlo simulation because its ability to preserve the theoretical mean reduces the size of the ensemble needed. The resulting Kalman filter proves to be efficient in correcting dynamic error and bias over the whole domain studied. The uncertainty analysis provides a reliable estimate of the error in the neighborhood of assimilation points but the simplicity of the covariance models leads to underestimation of the errors far from assimilation points.     

基于Gauss-Markov卡尔曼滤波的电离层数值同化现报预报系统的构建-以中国及周边地区为例的观测系统模拟试验

本文给出了一个基于Gauss-Markov卡尔曼滤波的电离层数据同化系统的初步构建和试验结果.我们选择中国及周边地区部分涉及电离层观测的台站(包括子午工程台站、中国地壳形变网和部分IGS台站)作为观测系统进行模拟试验,背景场利用IRI模式,观测值则由NeQuick模式计算得到.我们的同化结果表明,采用Kalman滤波算法,把部分斜TEC同化到背景模式当中,能够获得较好的同化结果,说明我们设计的算法可行、所选择的各种参数比较合理,采用Gauss-Markov假设进行短期预报也取得了较合理的结果.本项研究经过进一步的改进和完善,可以用来对中国地区的电离层进行现报和短期预报,一方面满足相关空间工程应用,另一方面可以提升现有观测系统的科学意义.     

Analysis and integrated modelling of groundwater infiltration to sewer networks

Infiltration of groundwater to sewer systems is a problem for the capacity of the system as well as for treatment processes at waste water treatment plants. This paper quantifies the infiltration of groundwater to a sewer system in Frederikshavn Municipality, Denmark, by measurements of sewer flow and novel model set‐up, which simulates the interaction between groundwater and sewer flow. The study area has a separate waste water sewer system, but the discharged volumes from the system are approximately twice the volumes from a tight system without infiltration. The model set‐up makes use of two commercial models: mike she for simulation of groundwater transport and mike urban (mouse ) [DHI, Hørsholm, Denmark] for simulation of sewer flow. By simulating the groundwater level and calibrating infiltration coefficients against sewer flow measurements, it has been possible to estimate the average infiltration to the sewer system with satisfying results. The infiltration processes are indeed complicated and to a large degree heterogeneous throughout the sewer system. The paper shows contribution from both saturated and unsaturated groundwater zones, which makes the modelling process complex. Copyright © 2016 John Wiley & Sons, Ltd.     

Classified real-time flood forecasting by coupling fuzzy clustering and neural network

This paper presented a new classified real-time flood forecasting framework by integrating a fuzzy clustering model and neural network with a conceptual hydrological model. A fuzzy clustering model was used to classify historical floods in terms of flood peak and runoff depth, and the conceptual hydrological model was calibrated for each class of floods. A back-propagation (BP) neural network was trained by using real-time rainfall data and outputs from the fuzzy clustering model. BP neural network provided a rapid on-line classification for real-time flood events. Based on the on-line classification, an appropriate parameter set of hydrological model was automatically chosen to produce real-time flood forecasting. Different parameter sets was continuously used in the flood forecasting process because of the changes of real-time rainfall data and on-line classification results. The proposed methodology was applied to a large catchment in Liaoning province, China. Results show that the classified framework provided a more accurate prediction than the traditional non-classified method. Furthermore, the effects of different index weights in fuzzy clustering were also discussed.     

位移和加速度融合的自适应多速率Kalman滤波方法

多速率Kalman滤波方法可用于低采样率的位移和高采样率的加速度数据融合,而未知的噪声协方差信息则显著制约着多速率Kalman滤波精度.本文通过将多速率Kalman滤波转换为传统的单速率Kalman滤波,建立了Kalman滤波增益的自协方差矢量与未知的加速度谱密度和观测噪声参数间的线性函数模型,并采用最小二乘估计方法对未知的噪声协方差参数进行估计,进而有效地提高了多速率Kalman滤波精度.数值仿真和震动台实验结果验证了本文方法的正确性和有效性.     

基于深度学习技术的地震实时流数据自动处理辅助工具开发

以JOPENS系统实时流接收为基础,应用Redis共享内存技术和近年来发展较快的深度学习震相自动识别技术,设计一套可7×24小时不间断稳定接收并实时识别连续地震流数据中P、S震相的系统,为地震台网实时数据处理提供一套辅助工具,并在福建省地震局测震台网128个台站的实时数据流上进行测试。该工具由Redis实时数据流共享模块与深度学习震相到时自动拾取、MSDP震相格式转换3个模块组成,可以实时接收并自动识别台网地震连续波形,生成P、S震相报告,并可导入MSDP人机交互工具进一步处理,在一定程度上可以减轻人工处理工作量。