Evaluation of the NMC Regional Ensemble Prediction System During the Beijing 2008 Olympic Games

Based on the B08RDP (Beijing 2008 Olympic Games Mesoscale Ensemble Prediction Research and Development Project) that was launched by the World Weather Research Programme (WWRP) in 2004, a regional ensemble prediction system (REPS) at a 15-km horizontal resolution was developed at the National Meteorological Center (NMC) of the China Meteorological Administration (CMA). Supplementing to the forecasters’ subjective affirmation on the promising performance of the REPS during the 2008 Beijing Olympic Games (BOG), this paper focuses on the objective verification of the REPS for precipitation forecasts during the BOG period. By use of a set of advanced probabilistic verification scores, the value of the REPS compared to the quasi-operational global ensemble prediction system (GEPS) is assessed for a 36-day period (21 July–24 August 2008). The evaluation here involves different aspects of the REPS and GEPS, including their general forecast skills, specific attributes (reliability and resolution), and related economic values. The results indicate that the REPS generally performs significantly better for the short-range precipitation forecasts than the GEPS, and for light to heavy rainfall events, the REPS provides more skillful forecasts for accumulated 6- and 24-h precipitation. By further identifying the performance of the REPS through the attribute-focused measures, it is found that the advantages of the REPS over the GEPS come from better reliability (smaller biases and better dispersion) and increased resolution. Also, evaluation of a decision-making score reveals that a much larger group of users benefits from using the REPS forecasts than using the single model (the control run) forecasts, especially for the heavy rainfall events.     

集合数值预报在洪水预报中的应用进展

水文集合预报是近几年正在形成和发展的水文预报分支,其发展大致可分为两个阶段:第1阶段是1970年至20世纪末进行的长期径流预报,第2阶段从21世纪开始,主要学习气象数值预报中集合预报的概念在短期水文集合预报中的应用。目前,除了单一预报中心的集合预报系统在水文集合预报中应用外,多个预报中心的集合预报大集合也逐渐被应用于流域水文预报,甚至一些小流域的洪水预报。如利用TIGGE(THORPEX Interactive Grand Global Ensemble)集合预报驱动形成的大气-水文-水力的串联系统进行早期的洪水预警研究,将全球集合预报作为洪水模型输入的有限区域模式的初始条件和侧边界条件的研究。这些均表明,基于水文集合预报的洪水预报增加了预报附加值,并能够延长预警提前时间。以欧洲中期天气预报中心的欧洲洪水预警系统(EFAS)和美国NOAA的先进水文预报系统(AHPS)为代表,实现了集合预报在洪水中的实时业务预报,但仍存在数据处理和计算量大,以及如何基于集合水文预报做决策等问题。对于水文集合预报的前处理和后处理的各种技术已处于探索和验证阶段,如何更好地理解基于概率预报的洪水预警决策仍存在许多困难和挑战。     

GRAPES区域集合预报两种侧边界扰动方法对比试验

中国气象局数值预报中心自2014年建立了区域集合预报业务系统,其使用的侧边界扰动由全球集合预报系统动力降尺度得到。为深入了解侧边界扰动对区域集合预报的影响,基于15 km水平分辨率的区域集合预报模式,使用动力降尺度方法和尺度化滞后平均法(scaled lagged average forecasting,SLAF)设计构造了两种侧边界扰动方案,并开展了2015年7月共6天的集合预报试验,利用集合均方根误差、集合离散度、连续分级概率评分、离群值、Brier Score及相对作用特征曲线面积等概率预报检验方法进行了多方面检验,分析了两种侧边界扰动方案对区域集合预报质量的影响。结果表明:动力降尺度侧边界扰动方案(DOWN)的扰动总能量在各垂直层次均大于SLAF方案,使得边界上前者的离散度大于后者,集合扰动增长更为合理;对于等压面要素和地面要素,DOWN方案的离散度、Outlier、CRPS等评分优于SLAF方案,反映了DOWN方案构造的侧边界扰动更加合理;在降水概率预报技巧方面,SLAF方案在评分上具有一定优势,但评分的提高没有通过显著性水平检验,因此认为两种方案对降水预报的改进基本相当。     

基于TIGGE多模式集合的24小时气温BMA 概率预报

利用TIGGE(THORPEX Interactive Grand Global Ensemble)单中心集合预报系统(ECMWF、United Kingdom Meteorological Office、China Meteorological Administration和NCEP)以及由此所构成的多中心模式超级集合预报系统24小时地面日均气温预报,结合淮河流域地面观测率定贝叶斯模型平均(Bayesian model averaging,BMA)参数,从而建立地面日均气温BMA概率预报模型.由此针对淮河流域进行地面日均气温BMA概率预报及其检验与评估,结果表明BMA模型比原始集合预报效果好;单中心的BMA概率预报都有较好的预报效果,其中ECMWF最好.多中心模式超级集合比单中心BMA概率预报效果更好,采用可替换原则比普通的多中心模式超级集合BMA模型计算量小,且在上述BMA集合预报系统中效果最好.它与原始集合预报相比其平均绝对误差减少近7%,其连续等级概率评分提高近10%.基于采用可替换原则的多中心模式超级集合BMA概率预报,针对研究区域提出了极端高温预警方案,这对防范高温天气有着重要意义.     

The Meta-Gaussian Bayesian Processor of Forecasts and Associated Preliminary Experiments

Public weather services are trending toward providing users with probabilistic weather forecasts, in place of traditional deterministic forecasts. Probabilistic forecasting techniques are continually being improved to optimize available forecasting information. The Bayesian Processor of Forecast (BPF), a new statistical method for probabilistic forecast, can transform a deterministic forecast into a probabilistic forecast according to the historical statistical relationship between observations and forecasts generated by that forecasting system. This technique accounts for the typical forecasting performance of a deterministic forecasting system in quantifying the forecast uncertainty. The meta-Gaussian likelihood model is suitable for a variety of stochastic dependence structures with monotone likelihood ratios. The meta-Gaussian BPF adopting this kind of likelihood model can therefore be applied across many fields, including meteorology and hydrology. The Bayes theorem with two continuous random variables and the normal-linear BPF are briefly introduced. The meta-Gaussian BPF for a continuous predictand using a single predictor is then presented and discussed. The performance of the meta-Gaussian BPF is tested in a preliminary experiment. Control forecasts of daily surface temperature at 0000 UTC at Changsha and Wuhan stations are used as the deterministic forecast data. These control forecasts are taken from ensemble predictions with a 96-h lead time generated by the National Meteorological Center of the China Meteorological Administration, the European Centre for Medium-Range Weather Forecasts, and the US National Centers for Environmental Prediction during January 2008. The results of the experiment show that the meta-Gaussian BPF can transform a deterministic control forecast of surface temperature from any one of the three ensemble predictions into a useful probabilistic forecast of surface temperature. These probabilistic forecasts quantify the uncertainty of the control forecast; accordingly, the performance of the probabilistic forecasts differs based on the source of the underlying deterministic control forecasts.     

Forecast Performance of the Pre-operational CMA-TRAMS (EPS) in South China During April - September 2020

The mesoscale ensemble prediction system based on the Tropical Regional Atmosphere Model for the South China Sea (CMA-TRAMS (EPS)) has been pre-operational since April 2020 at South China Regional Meteorological Center (SCRMC), which was developed by the Guangzhou Institute of Tropical and Marine Meteorology (GITMM). To better understand the performance of the CMA-TRAMS (EPS) and provide guidance to forecasters, we assess the performance of this system on both deterministic and probabilistic forecasts from April to September 2020 in this study through objective verification. Compared with the control (deterministic) forecasts, the ensemble mean of the CMATRAMS (EPS) shows advantages in most non-precipitation variables. In addition, the threat score indicates that the CMA-TRAMS (EPS) obviously improves light and heavy rainfall forecasts in terms of the probability-matched mean. Compared with the European Center for Medium-range Weather Forecasts operational ensemble prediction system (ECMWF-EPS), the CMA-TRAMS (EPS) improves the probabilistic forecasts of light rainfall in terms of accuracy, reliability and discrimination, and this system also improves the heavy rainfall forecasts in terms of discrimination. Moreover, two typical heavy rainfall cases in south China during the pre-summer rainy season are investigated to visually demonstrate the deterministic and probabilistic forecasts, and the results of these two cases indicate the differences and advantages (deficiencies) of the two ensemble systems.     

2013年欧洲中心台风集合预报的检验

广州中心气象台利用中国气象局下发的欧洲中心台风集合预报数据,制作了台风集合预报产品,供业务参考应用。利用欧洲中心台风集合预报数据,对2013年1307—1331号热带气旋的集合预报路径和强度进行检验,通过对比集合平均、模式高分辨率确定性预报和预报员主观预报,发现路径集合平均在24~120 h预报误差最小;在有限的预报样本数中,从热带风暴到台风级别的热带气旋,各预报时效路径集合平均的误差随强度增强而减小;强引导气流背景下的热带气旋预报误差小于弱引导气流的误差。对比强度集合平均和模式高分辨率确定性预报,发现各时效集合平均的误差比确定性预报大,随着预报时效的延长误差没有明显增大或减小的趋势,而且强度集合平均预报,在中心最低气压、中心最大风速、热带气旋等级都表现出明显的系统性偏弱特征;对不同级别的热带气旋强度预报,集合平均的误差随强度增强而增大,即强度集合预报对强度较弱的热带气旋有更高的准确率;对比受强、弱引导气流影响的两类热带气旋,集合平均对受弱引导气流影响的一类预报误差更小。     

Development and Application of an Atmospheric-Hydrologic-Hydraulic Flood Forecasting Model Driven by TIGGE Ensemble Forecasts

A coupled atmospheric-hydrologic-hydraulic ensemble flood forecasting model,driven by The Observing System Research and Predictability Experiment (THORPEX) Interactive Grand Global Ensemble (TIGGE) dat...     

Ensemble Forecast： A New Approach to Uncertainty and Predictability

Ensemble techniques have been used to generate daily numerical weather forecasts since the 1990s in numerical centers around the world due to the increase in computation ability. One of the main purposes of numerical ensemble forecasts is to try to assimilate the initial uncertainty （initial error） and the forecast uncertainty （forecast error） by applying either the initial perturbation method or the multi-model/multiphysics method. In fact, the mean of an ensemble forecast offers a better forecast than a deterministic （or control） forecast after a short lead time （3-5 days） for global modelling applications. There is about a 1-2-day improvement in the forecast skill when using an ensemble mean instead of a single forecast for longer lead-time. The skillful forecast （65% and above of an anomaly correlation） could be extended to 8 days （or longer） by present-day ensemble forecast systems. Furthermore, ensemble forecasts can deliver a probabilistic forecast to the users, which is based on the probability density function （PDF） instead of a single-value forecast from a traditional deterministic system. It has long been recognized that the ensemble forecast not only improves our weather forecast predictability but also offers a remarkable forecast for the future uncertainty, such as the relative measure of predictability （RMOP） and probabilistic quantitative precipitation forecast （PQPF）. Not surprisingly, the success of the ensemble forecast and its wide application greatly increase the confidence of model developers and research communities.     

Probabilistic temperature forecasting with statistical calibration in Hungary

Weather forecasting is based on the outputs of deterministic numerical weather forecasting models. Multiple runs of these models with different initial conditions result in forecast ensembles which are used for estimating the distribution of future atmospheric variables. However, these ensembles are usually under-dispersive and uncalibrated, so post-processing is required. In the present work, Bayesian model averaging (BMA) is applied for calibrating ensembles of temperature forecasts produced by the operational limited area model ensemble prediction system of the Hungarian Meteorological Service (HMS). We describe two possible BMA models for temperature data of the HMS and show that BMA post-processing significantly improves calibration and probabilistic forecasts although the accuracy of point forecasts is rather unchanged.     

一种基于数值模式诊断自适应的北京地区对流性降水临近集合预报新方法

局地触发及组织化发展中尺度系统的生消演变是影响对流性降水临近预报的核心和关键。本文结合雷达外推预报、专家系统以及快速循环更新的高分辨数值模式系统,发展和构造了一种适合北京地区的基于数值模式预报诊断自适应的对流性降水临近集合预报新方法（APEN）。APEN基于降水外推预报结果,采用模糊逻辑算法,利用北京市气象局快速循环更新同化系统（RMAPS-IN）提供的对流诊断因子,计算对流系统发展演变（新生、增加和减弱）概率;在此基础上,扰动诊断因子阈值和权重,形成对流发展的集合概率预报;最后综合专家经验,根据对流集合概率,在降水外推预报基础上进行对流性降水调整。应用APEN,针对北京两次强弱降水过程,进行了降水的临近预报试验,结果表明:基于RMAPS-IN多种诊断因子的对流发展集合概率在强弱两种天气背景下,都能较好的反映对流系统在临近时段的发展趋势;基于专家经验模型的三种对流发展状态（对流新生、增加和减弱）下的降水调整,能合理的表征对流系统发展演变对降水的影响。APEN降水预报和RMAPS-IN的业务预报的对比显示:无论是系统性对流过程还是局地激发对流过程,APEN预报的降水落区和强度都更接近于实况,尤其是考虑对流发展演变影响的降水强度预报明显优于RMAPS-IN,APEN在北京地区对流性降水的临近预报中有明显的优势和应用潜力。     

Evaluation of TIGGE Ensemble Forecasts of Precipitation in Distinct Climate Regions in Iran

The application of numerical weather prediction(NWP) products is increasing dramatically. Existing reports indicate that ensemble predictions have better skill than deterministic forecasts. In this study, numerical ensemble precipitation forecasts in the TIGGE database were evaluated using deterministic, dichotomous(yes/no), and probabilistic techniques over Iran for the period 2008–16. Thirteen rain gauges spread over eight homogeneous precipitation regimes were selected for evaluation.The Inverse Distance Weighting and Kriging methods were adopted for interpolation of the prediction values, downscaled to the stations at lead times of one to three days. To enhance the forecast quality, NWP values were post-processed via Bayesian Model Averaging. The results showed that ECMWF had better scores than other products. However, products of all centers underestimated precipitation in high precipitation regions while overestimating precipitation in other regions. This points to a systematic bias in forecasts and demands application of bias correction techniques. Based on dichotomous evaluation,NCEP did better at most stations, although all centers overpredicted the number of precipitation events. Compared to those of ECMWF and NCEP, UKMO yielded higher scores in mountainous regions, but performed poorly at other selected stations.Furthermore, the evaluations showed that all centers had better skill in wet than in dry seasons. The quality of post-processed predictions was better than those of the raw predictions. In conclusion, the accuracy of the NWP predictions made by the selected centers could be classified as medium over Iran, while post-processing of predictions is recommended to improve the quality.     

夏季亚欧中高纬度环流的集合预报效果检验

使用NCEP集合预报资料, 对亚洲中高纬地区2003年6—8月500 hPa高度场的集合预报效果进行了检验。环流预报效果检验结果表明:预报时效大于5 d时, 集合平均预报明显优于单一预报; 使用相同模式分辨率时, 集合平均能将可用预报时效延长12 h以上, 达到7.5 d; 通过集合预报可获得真正意义的概率预报结果, 取得较单一高分辨率预报好的预报效果。阻塞过程的个例分析也表明集合平均的预报效果明显优于单一确定性预报; 特征等值线可反映集合成员的不一致信息和少数集合成员的异常表现, 以此为基础, 可估计分析对象出现与否的概率, 达到提高预报效果的目的。     

A Comparison of Breeding and Ensemble Transform Vectors for Global Ensemble Generation

To compare the initial perturbation techniques using breeding vectors and ensemble transform vectors,three ensemble prediction systems using both initial perturbation methods but with different ensembl...     

A CRPS-Based Spatial Technique for the Verification of Ensemble Precipitation Forecasts

Traditional precipitation skill scores are affected by the well-known“double penalty”problem caused by the slight spatial or temporal mismatches between forecasts and observations. The fuzzy (neighborhood) method has been proposed for deterministic simulations and shown some ability to solve this problem. The increasing resolution of ensemble forecasts of precipitation means that they now have similar problems as deterministic forecasts. We developed an ensemble precipitation verification skill score, i.e., the Spatial Continuous Ranked Probability Score (SCRPS), and used it to extend spatial verification from deterministic into ensemble forecasts. The SCRPS is a spatial technique based on the Continuous Ranked Probability Score (CRPS) and the fuzzy method. A fast binomial random variation generator was used to obtain random indexes based on the climatological mean observed frequency, which were then used in the reference score to calculate the skill score of the SCRPS. The verification results obtained using daily forecast products from the ECMWF ensemble forecasts and quantitative precipitation estimation products from the OPERA datasets during June-August 2018 shows that the spatial score is not affected by the number of ensemble forecast members and that a consistent assessment can be obtained. The score can reflect the performance of ensemble forecasts in modeling precipitation and thus can be widely used.     

Preliminary Comparison of the CMA, ECMWF, NCEP, and JMA Ensemble Prediction Systems

Based on The Observing System Research and Predictability Experiment (THORPEX) Interactive Grand Global Ensemble (TIGGE) dataset,using various verification methods,the performances of four typical ense...     

Hydrological Evaluation of the Canadian Meteorological Ensemble Reforecast Product

This paper evaluates the quality of the ensemble hydrological reforecasts obtained using the 18-year ensemble meteorological reforecast dataset available from the Canadian Centre for Meteorological and Environmental Prediction (CCMEP). This study focuses on four large watersheds in the province of Quebec. A distribution-based scaling (DBS) post-processing method is used to correct the 18-year ensemble precipitation reforecasts. An Ensemble Kalman Filter (EnKF) assimilation technique is also assessed to improve the initial conditions of the hydrologic model. There is a slight improvement in performance and reliability after applying the DBS approach to precipitation reforecasts, but this technique induces a reduction in the spread. The impact of the integration of the post-processed precipitation into the hydrologic model is also quite marginal. However, the addition of an EnKF provides better ensemble hydrological reforecasts with high performance, reliability, and skill, especially in the first reforecast horizons. The best results are, however, generally obtained when using DBS and an EnKF together. Combining DBS and an EnKF, hydrological forecasts for the next two weeks are obtained using the CCMEP reforecast and also the second generation Global Ensemble Forecast System (GEFS v2) reforecast, which is considered a reference. Forecasts of comparable skill and spread are obtained, with CCMEP-based forecasts showing better spread during the first week, and GEFS v2–based reforecasts showing better skill and spread during the second week. Finally, it is shown that the two meteorological reforecast products assessed in this study have similar economic value for hydrological forecasting applications based on the cost–loss model.     

A Regional Ensemble Forecast System for Stratiform Precipitation Events in Northern China. Part I: A Case Study

A single-model,short-range,ensemble forecasting system (Institute of Atmospheric Physics,Regional Ensemble Forecast System,IAP REFS) with 15-km grid spacing,configured with multiple initial conditions,multiple lateral boundary conditions,and multiple physics parameterizations with 11 ensemble members,was developed using the Weather and Research Forecasting Model Advanced Research modeling system for prediction of stratiform precipitation events in northern China.This is the first part of a broader research project to develop a novel cloud-seeding operational system in a probabilistic framework.The ensemble perturbations were extracted from selected members of the National Center for Environmental Prediction Global Ensemble Forecasting System (NCEP GEFS) forecasts,and an inflation factor of two was applied to compensate for the lack of spread in the GEFS forecasts over the research region.Experiments on an actual stratiform precipitation case that occurred on 5-7 June 2009 in northern China were conducted to validate the ensemble system.The IAP REFS system had reasonably good performance in predicting the observed stratiform precipitation system.The perturbation inflation enlarged the ensemble spread and alleviated the underdispersion caused by parent forecasts.Centering the extracted perturbations on higher-resolution NCEP Global Forecast System forecasts resulted in less ensemble mean root-mean-square error and better accuracy in probabilistic quantitative precipitation forecasts (PQPF).However,the perturbation inflation and recentering had less effect on near-surface-level variables compared to the mid-level variables,and its influence on PQPF resolution was limited as well.     

基于非静力模式物理扰动的中尺度集合预报试验

以GRAPES中尺度有限区模式作为试验模式, 从模式的不确定性方面来构造中尺度的集合预报, 重点考虑物理因子与初始条件的扰动作用。针对2004年7月10日北京城区的突发性暴雨过程进行了36 h的集合预报试验。结果表明:GRAPES模式可有效地捕捉到中尺度过程的信息; 中尺度集合预报是可行的, 可改进中尺度暴雨过程落区、强度的预报; 不同集合方案的预报结果各不相同, 同一方案各个成员的预报结果也有差异, 即存在适宜的离散度; 在离散度分析中发现在北京附近存在一个明显大值区, 且在大气中低层的垂直结构表现出一致性, 表明这一区域的预报不确定性很大。从集合检验结果中得到:单纯考虑模式物理扰动来构造中尺度集合预报系统有一定难度, 当加入初始场不确定信息后, 同时考虑模式的不确定性和初始场的不确定性, 有助于捕捉更多的中尺度系统的不确定信息, 有助于构造更为有效的中尺度集合预报系统。     

Ensemble hindcasts of ENSO events over the past 120 years using a large number of ensembles

Based on an intermediate coupled model (ICM), a probabilistic ensemble prediction system (EPS) has been developed. The ensemble Kalman filter (EnKF) data assimilation approach is used for generating the initial ensemble conditions, and a linear, first-order Markov-Chain SST anomaly error model is embedded into the EPS to provide model-error perturbations. In this study, we perform ENSO retrospective forecasts over the 120 year period 1886–2005 using the EPS with 100 ensemble members and with initial conditi...