Comparison of prediction performance using statistical postprocessing methods

As the 2018 Winter Olympics are to be held in Pyeongchang, both general weather information on Pyeongchang and specific weather information on this region, which can affect game operation and athletic performance, are required. An ensemble prediction system has been applied to provide more accurate weather information, but it has bias and dispersion due to the limitations and uncertainty of its model. In this study, homogeneous and nonhomogeneous regression models as well as Bayesian model averaging (BMA) were used to reduce the bias and dispersion existing in ensemble prediction and to provide probabilistic forecast. Prior to applying the prediction methods, reliability of the ensemble forecasts was tested by using a rank histogram and a residualquantile-quantile plot to identify the ensemble forecasts and the corresponding verifications. The ensemble forecasts had a consistent positive bias, indicating over-forecasting, and were under-dispersed. To correct such biases, statistical post-processing methods were applied using fixed and sliding windows. The prediction skills of methods were compared by using the mean absolute error, root mean square error, continuous ranked probability score, and continuous ranked probability skill score. Under the fixed window, BMA exhibited better prediction skill than the other methods in most observation station. Under the sliding window, on the other hand, homogeneous and non-homogeneous regression models with positive regression coefficients exhibited better prediction skill than BMA. In particular, the homogeneous regression model with positive regression coefficients exhibited the best prediction skill.     

地面气象要素多模式集成预报研究进展

目前,集合预报已成为天气预报业务的主要支撑。然而,由于数值模式本身的限制与不完善以及集合系统存在初值扰动、集合大小等方面的局限,常存在预报偏差。不同预报模式通常具有不同的物理过程参数化方案、初始条件等,导致其预报能力各有不同。为此,如何纠正预报偏差以及如何充分有效地利用不同模式的预报信息以获得更加准确的天气预报广受关注。近年来,利用统计理论与预报诊断,基于多个集合预报系统的多模式集成预报技术得到快速发展,已成为有效消除预报偏差从而提高天气预报技巧的一种统计后处理方法。针对气温、降水和风3个最基本的地面气象要素,首先依据预报形式将应用范围较广的简单集合平均、消除偏差集合平均、超级集合、贝叶斯模式平均、集合模式输出统计等加权或等权平均多模式集成技术,分成确定性预报和概率预报两大类,并做系统介绍。最后,讨论使用和发展多模式集成技术需要关注的问题,包括考虑参与集成的模式个数、发展降水及风速分级预报模型和发展基于机器学习的多模式集成新技术。     

基于TIGGE多模式集合的24小时气温BMA 概率预报

利用TIGGE(THORPEX Interactive Grand Global Ensemble)单中心集合预报系统(ECMWF、United Kingdom Meteorological Office、China Meteorological Administration和NCEP)以及由此所构成的多中心模式超级集合预报系统24小时地面日均气温预报,结合淮河流域地面观测率定贝叶斯模型平均(Bayesian model averaging,BMA)参数,从而建立地面日均气温BMA概率预报模型.由此针对淮河流域进行地面日均气温BMA概率预报及其检验与评估,结果表明BMA模型比原始集合预报效果好;单中心的BMA概率预报都有较好的预报效果,其中ECMWF最好.多中心模式超级集合比单中心BMA概率预报效果更好,采用可替换原则比普通的多中心模式超级集合BMA模型计算量小,且在上述BMA集合预报系统中效果最好.它与原始集合预报相比其平均绝对误差减少近7%,其连续等级概率评分提高近10%.基于采用可替换原则的多中心模式超级集合BMA概率预报,针对研究区域提出了极端高温预警方案,这对防范高温天气有着重要意义.     

Intercomparison of prediction skills of ensemble methods using monthly mean temperature simulated by CMIP5 models

This study focuses on an objective comparison of eight ensemble methods using the same data, training period, training method, and validation period. The eight ensemble methods are: BMA (Bayesian Model Averaging), HMR (Homogeneous Multiple Regression), EMOS (Ensemble Model Output Statistics), HMR+ with positive coefficients, EMOS+ with positive coefficients, PEA_ROC (Performance-based Ensemble Averaging using ROot mean square error and temporal Correlation coefficient), WEA_Tay (Weighted Ensemble Averaging based on Taylor’s skill score), and MME (Multi-Model Ensemble). Forty-five years (1961-2005) of data from 14 CMIP5 models and APHRODITE (Asian Precipitation- Highly-Resolved Observational Data Integration Towards Evaluation of Water Resources) data were used to compare the performance of the eight ensemble methods. Although some models underestimated the variability of monthly mean temperature (MMT), most of the models effectively simulated the spatial distribution of MMT. Regardless of training periods and the number of ensemble members, the prediction skills of BMA and the four multiple linear regressions (MLR) were superior to the other ensemble methods (PEA_ROC, WEA_Tay, MME) in terms of deterministic prediction. In terms of probabilistic prediction, the four MLRs showed better prediction skills than BMA. However, the differences among the four MLRs and BMA were not significant. This resulted from the similarity of BMA weights and regression coefficients. Furthermore, prediction skills of the four MLRs were very similar. Overall, the four MLRs showed the best prediction skills among the eight ensemble methods. However, more comprehensive work is needed to select the best ensemble method among the numerous ensemble methods.     

Probabilistic temperature forecasting with statistical calibration in Hungary

Weather forecasting is based on the outputs of deterministic numerical weather forecasting models. Multiple runs of these models with different initial conditions result in forecast ensembles which are used for estimating the distribution of future atmospheric variables. However, these ensembles are usually under-dispersive and uncalibrated, so post-processing is required. In the present work, Bayesian model averaging (BMA) is applied for calibrating ensembles of temperature forecasts produced by the operational limited area model ensemble prediction system of the Hungarian Meteorological Service (HMS). We describe two possible BMA models for temperature data of the HMS and show that BMA post-processing significantly improves calibration and probabilistic forecasts although the accuracy of point forecasts is rather unchanged.     

多模式集成的概率天气预报和气候预测研究进展

基于大气的混沌特性,单一的确定性预报逐步向多值的不确定性概率预报转化已成为一种趋势。本文系统地评述了概率天气预报产生的背景,介绍了概率预报的相关概念及国内外的研究状况,着重讨论了多模式集成的概率预报的两种集成方法,即贝叶斯模式平均(Bayesian model averaging,BMA)和多元高斯集合核拟合法(Gaussian ensemble kernel dressing,GEKD),并给出了两个例子的概率预报试验结果。利用BMA方法制作的概率预报的方差较小,减小了预报的不确定性,因此预报结果更接近大气的真实值。作为另一种多模式集成方法,多元高斯集合核拟合法回报的地面气温距平均值及趋势的概率预测结果与实测结果基本一致。利用此方法建立了地面气温年代际变化的概率多模式集合预测模型,并从中提取年代际气候变化特征,对东亚季风区年代际预测具有重要应用价值。     

集合模式定量降水预报的统计后处理技术研究综述

集合数值模式预报已在定量降水预报业务中广泛应用,以获得预报不确定性、最可能预报结果以及极端天气预警。由于集合系统的数值模式不完善,且不能提供所有的不确定性信息,常表现出系统性偏差以及欠离散或过离散（如对于多模式集合）。为此,需要发展统计后处理技术,在尽量保持集合预报解析度的条件下,提高预报的技巧和可靠性。近年来,各种集合预报统计后处理技术得到快速发展。针对定量降水预报,依据技术方法的途径和成熟度将后处理研究归纳为3方面进行总结,包括:（1）不基于统计模型的非参数化后处理,包括集合定量降水预报偏差订正、多成员或模式信息集成以及基于空间分析的对流尺度模式后处理;（2）基于概率分布统计模型的参数化后处理,包括集合模式输出统计和贝叶斯模型平均两种方法框架;（3）考虑预报量的时间、空间和多变量间依赖关系或结构的处理方法,包括参数化和经验连接概率法。最后,讨论发展统计后处理技术需要关注的问题,包括考虑不同来源、不同尺度的多模式信息集成;提供高质量、高分辨率的降水分析资料;发展再预报技术扩充训练样本;基于不同的订正目的和应用场景来使用不同的后处理技术;发展面向海量预报数据、捕捉极端降水以及考虑预报量结构的新技术。      

Prospects of using Bayesian model averaging for the calibration of one-month forecasts of surface air temperature over South Korea

In this study, we investigated the prospect of calibrating probabilistic forecasts of surface air temperature (SAT) over South Korea by using Bayesian model averaging (BMA). We used 63 months of simulation results from four regional climate models (RCMs) with two boundary conditions (NCEP-DOE and ERA-interim) over the CORDEX East Asia. Rank histograms and residual quantile-quantile (R-Q-Q) plots showed that the simulation skills of the RCMs differ according to season and geographic location, but the RCMs show a systematic cold bias irrespective of season and geographic location. As a result, the BMA weights are clearly dependent on geographic location, season, and correlations among the models. The one-month equal weighted ensemble (EWE) outputs for the 59 stations over South Korea were calibrated using the BMA method for 48 monthly time periods based on BMA weights obtained from the previous 15 months of training data. The predictive density function was calibrated using BMA and the individual forecasts were weighted according to their performance. The raw ensemble forecasts were assessed using the flatness of the rank histogram and the R-Q-Q plot. The results showed that BMA improves the calibration of the EWE and the other weighted ensemble forecasts irrespective of season, simulation skill of the RCM, and geographic location. In addition, deterministic-style BMA forecasts usually perform better than the deterministic forecast of the single best member.     

Probabilistic Precipitation Forecasting Based on Ensemble Output Using Generalized Additive Models and Bayesian Model Averaging

A probabilistic precipitation forecasting model using generalized additive models (GAMs) and Bayesian model averaging (BMA) was proposed in this paper.GAMs were used to fit the spatial-temporal precipi...     

基于贝叶斯理论的单站地面气温的概率预报研究

基于贝叶斯理论,建立了将确定性预报向概率预报转换的基本模式,并利用TIGGE资料中欧洲中期天气预报中心(ECMWF)地面气温预报资料及地面气温观测资料,对概率化后的预报进行了评估与释用。结果表明,概率化后的预报结果不但能提供丰富的预报产品,而且所提供的预报均值优于原始的确定性预报。应用贝叶斯模式平均法(BMA)将中国气象局(CMA)、美国国家环境预报中心(NCEP)和ECMWF 3个模式的预报结果进行多模式集成,得到了更为合理的概率分布,其中分布的均值可作为模式的预报结果,方差和置信区间反映了预报量的可变范围。因此,基于贝叶斯预报模式的概率预报相对于确定性预报,不但能够提供更高精度的预报,而且能提供更全面的预报信息。BMA集成预报结果不但优于集合平均预报,而且还能定量描述预报的不确定性。利用ECMWF预报中心51个预报成员进行集成贝叶斯概率预报试验,发现BMA预报融合了各成员对预报不确定性的描述,还对概率预报的均值进行了调整,使之与观测值更为接近。BMA预报的概率密度分布更能反映大气的真实分布情况。     

Probabilistic prediction of Indian summer monsoon rainfall using global climate models

Probabilistic seasonal predictions of rainfall that incorporate proper uncertainties are essential for climate risk management. In this study, three different multi-model ensemble (MME) approaches are used to generate probabilistic seasonal hindcasts of the Indian summer monsoon rainfall based on a set of eight global climate models for the 1982–2009 period. The three MME approaches differ in their calculation of spread of the forecast distribution, treated as a Gaussian, while all three use the simple multi-model subdivision average to define the mean of the forecast distribution. The first two approaches use the within-ensemble spread and error residuals of ensemble mean hindcasts, respectively, to compute the variance of the forecast distribution. The third approach makes use of the correlation between the ensemble mean hindcasts and the observations to define the spread using a signal-to-noise ratio. Hindcasts are verified against high-resolution gridded rainfall data from India Meteorological Department in terms of meteorological subdivision spatial averages. The use of correlation for calculating the spread provides better skill than the other two methods in terms of rank probability skill score. In order to further improve the skill, an additional method has been used to generate multi-model probabilistic predictions based on simple averaging of tercile category probabilities from individual models. It is also noted that when such a method is used, skill of probabilistic forecasts is improved as compared with using the multi-model ensemble mean to define the mean of the forecast distribution and then probabilities are estimated. However, skill of the probabilistic predictions of the Indian monsoon rainfall is too low.     

冬奥延庆赛区风的预报技巧比较分析

为了提高2 min平均的10 m风预报精度,开展了多种建模和检验方法比较.根据欧洲数值中心集合预报系统产品及北京海陀山的5个测站资料,使用一元回归、岭回归、神经网络、粒子群-神经网络等方法建模,进行2021年2月逐日的未来3天6 h间隔预报误差订正,并从多个角度分析预报精度差异.结果为:(1)系统误差、预报准确率检验表...     

Improving Multi-Model Ensemble Forecasts of Tropical Cyclone Intensity Using Bayesian Model Averaging

This paper proposes a method for multi-model ensemble forecasting based on Bayesian model averaging (BMA), aiming to improve the accuracy of tropical cyclone (TC) intensity forecasts, especially forecasts of minimum surface pressure at the cyclone center (P_min). The multi-model ensemble comprises three operational forecast models: the Global Forecast System (GFS) of NCEP, the Hurricane Weather Research and Forecasting (HWRF) models of NCEP, and the Integrated Forecasting System (IFS) of ECMWF. The mean of a predictive distribution is taken as the BMA forecast. In this investigation, bias correction of the minimum surface pressure was applied at each forecast lead time, and the distribution (or probability density function, PDF) of P_min was used and transformed. Based on summer season forecasts for three years, we found that the intensity errors in TC forecast from the three models varied significantly. The HWRF had a much smaller intensity error for short lead-time forecasts. To demonstrate the proposed methodology, cross validation was implemented to ensure more efficient use of the sample data and more reliable testing. Comparative analysis shows that BMA for this three-model ensemble, after bias correction and distribution transformation, provided more accurate forecasts than did the best of the ensemble members (HWRF), with a 5%–7% decrease in root-mean-square error on average. BMA also outperformed the multi-model ensemble, and it produced “predictive variance” that represented the forecast uncertainty of the member models. In a word, the BMA method used in the multi-model ensemble forecasting was successful in TC intensity forecasts, and it has the potential to be applied to routine operational forecasting.     

Ensemble hindcasts of ENSO events over the past 120 years using a large number of ensembles

Based on an intermediate coupled model (ICM), a probabilistic ensemble prediction system (EPS) has been developed. The ensemble Kalman filter (EnKF) data assimilation approach is used for generating the initial ensemble conditions, and a linear, first-order Markov-Chain SST anomaly error model is embedded into the EPS to provide model-error perturbations. In this study, we perform ENSO retrospective forecasts over the 120 year period 1886–2005 using the EPS with 100 ensemble members and with initial conditi...     

东亚地区冬季地面气温延伸期概率预报研究

利用TIGGE资料中的ECMWF、NCEP、UKMO三个中心集合预报系统以及由此构成的多中心集合预报系统所提供的地面2 m气温10~15 d延伸期集合预报产品,建立贝叶斯模式平均(Bayesian Model Averaging,BMA)概率预报模型,对东亚地区冬季地面气温进行延伸期概率预报研究。采用距平相关系数、均方根误差、布莱尔评分、等级概率评分等指标分别对BMA确定性结果与概率预报进行评估。结果表明,BMA方法明显地改进了原始集合预报结果,预报技巧优于原始集合预报,且多中心BMA预报优于单中心BMA预报,最佳滑动训练期取35 d。BMA预报为气温的延伸期概率预报提供了更合理的概率分布,定量描述了预报的不确定性。     

基于TIGGE资料的地面气温多模式超级集合预报

基于TIGGE资料, 采用均方根误差分别对欧洲中期天气预报中心、日本气象厅、美国国家环境预报中心和英国气象局4个中心集合预报的地面气温场集合平均结果进行检验评估, 比较各中心地面气温的预报效果。并利用超级集合、多模式集合平均和消除偏差集合平均3种方法对4个中心的地面气温预报进行集成, 同时对预报结果进行分析。结果表明: 2007年夏季日本气象厅与欧洲中期天气预报中心在北半球大部分地区预报效果最好, 各中心在不同地区预报效果不同。超级集合与消除偏差集合平均降低了预报误差, 预报效果优于最好的单个中心预报和多模式集合平均。对于较长的预报时效, 消除偏差集合平均表现出了更好的预报性能。     

一个集合海浪预报系统及其初步试验

使用集合天气预报系统的多个成员的风场预报来驱动海浪模式WAVEWATCH Ⅲ, 计算出含多个成员的海浪预报场,并相应开发出各海浪要素的集合预报产品,如集合平均、离散度、集合概率等,建立了一个集合海浪数值预报系统。使用该系统进行了2007年9—10月为期两个月的预报试验,利用太平洋和大西洋海域范围的浮标观测资料对系统的预报水平的初步检验分析显示,该集合海浪预报方法能够有效地将传统的确定性预报扩展到概率预报领域,且集合平均的预报水平要优于单一的确定性预报,采用集合预报方法可以提供单纯确定性预报所不能够提供的额外信息,具有较好的应用潜力。     

Evaluation of summer temperature and precipitation predictions from NCEP CFSv2 retrospective forecast over China

National Centers for Environmental Prediction recently upgraded its operational seasonal forecast system to the fully coupled climate modeling system referred to as CFSv2. CFSv2 has been used to make seasonal climate forecast retrospectively between 1982 and 2009 before it became operational. In this study, we evaluate the model’s ability to predict the summer temperature and precipitation over China using the 120 9-month reforecast runs initialized between January 1 and May 26 during each year of the reforecast period. These 120 reforecast runs are evaluated as an ensemble forecast using both deterministic and probabilistic metrics. The overall forecast skill for summer temperature is high while that for summer precipitation is much lower. The ensemble mean reforecasts have reduced spatial variability of the climatology. For temperature, the reforecast bias is lead time-dependent, i.e., reforecast JJA temperature become warmer when lead time is shorter. The lead time dependent bias suggests that the initial condition of temperature is somehow biased towards a warmer condition. CFSv2 is able to predict the summer temperature anomaly in China, although there is an obvious upward trend in both the observation and the reforecast. Forecasts of summer precipitation with dynamical models like CFSv2 at the seasonal time scale and a catchment scale still remain challenge, so it is necessary to improve the model physics and parameterizations for better prediction of Asian monsoon rainfall. The probabilistic skills of temperature and precipitation are quite limited. Only the spatially averaged quantities such as averaged summer temperature over the Northeast China of CFSv2 show higher forecast skill, of which is able to discriminate between event and non-event for three categorical forecasts. The potential forecast skill shows that the above and below normal events can be better forecasted than normal events. Although the shorter the forecast lead time is, the higher deterministic prediction skill appears, the probabilistic prediction skill does not increase with decreased lead time. The ensemble size does not play a significant role in affecting the overall probabilistic forecast skill although adding more members improves the probabilistic forecast skill slightly.     

区域集合预报系统2 m温度预报的校准技术

采用非齐次高斯回归 (NGR) 技术对国家气象中心区域集合预报系统的2 m温度预报结果开展了一阶偏差和二阶离散度的校准研究。对预报结果比较详尽的检验分析表明:校准后的2 m温度预报可靠性和预报技巧均显著提高,表现为校准后集合预报成员的均方根误差与离散度更为接近;原Talagrand直方图中的“L”形分布现象得到有效改善;Brier评分、最小连续分级概率评分 (CRPS) 明显减小,相对作用特征 (ROC) 面积增大,说明校准后的2 m温度预报表现出更好的预报技能。此外,NGR技术与自适应误差订正技术的对比试验表明,NGR在消除集合平均偏差和提高集合离散度两个方面均有优势。     

中国东南地区复杂地形下降水概率预报的订正研究

使用TIGGE (the THORPEX interactive grand global ensemble)资料集下欧洲中期天气预报中心(the European Centre for Medium-Range Weather Forecasts,ECMWF)逐日起报的预报时效为24～168 h的日降水量集合预报资料,集合预报共包括51个成员,利用左删失的非齐次Logistic回归方法(left-Censored Non-homogeneous Logistic Regression,CNLR)和标准化的模式后处理方法(Standardized Anomaly Model Output Statistics,SAMOS)对具有复杂地形的中国东南部地区降水预报进行统计后处理。结果表明:采用CNLR方法能够有效改进原始集合预报的平均绝对误差(Mean Absolute Error,MAE)和连续分级概率评分(Continuous Ranked Probability Score,CRPS),提升了降水的定量预报和概率预报的预报技巧。而使用SAMOS方法对数据进行预处理,考虑地形等因素的影响,能在CNLR方法的基础上进一步订正由于地形影响造成的预报误差,并得到更加准确的全概率的降水概率预报。