| Storm流式技术在地面气象数据处理中的应用 |
| |
| 作者姓名： | 廖婷婷 王彪 肖卫青 李从英 郭茜 |
| |
| 作者单位： | 贵州省气象信息中心,贵州贵阳,550002;国家气象信息中心,北京,100081 |
| |
| 基金项目： | 贵州省科技计划项目([2019]2386): 基于大数据的气象防灾减灾服务平台研发 |
| |
| 摘 要： | 地面自动站气象数据的量级、观测传输频次日益增长,传输方式从传统的文本方式发展到消息传输,对数据处理时效有了更高的要求。该文提出采用Storm流式技术进行地面自动站气象数据的解码和入库,使用Spout节点连接外部数据源,将数据转化为Tuple,传递给解码Bolt、入库Bolt和DI/EI消息Bolt,分别进行实时的入库和监控,且达到灵活调整系统资源的目的。该应用与传统的简约流程对比,稳定性、处理效率、拓展性都有了显著提高。
|
| 关 键 词： | 地面气象数据 海量数据 Storm流式技术 高效处理 |
| 收稿时间： | 2019/4/23 0:00:00 |
| 修稿时间： | 2019/8/5 0:00:00 |
Application of Storm Flow Technology in Surface Meteorological Data Processing |
| |
| Authors: | Liao Tingting Wang Biao Xiao Weiqing Li Congying and Guo Xi |
| |
| Institution: | Guizhou Meteorological Information Center,Guiyang,Guizhou Meteorological Information Center,Guiyang,National Meteorological Information Center,Beijing,Guizhou Meteorological Information Center,Guiyang,Guizhou Meteorological Information Center,Guiyang |
| |
| Abstract: | The quantity and observational frequency of meteorological data by surficial automatic station are increasing. The transmission mode has developed from the traditional text mode to the message transmission mode. These changes make higher requirement for data processing timeliness. In this paper, we used Storm Flow Technology on decoding and warehouse of surficial automatic station data. It used Spout to connect external data sources, and convert data into Tuple, and then transmit to Bolt-decode, Bolt-warehouse and Bolt-DIEI. They can be used to real-time warehouse, monitoring, and quickly adjust system resource. Compared with the traditional simplified process, the stability, processing efficiency and expansibility of the application have been greatly improved. |
| |
| Keywords: | surface meteorological data big data storm flow technology high-activity processing |
| 本文献已被 CNKI 万方数据 等数据库收录！ |
| 点击此处可从《》浏览原始摘要信息 |
| 点击此处可从《》下载免费的PDF全文 |
|
