| 基于Spark Streaming的实时流数据处理模型化研究与实现 |
| |
| 引用本文： | 云惟英,苟宇,王京,王丽莉.基于Spark Streaming的实时流数据处理模型化研究与实现[J].测绘与空间地理信息,2017,40(Z1). |
| |
| 作者姓名： | 云惟英 苟宇 王京 王丽莉 |
| |
| 作者单位： | 1. 北京超图软件股份有限公司,北京100015;北京市地理信息核心软件与应用工程技术研究中心,北京100015;2. 北京超图软件股份有限公司,北京,100015 |
| |
| 摘 要： | 通过研究与分析,选取Spark Streaming技术实现对P实时流数据的处理.同时,研究出一套模型化的方式,实现动态装配软件的执行过程;并通过具体的实例展示了两者结合后,在数据处理的易用性、性能及吞吐量方面,都得到了大幅提升.
|
| 关 键 词： | Spark Streaming 空间大数据 时实流数据 |
Research and Implementation of the Modules Processing of Real Time Data Stream Based on Spark Streaming |
| |
| YUN Wei-ying,GOU Yu,WANG Jing,WANG Li-li.Research and Implementation of the Modules Processing of Real Time Data Stream Based on Spark Streaming[J].Geomatics & Spatial Information Technology,2017,40(Z1). |
| |
| Authors: | YUN Wei-ying GOU Yu WANG Jing WANG Li-li |
| |
| Abstract: | In this paper, Apache Spark Streaming is chosen to process real-time data stream after analyzing multiple technologies of distributed streaming computing. In addition, a set of modules are proposed to build the program for processing real-time data stream dynamically. Then an application example is shared based on the combination of these modules and Spark Streaming to prove the in-crease usability of data processing, and the substantially improved performance and throughput. |
| |
| Keywords: | Spark Streaming real-time stream data spatial big data |
| 本文献已被 万方数据 等数据库收录！ |
|
