基于MongoDB的矢量空间数据云存储与处理系统

近年来,海量空间数据存储与处理日益成为地理信息科学领域的研究热点。其中,矢量空间数据更因其较高的复杂性,成为该类研究的重点问题。本文基于文档数据库,探究了多用户数据存储、矢量空间数据存储、海量矢量空间数据并行处理等问题,给出了存储和处理矢量空间数据的方法。在三层式云存储架构基础上,设计并实现了矢量空间数据云存储与处理系统VectorDB,达到了海量矢量空间数据的高效存储与处理要求。系统采用文档数据库MongoDB存储矢量空间数据,使用OGR库实现不同格式矢量空间数据的转换与存储,并用Hadoop对数据库中的数据进行并行计算,以及用mongo-hadoop作为MongoDB与Hadoop之间的连接器。通过实验对比了VectorDB与PostGIS的矢量空间数据读写性能,并分析了VectorDB与MongoDB在海量数据并行处理性能方面的差异。结果表明:VectorDB具有更好的读取性能和海量数据处理性能,适合多用户不同格式、不同属性矢量空间数据存储,对海量矢量数据存储与处理问题具有参考价值。

Vector Spatial Data Cloud Storage and Processing Based on MongoDB

With the rapidly growing volume and complexity of spatial data, efficient storage and processing of massive geo-spatial data have become urgent research problems in GIScience and related fields. Vector spatial data storage and processing is particularly challenging due to its complexity in data representation, access and analysis. In this paper, we present an approach and a system for cloud-based vector data storage and analysis,with the ability to support multi-user access and parallel processing. Our system, named VectorDB, extends MongoDB（a document-oriented NoSQL database system） and integrates the Hadoop framework for parallel spatial data processing and analysis. With a three-layered browser-server architecture, the system consists of a suite of modules for data storage, conversion, query, and analysis. The OGR Simple Features Library is integrated to perform data conversions between MongoDB and various formats of vector spatial data. We use the MongoDB Connector for Hadoop（mongo-hadoop） to transfer data between MongoDB and Hadoop. An experiment is carried out using five physical servers to compare the performance of VectorDB and PostGIS for vector data reading,writing, and query. Preliminary results indicate that, although VectorDB is slightly slower in data writing, it gains significant power for data access and spatial query over PostGIS. We also compared VectorDB and MongoDB for massive vector data processing. Results show that VectorDB has a better performance than MongoDB in massive vector data processing. VectorDB is different from the traditional relational spatial database, and it can support dynamic schema and thus is much more flexible and effective for storing, accessing, and analyzing various vector spatial data models and data formats. Our approach and implemented system will be useful for a variety of applications that need to store and access vector spatial data in a cloud environment.