Geospatial web services pave new ways for server-based on-demand access and processing of Big Earth Data

ABSTRACT

Big Earth Data has experienced a considerable increase in volume in recent years due to improved sensing technologies and improvement of numerical-weather prediction models. The traditional geospatial data analysis workflow hinders the use of large volumes of geospatial data due to limited disc space and computing capacity. Geospatial web service technologies bring new opportunities to access large volumes of Big Earth Data via the Internet and to process them at server-side. Four practical examples are presented from the marine, climate, planetary and earth observation science communities to show how the standard interface Web Coverage Service and its processing extension can be integrated into the traditional geospatial data workflow. Web service technologies offer a time- and cost-effective way to access multi-dimensional data in a user-tailored format and allow for rapid application development or time-series extraction. Data transport is minimised and enhanced processing capabilities are offered. More research is required to investigate web service implementations in an operational mode and large data centres have to become more progressive towards the adoption of geo-data standard interfaces. At the same time, data users have to become aware of the advantages of web services and be trained how to benefit from them most.     

基于Spark的分布式空间数据存储结构设计与实现

Apache Spark分布式计算框架可用于空间大数据的管理与计算,为实现云GIS提供基础平台。针对Apache Spark的数据组织与计算模型,结合Apache HBase分布式数据库,从分布式GIS内核的理念出发,设计并实现了分布式空间数据存储结构与对象接口,并基于某国产GIS平台软件内核进行了实现。针对点、线、面数据的存储与查询,与传统空间数据库系统PostGIS进行了一系列对比实验,验证了提出的分布式空间数据存储架构的可行性与高效性。     

A geospatial hybrid cloud platform based on multi-sourced computing and model resources for geosciences

Cloud computing has been considered as the next-generation computing platform with the potential to address the data and computing challenges in geosciences. However, only a limited number of geoscientists have been adapting this platform for their scientific research mainly due to two barriers: 1) selecting an appropriate cloud platform for a specific application could be challenging, as various cloud services are available and 2) existing general cloud platforms are not designed to support geoscience applications, algorithms and models. To tackle such barriers, this research aims to design a hybrid cloud computing (HCC) platform that can utilize and integrate the computing resources across different organizations to build a unified geospatial cloud computing platform. This platform can manage different types of underlying cloud infrastructure (e.g., private or public clouds), and enables geoscientists to test and leverage the cloud capabilities through a web interface. Additionally, the platform also provides different geospatial cloud services, such as workflow as a service, on the top of common cloud services (e.g., infrastructure as a service) provided by general cloud platforms. Therefore, geoscientists can easily create a model workflow by recruiting the needed models for a geospatial application or task on the fly. A HCC prototype is developed and dust storm simulation is used to demonstrate the capability and feasibility of such platform in facilitating geosciences by leveraging across-organization computing and model resources.     

A Multi‐granularity Parallel Model for Unified Remote Sensing Image Processing WebServices

The growth of the Web has resulted in the Web‐based sharing of distributed geospatial data and computational resources. The Geospatial Processing Web (GeoPW) described here is a set of services that provide a wide array of geo‐processing utilities over the Web and make geo‐processing functionalities easily accessible to users. High‐performance remote sensing image processing is an important component of the GeoPW. The design and implementation of high‐performance image processing are, at present, an actively pursued research topic. Researchers have proposed various parallel strategies for single image processing algorithm, based on a computer science approach to parallel processing. This article proposes a multi‐granularity parallel model for various remote sensing image processing algorithms. This model has four hierarchical interfaces that are labeled the Region of Interest oriented (ROI‐oriented), Decompose/Merge, Hierarchical Task Chain and Dynamic Task interfaces or sub‐models. In addition, interfaces, definitions, parallel task scheduling and fault‐tolerance mechanisms are described in detail. Based on the model and methods, we propose an open‐source online platform named OpenRS‐Cloud. A number of parallel algorithms were uniformly and efficiently developed, thus certifying the validity of the multi‐granularity parallel model for unified remote sensing image processing web services.     

论万物互联时代的地球空间信息学

随着5G/6G、云计算、物联网和人工智能等新技术的发展,人类已经进入了万物互联时代。本文探讨万物互联时代地球空间信息技术的五大特点：定位技术从GNSS和地面测量走向无所不在的定位导航定时（PNT）服务体系;遥感技术从孤立的遥感卫星走向空天地传感网络;地理信息服务从地图数据库为主走向真三维实景和数字孪生;3S集成从移动测量发展到智能机器人服务;学科研究范围从对地观测走向物联监测和对人类活动的感知。笔者基于这些特点进一步剖析新时代面临的挑战,并提出新时代地球空间信息学发展亟待解决的三大科学技术问题：测绘学科如何服务人与机器人的共同需求？遥感影像解译的机理是什么和如何突破实现技术的瓶颈？如何利用时空大数据挖掘人与自然的关系,从空间感知走向空间认知？万物互联时代的地球空间信息学,必须且完全可能为万物互联的数字地球和智慧社会做出更大的贡献！     

Harnessing the power of immersive virtual reality - visualization and analysis of 3D earth science data sets

ABSTRACT

The availability and quantity of remotely sensed and terrestrial geospatial data sets are on the rise. Historically, these data sets have been analyzed and quarried on 2D desktop computers; however, immersive technologies and specifically immersive virtual reality (iVR) allow for the integration, visualization, analysis, and exploration of these 3D geospatial data sets. iVR can deliver remote and large-scale geospatial data sets to the laboratory, providing embodied experiences of field sites across the earth and beyond. We describe a workflow for the ingestion of geospatial data sets and the development of an iVR workbench, and present the application of these for an experience of Iceland’s Thrihnukar volcano where we: (1) combined satellite imagery with terrain elevation data to create a basic reconstruction of the physical site; (2) used terrestrial LiDAR data to provide a geo-referenced point cloud model of the magmatic-volcanic system, as well as the LiDAR intensity values for the identification of rock types; and (3) used Structure-from-Motion (SfM) to construct a photorealistic point cloud of the inside volcano. The workbench provides tools for the direct manipulation of the georeferenced data sets, including scaling, rotation, and translation, and a suite of geometric measurement tools, including length, area, and volume. Future developments will be inspired by an ongoing user study that formally evaluates the workbench’s mature components in the context of fieldwork and analyses activities.     

An automated and optimized approach for online spatial biodiversity model: a case study of OGC web processing service

An online spatial biodiversity model (SBM) for optimized and automated spatial modelling and analysis of geospatial data is proposed, which is based on web processing service (WPS) and web service orchestration (WSO) in parallel computing environment. The developed model integrates distributed geospatial data in geoscientific processing workflow to compute the algorithms of spatial landscape indices over the web using free and open source software. A case study for Uttarakhand state of India demonstrates the model outputs such as spatial biodiversity disturbance index (SBDI) and spatial biological richness index (SBRI). In order to optimize and automate, an interactive web interface is developed using participatory GIS approaches for implementing fuzzy AHP. In addition, sensitivity analysis and geosimulation experiments are also performed under distributed GIS environment. Results suggest that parallel algorithms in SBM execute faster than sequential algorithms and validation of SBRI with biological diversity shows significant correlation by indicating high R² values.     

Geocomputation over the Emerging Heterogeneous Computing Infrastructure

Emerging computer architectures and systems that combine multi‐core CPUs and accelerator technologies, like many‐core Graphic Processing Units (GPUs) and Intel's Many Integrated Core (MIC) coprocessors, would provide substantial computing power for many time‐consuming spatial‐temporal computation and applications. Although a distributed computing environment is suitable for large‐scale geospatial computation, emerging advanced computing infrastructure remains unexplored in GIScience applications. This article introduces three categories of geospatial applications by effectively exploiting clusters of CPUs, GPUs and MICs for comparative analysis. Within these three benchmark tests, the GPU clusters exemplify advantages in the use case of embarrassingly parallelism. For spatial computation that has light communication between the computing nodes, GPU clusters present a similar performance to that of the MIC clusters when large data is applied. For applications that have intensive data communication between the computing nodes, MIC clusters could display better performance than GPU clusters. This conclusion will be beneficial to the future endeavors of the GIScience community to deploy the emerging heterogeneous computing infrastructure efficiently to achieve high or better performance spatial computation over big data.     

全球电离层模型的分布式并行解算

为满足GNSS数据处理效率不断提升的需求,提出并开发了一套分布式并行计算框架,并基于该框架实现了全球电离层模型的分布式并行解算。采用2台服务器和4台台式机,对全球电离层建模分别测试了单机多线程、多机分布式等并行计算方案,并分析了不同方案建模的数据处理效率。结果表明,采用多线程并行计算可以大幅提高数据处理效率,且当开启线程数与计算机CPU核心数一致时效率提升最佳;采用多机分布式并行计算可进一步提高数据处理效率,使用4台台式机相对于单台台式机解算时间减少约60%,使用2台服务器相对于单台服务器解算时间减少约18%;采用分布式并行计算方案,可充分利用多台计算机资源来提高全球电离层建模效率,对电离层产品快速发布、建模算法的测试等具有重要的意义,对多系统GNSS精密定轨与定位、大网解算也具有很好的参考价值。     

Big Data and cloud computing: innovation opportunities and challenges

ABSTRACT

Big Data has emerged in the past few years as a new paradigm providing abundant data and opportunities to improve and/or enable research and decision-support applications with unprecedented value for digital earth applications including business, sciences and engineering. At the same time, Big Data presents challenges for digital earth to store, transport, process, mine and serve the data. Cloud computing provides fundamental support to address the challenges with shared computing resources including computing, storage, networking and analytical software; the application of these resources has fostered impressive Big Data advancements. This paper surveys the two frontiers – Big Data and cloud computing – and reviews the advantages and consequences of utilizing cloud computing to tackling Big Data in the digital earth and relevant science domains. From the aspects of a general introduction, sources, challenges, technology status and research opportunities, the following observations are offered: (i) cloud computing and Big Data enable science discoveries and application developments; (ii) cloud computing provides major solutions for Big Data; (iii) Big Data, spatiotemporal thinking and various application domains drive the advancement of cloud computing and relevant technologies with new requirements; (iv) intrinsic spatiotemporal principles of Big Data and geospatial sciences provide the source for finding technical and theoretical solutions to optimize cloud computing and processing Big Data; (v) open availability of Big Data and processing capability pose social challenges of geospatial significance and (vi) a weave of innovations is transforming Big Data into geospatial research, engineering and business values. This review introduces future innovations and a research agenda for cloud computing supporting the transformation of the volume, velocity, variety and veracity into values of Big Data for local to global digital earth science and applications.     

A scalable cyberinfrastructure and cloud computing platform for forest aboveground biomass estimation based on the Google Earth Engine

ABSTRACT

Earth observation (EO) data, such as high-resolution satellite imagery or LiDAR, has become one primary source for forests Aboveground Biomass (AGB) mapping and estimation. However, managing and analyzing the large amount of globally or locally available EO data remains a great challenge. The Google Earth Engine (GEE), which leverages cloud-computing services to provide powerful capabilities on the management and rapid analysis of various types of EO data, has appeared as an inestimable tool to address this challenge. In this paper, we present a scalable cyberinfrastructure for on-the-fly AGB estimation, statistics, and visualization over a large spatial extent. This cyberinfrastructure integrates state-of-the-art cloud computing applications, including GEE, Fusion Tables, and the Google Cloud Platform (GCP), to establish a scalable, highly extendable, and high-performance analysis environment. Two experiments were designed to demonstrate its superiority in performance over the traditional desktop environment and its scalability in processing complex workflows. In addition, a web portal was developed to integrate the cyberinfrastructure with some visualization tools (e.g. Google Maps, Highcharts) to provide a Graphical User Interfaces (GUI) and online visualization for both general public and geospatial researchers.     

Developing Web-based Mapping Applications Through Distributed Object Technology

Emerging web-based mapping technologies use the World Wide Web (WWW) and Internet protocols to provide the ability to distribute, access, and visualize geospatial information over the Internet. Many web-based mapping applications have been developed to deliver geospatial information within and across organizations and even to the public at large. A major technological challenge is to achieve interoperability amongst web-based mapping applications so that mapping and geoprocessing resources distributed over the Internet can be shared and integrated. This paper presents an approach to the development of web-based mapping applications using distributed object technology in order to enable interoperability. Distributed object technology combines object technology, which utilizes reusable software components (called objects) that model real-world entities to build software systems, and distributed computing, which allows computing resources to be distributed and accessed over computer networks. The paper introduces a distributed object technology, the Common Object Request Broker Architecture (CORBA); proposes an architecture for web-based mapping using CORBA; and presents a prototype implementation.     

GeoCube： 面向大规模分析的多源对地观测时空立方体

随着对地立体观测体系的建立,遥感大数据不断累积。传统基于文件、景/幅式的影像组织方式,时空基准不够统一,集中式存储不利于大规模并行分析。对地观测大数据分析仍缺乏一套统一的数据模型与基础设施理论。近年来,数据立方体的研究为对地观测领域大数据分析基础设施提供了前景。基于统一的分析就绪型多维数据模型和集成对地观测数据分析功能,可构建一个基于数据立方的对地观测大数据分析基础设施。因此,本文提出了一个面向大规模分析的多源对地观测时空立方体,相较于现有的数据立方体方法,强调多源数据的统一组织、基于云计算的立方体处理模式以及基于人工智能优化的立方体计算。研究有助于构建时空大数据分析的新框架,同时建立与商业智能领域的数据立方体关联,为时空大数据建立统一的时空组织模型,支持大范围、长时序的快速大规模对地观测数据分析。本文在性能上与开源数据立方做了对比,结果证明提出的多源对地观测时空立方体在处理性能上具有明显优势。     

面向星群的遥感影像智能服务关键问题EI北大核心CSCD

全天时、全天候和全球的遥感信息实时智能服务是对地观测系统建设的目标。近几年来,随着我国高分专项和商业卫星的发展,在轨卫星数量急剧增加,对地观测能力得到极大增强,使得传统的单星和星座卫星系统的运控、接收、处理和应用服务模式面临严峻挑战,亟须统筹规划卫星应用各环节资源,充分发挥多星协同优势,构建统一的遥感影像实时智能服务体系和系统。本文针对遥感星群卫星体系特点和对地观测用户需求特征,开展面向星群的遥感影像智能服务关键问题研究。提出了面向任务的全球多尺度语义描述网格,统筹全球动静态的任务语义描述,在此基础上重点分析了面向星群的自主任务管理、精准动态规划和协同智能处理等关键技术问题,形成集任务描述、任务管控、任务规划、在轨处理、终端分发一体化的星群智能服务技术体系。通过充分发挥星群协同的优势,结合在轨处理和人工智能技术来降低各环节时间延迟,提高数据处理精度,从而实现完全自动化和智能化的近实时星群智能服务,为对地观测的全天时、全天候快速高效智能服务奠定基础。     

Spatial cloud computing: how can the geospatial sciences use and help shape cloud computing?

Abstract

The geospatial sciences face grand information technology (IT) challenges in the twenty-first century: data intensity, computing intensity, concurrent access intensity and spatiotemporal intensity. These challenges require the readiness of a computing infrastructure that can: (1) better support discovery, access and utilization of data and data processing so as to relieve scientists and engineers of IT tasks and focus on scientific discoveries; (2) provide real-time IT resources to enable real-time applications, such as emergency response; (3) deal with access spikes; and (4) provide more reliable and scalable service for massive numbers of concurrent users to advance public knowledge. The emergence of cloud computing provides a potential solution with an elastic, on-demand computing platform to integrate – observation systems, parameter extracting algorithms, phenomena simulations, analytical visualization and decision support, and to provide social impact and user feedback – the essential elements of the geospatial sciences. We discuss the utilization of cloud computing to support the intensities of geospatial sciences by reporting from our investigations on how cloud computing could enable the geospatial sciences and how spatiotemporal principles, the kernel of the geospatial sciences, could be utilized to ensure the benefits of cloud computing. Four research examples are presented to analyze how to: (1) search, access and utilize geospatial data; (2) configure computing infrastructure to enable the computability of intensive simulation models; (3) disseminate and utilize research results for massive numbers of concurrent users; and (4) adopt spatiotemporal principles to support spatiotemporal intensive applications. The paper concludes with a discussion of opportunities and challenges for spatial cloud computing (SCC).     

Distributed geospatial information processing: sharing distributed geospatial resources to support Digital Earth

Abstract

This paper introduces a new concept, distributed geospatial information processing (DGIP), which refers to the process of geospatial information residing on computers geographically dispersed and connected through computer networks, and the contribution of DGIP to Digital Earth (DE). The DGIP plays a critical role in integrating the widely distributed geospatial resources to support the DE envisioned to utilise a wide variety of information. This paper addresses this role from three different aspects: 1) sharing Earth data, information, and services through geospatial interoperability supported by standardisation of contents and interfaces; 2) sharing computing and software resources through a GeoCyberinfrastructure supported by DGIP middleware; and 3) sharing knowledge within and across domains through ontology and semantic searches. Observing the long-term process for the research and development of an operational DE, we discuss and expect some practical contributions of the DGIP to the DE.     

面向开放科学的地球观测信息溯源表达方法研究

随着开放科学时代的来临和空天技术的高速发展,各类地球观测信息在分布式网络环境下得以充分共享,信息的可信度成为各界关注的焦点.地球观测信息溯源作为通过过程追踪保证信息质量的关键技术变得越来越重要.本文在对国内外地球观测信息溯源现状研究的基础上,提出了一种基于PROV模型扩展的地球观测信息产品溯源表达方法,通过对溯源模型中...     

空间信息全球惟一编码GeoID模型初探

为了解决由于不同数据模型中同一空间实体的编码不惟一而导致数据共享困难以及标识的静态性和无空间位置等问题,本文提出了基于全球剖分格网的空间信息全球惟一编码模型(GeoID),并设计了该编码模型的系统架构,试图实现全球连续的、多层次空间信息的惟一标识。通过对遥感影像中的空间实体进行了编码探讨,结果表明空间信息全球惟一编码模型将大大加快信息的查询与检索速度。     

Asynchronous Geospatial Processing: An Event‐Driven Push‐Based Architecture for the OGC Web Processing Service

Geospatial processing tasks like solar potential analyses or floodplain investigations within flood scenarios are often complex and deal with large amounts of data. If such analysis operations are performed in distributed web‐based systems, technical capabilities are mostly not sufficient. Major shortcomings comprise the potentially long execution times and the vast amount of messaging overhead that arise from common poll‐based approaches. To overcome these issues, an approach for an event‐driven architecture for web‐based geospatial processing is proposed within this article. First, this article presents a thorough qualitative discussion of different available technologies for push‐based notifications. The aim of this discussion is to find the most suitable push‐based messaging technologies for application with OGC Web Processing Services (WPS). Based on this, an event‐driven architecture for asynchronous geospatial processing with the WPS is presented, building on the Web Socket Protocol as the transport protocol and the OGC Event Service as the message‐oriented middleware. The proposed architecture allows pushing notifications to clients once a task has completed. This paradigm enables the efficient execution of web‐based geospatial processing tasks as well as the integration of geographical analyses into event‐driven real‐time workflows.