数字地球网格计算雏议

数字地球将为人类提供关于我们地球的海量自然和人文数据与信息 ,是我们生活的行星的一个多分辨率 ,四维虚拟表达。网格计算被认为是解决数字地球问题的最好方法。数字地球问题的解决必须通过异构的计算资源 ,信息系统 ,设备 ,人之间的相互协作 ,而这些都是地理位置或组织结构分散的。本文介绍我们在生成用于解决数字地球问题的核心中间件的研究工作以及结果。因为网格计算本身是一门比较新的领域 ,网格计算与数字地球的有机结合将为数字地球提供一个全新的计算工具     

Digital Earth: a new challenge and new vision

ABSTRACT

Digital Earth has seen great progress during the last 19 years. When it entered into the era of big data, Digital Earth developed into a new stage, namely one characterized by ‘Big Earth Data’, confronting new challenges and opportunities. In this paper we give an overview of the development of Digital Earth by summarizing research achievements and marking the milestones of Digital Earth’s development. Then, the opportunities and challenges that Big Earth Data faces are discussed. As a data-intensive scientific research approach, Big Earth Data provides a new vision and methodology to Earth sciences, and the paper identifies the advantages of Big Earth Data to scientific research, especially in knowledge discovery and global change research. We believe that Big Earth Data will advance and promote the development of Digital Earth.     

Geospatial web services pave new ways for server-based on-demand access and processing of Big Earth Data

ABSTRACT

Big Earth Data has experienced a considerable increase in volume in recent years due to improved sensing technologies and improvement of numerical-weather prediction models. The traditional geospatial data analysis workflow hinders the use of large volumes of geospatial data due to limited disc space and computing capacity. Geospatial web service technologies bring new opportunities to access large volumes of Big Earth Data via the Internet and to process them at server-side. Four practical examples are presented from the marine, climate, planetary and earth observation science communities to show how the standard interface Web Coverage Service and its processing extension can be integrated into the traditional geospatial data workflow. Web service technologies offer a time- and cost-effective way to access multi-dimensional data in a user-tailored format and allow for rapid application development or time-series extraction. Data transport is minimised and enhanced processing capabilities are offered. More research is required to investigate web service implementations in an operational mode and large data centres have to become more progressive towards the adoption of geo-data standard interfaces. At the same time, data users have to become aware of the advantages of web services and be trained how to benefit from them most.     

Big Data and cloud computing: innovation opportunities and challenges

ABSTRACT

Big Data has emerged in the past few years as a new paradigm providing abundant data and opportunities to improve and/or enable research and decision-support applications with unprecedented value for digital earth applications including business, sciences and engineering. At the same time, Big Data presents challenges for digital earth to store, transport, process, mine and serve the data. Cloud computing provides fundamental support to address the challenges with shared computing resources including computing, storage, networking and analytical software; the application of these resources has fostered impressive Big Data advancements. This paper surveys the two frontiers – Big Data and cloud computing – and reviews the advantages and consequences of utilizing cloud computing to tackling Big Data in the digital earth and relevant science domains. From the aspects of a general introduction, sources, challenges, technology status and research opportunities, the following observations are offered: (i) cloud computing and Big Data enable science discoveries and application developments; (ii) cloud computing provides major solutions for Big Data; (iii) Big Data, spatiotemporal thinking and various application domains drive the advancement of cloud computing and relevant technologies with new requirements; (iv) intrinsic spatiotemporal principles of Big Data and geospatial sciences provide the source for finding technical and theoretical solutions to optimize cloud computing and processing Big Data; (v) open availability of Big Data and processing capability pose social challenges of geospatial significance and (vi) a weave of innovations is transforming Big Data into geospatial research, engineering and business values. This review introduces future innovations and a research agenda for cloud computing supporting the transformation of the volume, velocity, variety and veracity into values of Big Data for local to global digital earth science and applications.     

大数据时代的农情监测与预警

农情信息是世界粮农组织、各国政府、粮食贸易企业以及农场管理迫切需要掌握的信息。大数据时代的农情监测与预警正在由模型驱动向数据驱动转变,大数据正逐渐成为监测与预警的核心驱动力。伴随着农情监测与预警大数据的爆炸式增长,大数据与云计算技术的发展为农情监测与预警提供了全新的技术手段。2013年以来,全球农情遥感速报系统(CropWatch)已逐步引入聚类分析、时间序列分析、关联分析、时空变化异常诊断等大数据分析方法,并应用于业务化运行的农情监测与预警中。大数据技术提升了CropWatch的数据挖掘能力,对CropWatch农情监测与预警时空尺度的拓展以及农情监测内容的精细化起到推动作用,促进了面向需求的CropWatch农情信息与预警精准云服务的发展,促成了大数据时代CropWatch农情监测与预警技术体系的升级。未来,大数据时代的农情监测与预警将逐渐向全自动化监测、实时化精准农业管理与智能化信息服务方向发展;通过众源采集技术高效低廉的获取农情观测大数据将成为未来的发展趋势;大数据技术跨领域数据挖掘的能力,使得丰富多元化的跨界信息服务将成为大数据时代农情监测与预警的主流发展方向。大数据时代的CropWatch正在向基于大数据的农情监测与预警系统全速迈进。     

GEOID: GRID Services for Earth Observation Image Data Processing

Abstract

Recent developments in space technology and exponential increase in demand of earth observation data from space have generated a requirement of a data processing environment, where users can discover the data and process, based on their requirements. Grid Services for Earth Observation Image Data Processing (GEOID) is proposed with a motivation to cater to future earth observation applications requirements of digital earth. This paper discusses the overview of the GEOID architecture, its deployment scenario, use-cases and simulation results. Core technologies used for implementation include Grid computing and Service Oriented Architecture. GEOID provides capability to address requirements of applications such as real-time monitoring, time series data processing and processing with user required quality to meet the requirements of end user applications. GEOID allows users to access the archive products or the raw satellite data stream and process their area of interest. Simulations show that applications such as time series analysis show considerable improvement in processing time by using GEOID.     

遥感大数据时代与智能信息提取

近年来,天地一体化对地观测系统与智能计算技术的快速发展为遥感科技进步甚至变革提供了难得的机遇。遥感信息技术在历经20世纪60~80年代以统计数学模型为核心的数字信号处理时代、从90年代至今以遥感信息物理量化为标志的定量遥感时代之后,现在正逐渐进入一个以数据模型驱动、大数据智能分析为特征的遥感大数据时代。在总结遥感信息技术历史发展脉络的基础上,阐述了遥感大数据的内涵和智能信息提取的时代特点,并从遥感大数据的理念出发,论述了面向对象的遥感知识库构建,分析了融合遥感知识和深度学习算法的大数据智能信息提取策略。通过典型实例,介绍了以深度学习为代表的智能算法在遥感大数据目标检测、精细分类、参数反演等方面的发展现状与趋势,并讨论了深度学习在遥感大数据时代的智能信息提取方面的应用潜力。     

IPUMS-Terra: integrated big heterogeneous spatiotemporal data analysis system

Big Geo Data promises tremendous benefits to the GIS Science community in particular and the broader scientific community in general, but has been primarily of use to the relatively small body of GIScientists who possess the specialized knowledge and methods necessary for working with this class of data. Much of the greater scientific community is not equipped with the expert knowledge and techniques necessary to fully take advantage of the promise of big spatial data. IPUMS-Terra provides integrated spatiotemporal data to these scholars by simplifying access to thousands of raster and vector datasets, integrating them and providing them in formats that are useable to a broad array of research disciplines. IPUMS-Terra exemplifies a new class of National Spatial Data Infrastructure because it connects a large spatial data repository to advanced computational resources, allowing users to access the needle of information they need from the haystack of big spatial data. The project is trailblazing in its commitment to the open sharing of spatial data and spatial tool development, including describing its architecture, process development workflows, and openly sharing its products for the general use of the scientific community.     

大数据时代的空间数据挖掘综述

随着大数据时代的到来,数据挖掘技术再度受到人们关注。本文回顾了传统空间数据挖掘面临的问题,介绍了国内外研究中利用大数据处理工具和云计算技术,在空间数据的存储、管理和挖掘算法等方面的做法,并指出了该类研究存在的不足。最后,探讨了空间数据挖掘的发展趋势。     

Overview and preliminary idea for building Digital Earth with Grid computing technology

Abstract

Digital Earth is an important field of information technology and a research frontier of geosciences in the 21st century. So far, the Grid computing technique is one of the best solutions for Digital Earth infrastructure. Digital Earth can only be realised through the interaction of people, heterogeneous computing resources, information systems, and instruments, all of which are geographically and organisationally dispersed. Earth observation (EO) includes information acquisition, processing and applications. Information acquisition provides a vast amount of spatial data for building the fabric resource infrastructure. Information processing means that spatial information processing middleware is used with large amounts of secure Grid computing resources for real-time processing of all kinds of spatial data. We are currently working on the development of core-middleware for EO data processing and applications for the Digital Earth Prototype System, which is available in the Institute of Remote Sensing Applications (IRSA), Chinese Academy of Sciences (CAS) The further results will be available soon.     

Building geospatial infrastructure

ABSTRACT

Many visions for geospatial technology have been advanced over the past half century. Initially researchers saw the handling of geospatial data as the major problem to be overcome. The vision of geographic information systems arose as an early international consensus. Later visions included spatial data infrastructure, Digital Earth, and a nervous system for the planet. With accelerating advances in information technology, a new vision is needed that reflects today’s focus on open and multimodal access, sharing, engagement, the Web, Big Data, artificial intelligence, and data science. We elaborate on the concept of geospatial infrastructure, and argue that it is essential if geospatial technology is to contribute to the solution of problems facing humanity.     

Developments in digital map production and data integration within the British Geological Survey

Abstract

Digital techniques for cartographic data capture and high quality map production have been developed and applied over some 18 years to the mapping of the geoscientific datasets of the British Geological Survey, in particular to the geochemical dataset. Over this period, technological advances and developments in both vector and raster data processing techniques have facilitated high quality map production and the integration and display of multiple datasets. This paper reviews the developments in high quality map production and interpretation of survey information, with particular regard to the Regional Geochemical Atlas Series, through the application of image processing techniques on the display and analysis of multiple datasets.     

Data Cube on Demand (DCoD): Generating an earth observation Data Cube anywhere in the world

To tackle Big Data challenges such as Volume, Variety, and Velocity, the Earth Observations Data Cube (EODC) concept has emerged as a solution for lowering barriers and offering new possibilities to harness the information power of satellite EO data. However, installing, configuring, and managing an EODC instance is still difficult requiring specific knowledge and capabilities. Consequently, facilitating and automating the generation and provision of EODC given specific user’s requirements can be beneficial.In response to this issue, this paper presents the Data Cube on Demand (DCoD) approach, a proof-of-concept that aims at facilitating the generation and use of an EODC instance virtually anywhere in the World. Users are only required to specify an area of interest; select the types of sensors between Landsat 5-7-8 and Sentinel-2; choose a desired temporal frame; and provide their email address to receive notifications. Then automatically an empty ODC instance is instantiated and desired data are ingested.The proposed approach has been successfully tested in two sites in Bolivia and DRC in the field of environmental monitoring. It has lowered many complexity barriers of such a new technology; greatly facilitated the generation and use of the Data Cube technology; enhanced data sovereignty; and ultimately can help reaching large adoption and acceptance.     

大数据环境下地理信息产业发展的几点思考

大数据背景下地理信息产业的发展有了新的驱动力,通过思考这种驱动力对产业发展可能产生的影响,分析指出了大数据在地理信息产业中的中心地位和重要价值,综述了大数据给地理信息产业的涵盖范围、关键技术和服务模式所带来的重大变革,探讨了大数据技术对于GIS传统技术优化升级的几个着力点,并剖析了大数据技术在地理信息产业的相关应用中所面临的制约因素。     

Multidimensional and quantitative interlinking approach for Linked Geospatial Data

ABSTRACT

Linked Data is known as one of the best solutions for multisource and heterogeneous web data integration and discovery in this era of Big Data. However, data interlinking, which is the most valuable contribution of Linked Data, remains incomplete and inaccurate. This study proposes a multidimensional and quantitative interlinking approach for Linked Data in the geospatial domain. According to the characteristics and roles of geospatial data in data discovery, eight elementary data characteristics are adopted as data interlinking types. These elementary characteristics are further combined to form compound and overall data interlinking types. Each data interlinking type possesses one specific predicate to indicate the actual relationship of Linked Data and uses data similarity to represent the correlation degree quantitatively. Therefore, geospatial data interlinking can be expressed by a directed edge associated with a relation predicate and a similarity value. The approach transforms existing simple and qualitative geospatial data interlinking into complete and quantitative interlinking and promotes the establishment of high-quality and trusted Linked Geospatial Data. The approach is applied to build data intra-links in the Chinese National Earth System Scientific Data Sharing Network (NSTI-GEO) and data -links in NSTI-GEO with the Chinese Meteorological Data Network and National Population and Health Scientific Data Sharing Platform.     

基于Web地理可视化分析系统的研究与设计

气候研究涉及海量时空数据集,其产生和存储在分布式计算资源中。科学家们需要一个直观且方便地工具去研究分布式时空数据。地理可视化分析工具可直观且方便地访问气候资料、探索各种气候参数之间的关系及交流研究成果。本文对基于Web的地理可视化分析系统的研究与设计做一些探讨,阐述该系统具有以下功能：①互联网上海量数据集管理;②时空数据的2D/3D可视化;③气候研究中各种时空统计分析;④交互式数据分析和知识发现。此外,本文也为管理、分发、分析大数据提供参考。     

Parallel processing for geographical applications: A layered approach

Significant trends in the processing of geographical data require increasingly powerful software and hardware, consistent with the exploitation of parallel computing. Despite recent progress in technology, exploiting parallel processing is still difficult so that few applications have been developed in the environmental and geographical domains.  Key issues which must be addressed in the design of parallel geographical software are described with reference to designs for three examples which use grid and raster data. The implications for parallel processing with vector-topological data are then explored. The emphasis is upon MIMD architectures using strategies of decomposition into subareas, and upon the need to facilitate development of parallel geographical applications by encapsulating the parallelism in a low-level layer of software, forming a skeletal framework upon which application algorithms can be built. The parallel layer will support distribution of datasets across the multiple processors, and the creation and collation of datasets from those processors.     

A general-purpose framework for parallel processing of large-scale LiDAR data

ABSTRACT

Light detection and ranging (LiDAR) data are essential for scientific discoveries such as Earth and ecological sciences, environmental applications, and responding to natural disasters. While collecting LiDAR data over large areas is quite possible the subsequent processing steps typically involve large computational demands. Efficiently storing, managing, and processing LiDAR data are the prerequisite steps for enabling these LiDAR-based applications. However, handling LiDAR data poses grand geoprocessing challenges due to data and computational intensity. To tackle such challenges, we developed a general-purpose scalable framework coupled with a sophisticated data decomposition and parallelization strategy to efficiently handle ‘big’ LiDAR data collections. The contributions of this research were (1) a tile-based spatial index to manage big LiDAR data in the scalable and fault-tolerable Hadoop distributed file system, (2) two spatial decomposition techniques to enable efficient parallelization of different types of LiDAR processing tasks, and (3) by coupling existing LiDAR processing tools with Hadoop, a variety of LiDAR data processing tasks can be conducted in parallel in a highly scalable distributed computing environment using an online geoprocessing application. A proof-of-concept prototype is presented here to demonstrate the feasibility, performance, and scalability of the proposed framework.     

地理国情大数据研究框架

地理国情监测作为大数据时代测绘地理信息领域一个新的、重要战略方向,其发展迫切需要顶层设计与新型技术的支撑,需要建立一种灵活、高效、低成本的大数据处理模式与服务方式。本文以地理国情监测与大数据研究相结合为切入点,阐述了地理国情大数据的分类及特点,提出了云计算环境下地理国情大数据研究架构,并从地理国情大数据存储、处理、挖掘、应用服务4个方面探讨了地理国情大数据云平台建设思路。本文将有助于地理国情监测的生产方式与服务模式变革,推动地理国情监测的广泛应用与产业化发展。