Advancing Science with VGI: Reproducibility and Replicability of Recent Studies using VGI

In scientific research, reproducibility and replicability are requirements to ensure the advancement of our body of knowledge. This holds true also for VGI‐related research and studies. However, the characteristics of VGI suggest particular difficulties in ensuring reproducibility and replicability. In this article, we aim to examine the current situation in VGI‐related research, and identify strategies to ensure realization of its full potential. To do so, we first investigate the different aspects of reproducibility and replicability and their impact on VGI‐related research. These impacts are different depending on the objectives of the study. Therefore, we examine the study focus of VGI‐related research to assess the current body of research and structure our assessment. This work is based on a rigorous review of the elements of reproducibility and a systematic mapping and analysis of 58 papers on the use of VGI in the crisis management field. Results of our investigation show that reproducibility issues related to data are a serious concern, while reproducibility issues related to analysis methods and processes face fewer challenges. However, since most studies still focus on analyzing the source data, reproducibility and replicability are still an unsolved problem in VGI‐related research. Therefore, we show initiatives tackling the problem, and finally formulate strategies to improve the situation.     

Applying VGI to collaborative research in the humanities: the case of ARTL@S

ABSTRACT

Using Artl@s as an example of a project that relies on volunteered geographic information (VGI), this article examines the specific challenges that exist, beyond those frequently discussed in general VGI systems (e.g., participants’ motivation and data quality control) in regard to sharing research data in humanities: (1) most data from the humanities is qualitative and collected from multiple data sources which are often inconsistent and unmappable; (2) data is usually interconnected with multiple relationships among different tables which creates challenges for both mapping and query functionality; (3) data is both geographical and historical. Consequently addresses that no longer exist have to be geolocated and visualized on historical basemaps and spaces must be represented diachronically; (4) the design of web map application needs to balance both sophisticated research requirements and a user-friendly interface; (5) finally contributors expect their data to be cited or acknowledged when used in other studies and users need metadata and citation information in order to reuse and repurpose datasets.

In this article, we discuss how Artl@s, a project which developed a georeferenced historical database of exhibition catalogues, addresses these challenges. Artl@s provides a case study for VGI adoption by digital humanities scholars for research data sharing, as it offers features, such as flexible batch data contribution, interrelated spatial query, automatic geolocalization of historical addresses, and data citation mechanisms.     

面向灾害管理的自发地理信息研究及应用

针对目前自发地理信息(VGI)应用于灾害管理的巨大潜力和需求,该文系统地介绍了VGI应用于灾害管理中所面临的机遇及数据可用性、数据质量、数据管理和法律议题等方面的挑战;在此基础上,针对VGI数据质量问题,结合我国灾害管理实际情况,提出了3种提高VGI数据质量的方法,即VGI数据贡献者分类与权限设计、政府引导下的灾害管理VGI数据采集框架以及VGI数据质量评估体系;最后,以开发的地震灾情信息管理原型系统为例,讨论VGI应用于灾害管理中的数据采集框架、数据模型等关键技术问题,具有一定的推广和应用价值。     

VGIS-AntiJitter: an effective framework for solving jitter problems in virtual geographic information systems

Abstract

With the proposition of the Digital Earth (DE) concept, Virtual Geographic Information System (VGIS) has started to play the role of a Digital Earth prototype system. Many core problems involved in VGIS, such as out-of-core management and interactive rendering of very large scale terrain and image data, have been well studied in the past decades. However, the jitter problem, a common problem in VGIS that often causes annoying visual artefacts and deteriorates the output image quality, draws little attention. In this paper, after an intensive analysis of the jitter problem, a comprehensive framework is proposed to address such a problem while accounting for the characteristics of different data types in VGIS, such as terrain or ocean mesh data, vector data and 3-D model data. Specifically, this framework provides an improved dynamic local coordinate system (DLCS) method for terrain or ocean mesh data. For vector data, the framework provides a simple and effective multiple local coordinate systems (MLCS) method. The framework provides a MLCS method for 3-D model data making full use of the existing local coordinate system of the model. The advantages of the proposed methods over current approaches are analysed and highlighted through case studies involving large GIS datasets.     

兼容VGI与众包的灾害信息管理系统研究

针对地理信息采集VGI和众包传播模式在灾害救援中发挥的巨大作用,设计并实现了兼容VGI与众包的灾害信息管理系统VDMS,实例验证了该系统的有效性。结果表明:VDMS系统可以将VGI众包采集流程集成到灾害信息管理中,提高了灾害信息采集效率。设计了一种灾害信息管理系统VDMS,实现了灾害发生时海量数据的高效管理与有效利用。     

Matching authority and VGI road networks using an extended node-based matching algorithm

The amount of volunteered geographic information (VGI) has increased over the past decade, and several studies have been conducted to evaluate the quality of VGI data. In this study, we evaluate the completeness of the road network in the VGI data set OpenStreetMap (OSM). The evaluation is based on an accurate and efficient network-matching algorithm. The study begins with a comparison of the two main strategies for network matching: segment-based and node-based matching. The comparison shows that the result quality is comparable for the two strategies, but the node-based result is considerably more computationally efficient. Therefore, we improve the accuracy of node-based algorithm by handling topological relationships and detecting patterns of complicated network components. Finally, we conduct a case study on the extended node-based algorithm in which we match OSM to the Swedish National Road Database (NVDB) in Scania, Sweden. The case study reveals that OSM has a completeness of 87% in the urban areas and 69% in the rural areas of Scania. The accuracy of the matching process is approximately 95%. The conclusion is that the extended node-based algorithm is sufficiently accurate and efficient for conducting surveys of the quality of OSM and other VGI road data sets in large geographic regions.     

VGI and crowdsourced data credibility analysis using spam email detection techniques

Volunteered geographic information (VGI) can be considered a subset of crowdsourced data (CSD) and its popularity has recently increased in a number of application areas. Disaster management is one of its key application areas in which the benefits of VGI and CSD are potentially very high. However, quality issues such as credibility, reliability and relevance are limiting many of the advantages of utilising CSD. Credibility issues arise as CSD come from a variety of heterogeneous sources including both professionals and untrained citizens. VGI and CSD are also highly unstructured and the quality and metadata are often undocumented. In the 2011 Australian floods, the general public and disaster management administrators used the Ushahidi Crowd-mapping platform to extensively communicate flood-related information including hazards, evacuations, emergency services, road closures and property damage. This study assessed the credibility of the Australian Broadcasting Corporation’s Ushahidi CrowdMap dataset using a Naïve Bayesian network approach based on models commonly used in spam email detection systems. The results of the study reveal that the spam email detection approach is potentially useful for CSD credibility detection with an accuracy of over 90% using a forced classification methodology.     

Assessing Resistance to Volunteered Geographic Information Reporting within Local Government

Using volunteered geographic information (VGI) as a cost‐saving environmental monitoring and public participation mechanism has been thought to be a proof of concept to assist environmental decision‐making with insufficient managerial resources. However, research literature also often found institutional resistance when implementing VGI as part of the decision‐making process. This research develops a conceptual framework as the foundation to conduct a cross‐case comparison for four empirical cases in recreation and invasive species monitoring, but general enough for other such case comparisons. The VGI Reporting Assessment Framework (VGI‐RAF) adopted aspects of the Decision Situation Assessment (DSA) framework as the foundation to address institutional flexibility, stakeholder dynamics, and technical considerations. Using a cross‐case comparison can accumulate deep observations from cases and increase validity of insights through comparing processes. Results show that eight aspects of the VGI‐RAF reveal the suitability of embedding voluntary information within existing institutional decision‐making processes. VGI‐RAF depicts considerations for: (1) existing participation arrangements, (2) institutional arrangements, (3) managers’ needs and stakeholder expectation, and (4) interaction mechanism and technical needs. The cross‐comparison results indicate that a public participation mechanism should be accompanied by VGI development to increase effectiveness. Institutional arrangements should match local government's information needs with participants’ expectations. Knowledge gaps between institutional information needs and volunteer knowledge should be filled through technology enhancements. For ease of implementation and to promote long‐term success, we recommend that volunteered information be connected with existing communication channels to minimize changes to existing information systems.     

Understanding Design with VGI using an Information Relevance Framework

Volunteered Geographic Information (VGI) has the potential to offer benefits to end‐users over and above those of Professional Geographic Information (PGI). A multi‐methods approach, consisting of participatory observation, focus groups and diary studies, was used to study how VGI and PGI were actually used by a target user group. A theoretical framework of information relevance was used to understand the attributes of information that were most important in relation to the characteristics of the users' activity and their community. The key finding was that the discussion amongst GIS designers should not be whether to choose VGI or PGI as the information data set, but to consider which combination of VGI and PGI relating to different geographic features and task characteristics would best meet the users' needs. VGI is likely to be most relevant to the user when a geographic feature is dynamic rather than static in nature, and can also provide a level of detail that is unlikely to arise through PGI. These findings have implications for how different forms of information are most effective for different usage situations. Above all, a case is presented for the implementation of User Centred Design (UCD) principles when integrating VGI and PGI together in a single mashup‐based product.     

Integrating VGI and 2D hydraulic models into a data assimilation framework for real time flood forecasting and mapping

ABSTRACT

Crowdsourced data can effectively observe environmental and urban ecosystem processes. The use of data produced by untrained people into flood forecasting models may effectively allow Early Warning Systems (EWS) to better perform while support decision-making to reduce the fatalities and economic losses due to inundation hazard. In this work, we develop a Data Assimilation (DA) method integrating Volunteered Geographic Information (VGI) and a 2D hydraulic model and we test its performances. The proposed framework seeks to extend the capabilities and performances of standard DA works, based on the use of traditional in situ sensors, by assimilating VGI while managing and taking into account the uncertainties related to the quality, and the location and timing of the entire set of observational data. The November 2012 flood in the Italian Tiber River basin was selected as the case study. Results show improvements of the model in terms of uncertainty with a significant persistence of the model updating after the integration of the VGI, even in the case of use of few-selected observations gathered from social media. This will encourage further research in the use of VGI for EWS considering the exponential increase of quality and quantity of smartphone and social media user worldwide.     

Integrating GIS and Data Visualization Tools for Distributed Hydrologic Modeling

Distributed hydrologic models are an integral component of spatial decision support systems for watershed management. At these basin (and larger) scales, a common problem in hydrology and other environmental sciences arises from the need to cope with the increasingly sophisticated models and software tools that are used in the management, processing, and analysis of large volumes of data collected from advanced in situ and remote monitoring instruments. The diversity of these tools, and the need for experts and non-experts alike to be able to easily use them, suggests a non-trivial integration problem of building customized systems from disparate (and in many cases proprietary) components. In this paper we address the systems integration problem within the context of a study of surface-subsurface-atmosphere interactions involving soil moisture remote sensing and hydrologic simulation. The simple early steps that were taken highlight the role of the different components and tools in the integration effort. An alternative approach is presented for creating an integrated system, based on a data flow model with the modular components linked within a flexible framework. An example is developed based on established systems integration techniques from the field of computer aided design.     

Validity of historical volunteered geographic information: Evaluating citizen data for mapping historical geographic phenomena

Studies on volunteered geographic information (VGI) have focused on examining its validity to reveal geographic phenomena in relatively recent periods. Empirical evaluation of the validity of VGI to reveal geographic phenomena in historical periods (e.g., decades ago) is lacking, although such evaluation is desirable for assessing the possibility of broadening the temporal scope of VGI applications. This article presents an evaluation of the validity of VGI to reveal historical geographic phenomena through a citizen data‐based habitat suitability mapping case study. Citizen data (i.e., sightings) of the black‐and‐white snub‐nosed monkey (Rhinopithecus bieti) were elicited from local residents through three‐dimensional (3D) geovisualization interviews in Yunnan, China. The validity of the elicited sightings to reveal the historical R. bieti distribution was evaluated through habitat suitability mapping using the citizen data in historical periods. The results of controlled experiments demonstrated that suitability maps predicted using the historical citizen data had a consistent spatial pattern (correlation above 0.60) that reflects the R. bieti distribution (Boyce index around 0.90) in areas free of significant environmental change across historical periods. This in turn suggests that citizen data have validity for mapping historical geographic phenomena. It provides supporting empirical evidence for potentially broadening the temporal scope of VGI applications.     

众源地理信息研究现状与展望

众源地理信息作为新型地理信息模式,是指由互联网用户借助交互式平台,通过直接上传传感器数据或提供数字化劳动而贡献的地理信息数据。近20年来,众源地理信息热度上升,并受到越来越多研究者的关注,正在逐渐成为一种重要的地理信息数据来源。本文对近十几年来关于众源地理信息的中文和英文文献进行了系统性的分析与研究,力图展现出众源地理信息研究的发展现状,同时以这些研究发现的规律为基础,结合当前的国际形势,探讨了众源地理信息的发展机遇与挑战,同时对发展自主众源地理信息给出了建议。     

Improvement of a location-aware recommender system using volunteered geographic information

Abstract

Recommender systems (RS), as supportive tools, filter information from a massive amount of data based on the determined preferences. Most of the RS require information about the context of users such as their locations. In such cases, location-aware recommender systems (LARS) can be employed to suggest more personalized items to the users. The most current research projects on LARS focus on the development of algorithms, evaluation methods and applications. However, the role of up-to-date spatial databases in LARS is not a well-researched area. The up-to-date spatial information would potentially improve the accuracy of items which are recommended by LARS. Volunteered geographic information (VGI) could be a low-cost source of up-to-date spatial information for LARS. This article proposes an approach to enrich spatial databases of LARS by VGI. Since not all records of VGI are fitted for use in LARS, a mechanism is developed to identify useful information. Some VGI data sets refer to existing spatial data in the database while other VGI data sets are shared for the first time. Therefore, the proposed method assessed the quality of VGI with reference source (for VGI which is existed in the database) and VGI without reference source (for VGI which is shared for the first time). To demonstrate the feasibility of the proposed approach, a mobile application has been developed to recommend suitable restaurants to the users based on their geospatial locations. The evaluation of the method indicates that VGI can potentially enhance the functionality of the LARS in predicting the users’ interests.     

浅论自发地理信息的数据管理

分析了自发地理信息(volunteered geographic information,VGI)数据的来源、分类、特点与管理要求,探讨了VGI数据清理与质量控制,研究了以高效处理绘图查询与数据更新为目标的VGI图形数据管理问题,提出了动态线综合二叉树与缩放四叉树的设计思想,以解决VGI图形数据管理中的难点问题。     

Modeling aggregated expertise of user contributions to assess the credibility of OpenStreetMap features

The emergence of volunteered geographic information (VGI) during the past decade has fueled a wide range of research and applications. The assessment of VGI quality and fitness‐of‐use is still a challenge because of the non‐standardized and crowdsourced data collection process, as well as the unknown skill and motivation of the contributors. However, the frequent approach of assessing VGI quality against external data sources using ISO quality standard measures is problematic because of a frequent lack of available external (reference) data, and because for certain types of features, VGI might be more up‐to‐date than the reference data. Therefore, a VGI‐intrinsic measure of quality is highly desirable. This study proposes such an intrinsic measure of quality by developing the concept of aggregated expertise based on the characteristics of a feature's contributors. The article further operationalizes this concept and examines its feasibility through a case study using OpenStreetMap (OSM). The comparison of model OSM feature quality with information from a field survey demonstrates the successful implementation of this novel approach.     

Finding meaningful participation in volunteer geographic information and citizen science: a case comparison in environmental application

ABSTRACT

Volunteer geographic information (VGI) and citizen science (CS) have been used as a way to promote a form of democratization of knowledge. Crowdsourcing facilitates the massive collection of data through nonexpert volunteers. This study compares 57 VGI and CS projects through a cross-case comparison method to explore the level of involvement of participants, based on three main analytic dimensions (project rationale, actor interaction, and data flow) and 12 sub-variables. The primary goal of this research is to analyze the purpose and the role of technology in VGI and CS projects, using three matrices to explore emerging trade-offs. Finally, we synthesized the workflow of VGI and CS project and compared based on project purposes. At the end of this study, five main purposes were found among the selected projects, uncovering that data collection is the most common purpose. A general workflow can be conceptualized, but detail implementation will involve various trade-offs, especially between local involvement and large-area expert-oriented protocol implementations. By exploring diverse relationships among selected variables of analysis in VGI and CS projects, this study pretends to build a foundation for larger, global meta-analysis that can bring some transparency in the epistemic interactions between experts and nonexperts.     

Towards Usability Engineering for Online Editors of Volunteered Geographic Information: A Perspective on Learnability

This article presents one of the first systematic usability investigations for a Volunteered Geographic Information (VGI) editor. This research is motivated by the fact that although VGI is now widely consumed, contribution rates are lagging considerably. Compared to traditional GIS interfaces, with complex interfaces resulting in high cognitive loads and barriers to participation, VGI tools and interfaces need to be easy to use and learn to encourage and facilitate contributions. This article develops a case study of OpenStreetMap, one of the most successful VGI projects, and its default editor, Potlatch2. Ten participants with no prior experience of VGI contribution, were instructed to contribute data to OSM in a structured exercise, while being monitored using an eye tracker and audio/video screen capture software. Each participant was asked to Think Aloud, i.e. describe what they were thinking and doing as they completed the tasks. The results highlight significant usability issues impacting learnability, especially from the perspective of a new contributor: hidden functionality, lack of user feedback between interactions and the inefficient and inconsistent placement of functionality and map controls. The facilitation of VGI contributions clearly depends on designing targeted interfaces, optimized to the needs of specific levels of contributors with defined goals and expectations.     

The Scale of VGI in Map Production: A Perspective on European National Mapping Agencies

The perspective of European National Mapping Agencies (NMA) on the role of citizen sensing in map production was explored. The NMAs varied greatly in their engagement with the community generating volunteered geographic information (VGI) and in their future plans. From an assessment of NMA standard practices, it was evident that much VGI was acquired with a positional accuracy that, while less than that typically acquired by NMAs, actually exceeded the requirements of the nominal data capture scale used by most NMAs. Opportunities for VGI use in map revision and updating were evident, especially for agencies that use a continuous rather than cyclical updating policy. Some NMAs had also developed systems to engage with citizen sensors and examples are discussed. Only rarely was VGI used to collect data on features beyond the standard set used by the NMAs. The potential role of citizen sensing and so its current scale of use by NMAs is limited by a series of concerns, notably relating to issues of data quality, the nature and motivation of the contributors, legal issues, the sustainability of data source, and employment fears of NMA staff. Possible priorities for future research and development are identified to help ensure that the potential of VGI in mapping is realized.