Data trustworthiness and user reputation as indicators of VGI quality

ABSTRACT

Volunteered geographic information (VGI) has entered a phase where there are both a substantial amount of crowdsourced information available and a big interest in using it by organizations. But the issue of deciding the quality of VGI without resorting to a comparison with authoritative data remains an open challenge. This article first formulates the problem of quality assessment of VGI data. Then presents a model to measure trustworthiness of information and reputation of contributors by analyzing geometric, qualitative, and semantic aspects of edits over time. An implementation of the model is running on a small data-set for a preliminary empirical validation. The results indicate that the computed trustworthiness provides a valid approximation of VGI quality.     

Improvement of a location-aware recommender system using volunteered geographic information

Abstract

Recommender systems (RS), as supportive tools, filter information from a massive amount of data based on the determined preferences. Most of the RS require information about the context of users such as their locations. In such cases, location-aware recommender systems (LARS) can be employed to suggest more personalized items to the users. The most current research projects on LARS focus on the development of algorithms, evaluation methods and applications. However, the role of up-to-date spatial databases in LARS is not a well-researched area. The up-to-date spatial information would potentially improve the accuracy of items which are recommended by LARS. Volunteered geographic information (VGI) could be a low-cost source of up-to-date spatial information for LARS. This article proposes an approach to enrich spatial databases of LARS by VGI. Since not all records of VGI are fitted for use in LARS, a mechanism is developed to identify useful information. Some VGI data sets refer to existing spatial data in the database while other VGI data sets are shared for the first time. Therefore, the proposed method assessed the quality of VGI with reference source (for VGI which is existed in the database) and VGI without reference source (for VGI which is shared for the first time). To demonstrate the feasibility of the proposed approach, a mobile application has been developed to recommend suitable restaurants to the users based on their geospatial locations. The evaluation of the method indicates that VGI can potentially enhance the functionality of the LARS in predicting the users’ interests.     

Obituary

Abstract

title/>

In the area of volunteered geographical information (VGI), the issue of spatial data quality is a clear challenge. The data that are contributed to VGI projects do not comply with standard spatial data quality assurance procedures, and the contributors operate without central coordination and strict data collection frameworks. However, similar to the area of open source software development, it is suggested that the data hold an intrinsic quality assurance measure through the analysis of the number of contributors who have worked on a given spatial unit. The assumption that as the number of contributors increases so does the quality is known as ‘Linus’ Law’ within the open source community. This paper describes three studies that were carried out to evaluate this hypothesis for VGI using the OpenStreetMap dataset, showing that this rule indeed applies in the case of positional accuracy.     

Modeling aggregated expertise of user contributions to assess the credibility of OpenStreetMap features

The emergence of volunteered geographic information (VGI) during the past decade has fueled a wide range of research and applications. The assessment of VGI quality and fitness‐of‐use is still a challenge because of the non‐standardized and crowdsourced data collection process, as well as the unknown skill and motivation of the contributors. However, the frequent approach of assessing VGI quality against external data sources using ISO quality standard measures is problematic because of a frequent lack of available external (reference) data, and because for certain types of features, VGI might be more up‐to‐date than the reference data. Therefore, a VGI‐intrinsic measure of quality is highly desirable. This study proposes such an intrinsic measure of quality by developing the concept of aggregated expertise based on the characteristics of a feature's contributors. The article further operationalizes this concept and examines its feasibility through a case study using OpenStreetMap (OSM). The comparison of model OSM feature quality with information from a field survey demonstrates the successful implementation of this novel approach.     

Matching authority and VGI road networks using an extended node-based matching algorithm

The amount of volunteered geographic information (VGI) has increased over the past decade, and several studies have been conducted to evaluate the quality of VGI data. In this study, we evaluate the completeness of the road network in the VGI data set OpenStreetMap (OSM). The evaluation is based on an accurate and efficient network-matching algorithm. The study begins with a comparison of the two main strategies for network matching: segment-based and node-based matching. The comparison shows that the result quality is comparable for the two strategies, but the node-based result is considerably more computationally efficient. Therefore, we improve the accuracy of node-based algorithm by handling topological relationships and detecting patterns of complicated network components. Finally, we conduct a case study on the extended node-based algorithm in which we match OSM to the Swedish National Road Database (NVDB) in Scania, Sweden. The case study reveals that OSM has a completeness of 87% in the urban areas and 69% in the rural areas of Scania. The accuracy of the matching process is approximately 95%. The conclusion is that the extended node-based algorithm is sufficiently accurate and efficient for conducting surveys of the quality of OSM and other VGI road data sets in large geographic regions.     

面向灾害管理的自发地理信息研究及应用

针对目前自发地理信息(VGI)应用于灾害管理的巨大潜力和需求,该文系统地介绍了VGI应用于灾害管理中所面临的机遇及数据可用性、数据质量、数据管理和法律议题等方面的挑战;在此基础上,针对VGI数据质量问题,结合我国灾害管理实际情况,提出了3种提高VGI数据质量的方法,即VGI数据贡献者分类与权限设计、政府引导下的灾害管理VGI数据采集框架以及VGI数据质量评估体系;最后,以开发的地震灾情信息管理原型系统为例,讨论VGI应用于灾害管理中的数据采集框架、数据模型等关键技术问题,具有一定的推广和应用价值。     

面向道路网VGI数据的匹配与融合方法

自发地理信息(VGI)是一种新兴的地理数据采集方式,具有数据更新快、细节丰富、覆盖范围广等优势。利用VGI数据可以对道路网实现快速更新;但是由于VGI数据是非专业自发共享的,且数据采集时多使用非专业设备,所以存在数据质量不高的问题。大量VGI数据对同一地理要素的重复采集与融合处理则可改善数据的质量,文中以多人采集的道路网数据为例,结合矢量要素的匹配与融合理论,设计一种适用于道路网VGI数据的匹配与融合算法。首先在路段结点处建立缓冲区进行结点匹配,再根据路段距离相似度进行路段匹配,最后再利用Delaunay三角剖分融合算法对匹配后的同名路段进行融合。将匹配融合后的道路网与原始道路网VGI数据及Google影像图叠加对比分析,结果表明利用本文算法可有效地实现道路网VGI数据的匹配与融合。     

Trust as a proxy indicator for intrinsic quality of Volunteered Geographic Information in biodiversity monitoring programs

In this article, we present a fuzzy model for intrinsic quality assessment of Volunteered Geographic Information (VGI) on species occurrences obtained by Citizen Science (CS) biodiversity monitoring programs. The proposed VGI quality assurance approach evaluates the thematic and positional quality of the crowdsourced biodiversity observation in terms of the trustworthiness of the observation by combining three indicators of consistency with habitat, consistency with surroundings, and reputation of contributor, that characterize the geographical and social aspects of trust in VGI. To evaluate the performance and usability of the proposed approach for evaluating the trustworthiness of crowdsourced observations and detecting thematic and positional errors in crowdsourced observations, the developed approach was applied to the crowdsourced observations on Acer macrophyllum collected through the CS biodiversity monitoring projects of E-Flora BC and iNaturalist. The result of a conformity test at the optimal acceptance threshold (sensitivity = 0.99, specificity = 0.8, and Cohen’s kappa = 0.79), the achieved area under the curve (AUC) value (AUC = 0.98), and the results of the complementary investigation on the predictions of the proposed model indicated that the proposed fuzzy trust model exhibited promising predictive performance and was able to flag the majority of attribute and positional errors in the crowdsourced biodiversity observations.     

Assessing relevance of tweets for risk communication

ABSTRACT

Although Twitter is used for emergency management activities, the relevance of tweets during a hazard event is still open to debate. In this study, six different computational (i.e. Natural Language Processing) and spatiotemporal analytical approaches were implemented to assess the relevance of risk information extracted from tweets obtained during the 2013 Colorado flood event. Primarily, tweets containing information about the flooding events and its impacts were analysed. Examination of the relationships between tweet volume and its content with precipitation amount, damage extent, and official reports revealed that relevant tweets provided information about the event and its impacts rather than any other risk information that public expects to receive via alert messages. However, only 14% of the geo-tagged tweets and only 0.06% of the total fire hose tweets were found to be relevant to the event. By providing insight into the quality of social media data and its usefulness to emergency management activities, this study contributes to the literature on quality of big data. Future research in this area would focus on assessing the reliability of relevant tweets for disaster related situational awareness.     

浅论自发地理信息的数据管理

分析了自发地理信息(volunteered geographic information,VGI)数据的来源、分类、特点与管理要求,探讨了VGI数据清理与质量控制,研究了以高效处理绘图查询与数据更新为目标的VGI图形数据管理问题,提出了动态线综合二叉树与缩放四叉树的设计思想,以解决VGI图形数据管理中的难点问题。     

Applying VGI to collaborative research in the humanities: the case of ARTL@S

ABSTRACT

Using Artl@s as an example of a project that relies on volunteered geographic information (VGI), this article examines the specific challenges that exist, beyond those frequently discussed in general VGI systems (e.g., participants’ motivation and data quality control) in regard to sharing research data in humanities: (1) most data from the humanities is qualitative and collected from multiple data sources which are often inconsistent and unmappable; (2) data is usually interconnected with multiple relationships among different tables which creates challenges for both mapping and query functionality; (3) data is both geographical and historical. Consequently addresses that no longer exist have to be geolocated and visualized on historical basemaps and spaces must be represented diachronically; (4) the design of web map application needs to balance both sophisticated research requirements and a user-friendly interface; (5) finally contributors expect their data to be cited or acknowledged when used in other studies and users need metadata and citation information in order to reuse and repurpose datasets.

In this article, we discuss how Artl@s, a project which developed a georeferenced historical database of exhibition catalogues, addresses these challenges. Artl@s provides a case study for VGI adoption by digital humanities scholars for research data sharing, as it offers features, such as flexible batch data contribution, interrelated spatial query, automatic geolocalization of historical addresses, and data citation mechanisms.     

The Scale of VGI in Map Production: A Perspective on European National Mapping Agencies

The perspective of European National Mapping Agencies (NMA) on the role of citizen sensing in map production was explored. The NMAs varied greatly in their engagement with the community generating volunteered geographic information (VGI) and in their future plans. From an assessment of NMA standard practices, it was evident that much VGI was acquired with a positional accuracy that, while less than that typically acquired by NMAs, actually exceeded the requirements of the nominal data capture scale used by most NMAs. Opportunities for VGI use in map revision and updating were evident, especially for agencies that use a continuous rather than cyclical updating policy. Some NMAs had also developed systems to engage with citizen sensors and examples are discussed. Only rarely was VGI used to collect data on features beyond the standard set used by the NMAs. The potential role of citizen sensing and so its current scale of use by NMAs is limited by a series of concerns, notably relating to issues of data quality, the nature and motivation of the contributors, legal issues, the sustainability of data source, and employment fears of NMA staff. Possible priorities for future research and development are identified to help ensure that the potential of VGI in mapping is realized.     

Updating digital elevation models via change detection and fusion of human and remote sensor data in urban environments

OpenStreetMap (OSM) currently represents the most popular project of Volunteered Geographic Information (VGI): geodata are collected by common people and made available for public use. Airborne Laser Scanning (ALS) enables the acquisition of high-resolution digital elevation models that are used for many applications. This study combines the advantages of both ALS and OSM, offering a promising new approach that enhances data quality and allows change detection: the mainly up-to-date 2D data of OSM can be combined with the high-resolution – but rarely updated – elevation information provided by ALS. This case study investigates building objects of OSM and ALS data of the city of Bregenz, Austria. Data quality of OSM is discerned by the comparison of building footprints using different true positive definitions (e.g. overlapping area). High quality of OSM data is revealed, yet also limitations of each method with respect to heterogeneous regions and building outlines are identified. For the first time, an up-to-date Digital Surface Model (DSM) combining 2D OSM and ALS data is achieved. A multitude of applications such as flood simulations and solar potential assessments can directly benefit from this data combination, since their value and reliability strongly depend on an up-to-date DSM.     

Geo-questionnaires in urban planning: recruitment methods,participant engagement,and data quality

ABSTRACT

Recent focus on sustainable urban development and livability has increased the demand for new data sourcing techniques to capture experiences and preferences of urban dwellers. At the same time, developments of geospatial technologies and social media have enabled new types of user-generated geographic information and spatially explicit online communication. As a result, new public participation GIS methods for engaging large groups of individuals have emerged. One such method is geo-questionnaire, an online questionnaire with mapping capabilities, which has been used to elicit geographic data in variety of topics and geographical contexts. This article presents two recent cases, in which geo-questionnaires have been used in Polish cities to obtain public input on quality of life and development preferences in local land use planning. The article evaluates participant recruitment methods focusing on sample representativeness, participant engagement, and data quality. Recruitment via social media was found to increase bias towards younger population. Paper questionnaires used along the online version provided for better representation of target population’s age structure, but did not reduce bias related to educational attainment. We discuss how these issues relate to data usability and generalizability in the context of digital divide, and suggest directions for future research.     

The socio-environmental data explorer (SEDE): a social media–enhanced decision support system to explore risk perception to hazard events

ABSTRACT

Social media are increasingly recognized as a useful data source for understanding social response to hazard events in real time and in post-event analysis. This article establishes social media–enhanced decision support systems (SME-DSS) as a synergistic integration of social media and decision support systems (DSSs) to provide structured access to native, near real-time data from a large and diverse population to assess social response to social, environmental, and technological risk and hazard events. We introduce a prototype SME-DSS entitled socio-environmental data explorer (SEDE) to explore the opportunities and challenges of leveraging social media for decision support. We use a winter storm during 25–28 January 2015 that accumulated record amounts of snow along the East Coast of the United States as a case study to evaluate SEDE in helping assess social response to environmental risk and hazard events as well as evaluate social media as a theoretical component within the social amplification of risk framework (SARF) that serves as a theoretical foundation for SME-DSS.     

VGI and crowdsourced data credibility analysis using spam email detection techniques

Volunteered geographic information (VGI) can be considered a subset of crowdsourced data (CSD) and its popularity has recently increased in a number of application areas. Disaster management is one of its key application areas in which the benefits of VGI and CSD are potentially very high. However, quality issues such as credibility, reliability and relevance are limiting many of the advantages of utilising CSD. Credibility issues arise as CSD come from a variety of heterogeneous sources including both professionals and untrained citizens. VGI and CSD are also highly unstructured and the quality and metadata are often undocumented. In the 2011 Australian floods, the general public and disaster management administrators used the Ushahidi Crowd-mapping platform to extensively communicate flood-related information including hazards, evacuations, emergency services, road closures and property damage. This study assessed the credibility of the Australian Broadcasting Corporation’s Ushahidi CrowdMap dataset using a Naïve Bayesian network approach based on models commonly used in spam email detection systems. The results of the study reveal that the spam email detection approach is potentially useful for CSD credibility detection with an accuracy of over 90% using a forced classification methodology.     

Assessing quality of volunteer crowdsourcing contributions: lessons from the Cropland Capture game

Volunteered geographic information (VGI) is the assembly of spatial information based on public input. While VGI has proliferated in recent years, assessing the quality of volunteer-contributed data has proven challenging, leading some to question the efficiency of such programs. In this paper, we compare several quality metrics for individual volunteers’ contributions. The data were the product of the ‘Cropland Capture’ game, in which several thousand volunteers assessed 165,000 images for the presence of cropland over the course of 6 months. We compared agreement between volunteer ratings and an image's majority classification with volunteer self-agreement on repeated images and expert evaluations. We also examined the impact of experience and learning on performance. Volunteer self-agreement was nearly always higher than agreement with majority classifications, and much greater than agreement with expert validations although these metrics were all positively correlated. Volunteer quality showed a broad trend toward improvement with experience, but the highest accuracies were achieved by a handful of moderately active contributors, not the most active volunteers. Our results emphasize the importance of a universal set of expert-validated tasks as a gold standard for evaluating VGI quality.     

Analyzing the effect of earthquakes on OpenStreetMap contribution patterns and tweeting activities

ABSTRACT

Natural disasters, such as wildfires, earthquakes, landslides, or floods, lead to an increase in topical information shared on social media and in increased mapping activities in volunteered geographic information (VGI) platforms. Using earthquakes in Nepal and Central Italy as case studies, this research analyzes the effects of natural disasters on short-term (weeks) and longer-term (half year) changes in OpenStreetMap (OSM) mapping behavior and tweet activities in the affected regions. An increase of activities in OSM during the events can be partially attributed to those focused OSM mapping campaigns, for example, through the Humanitarian OSM Team (HOT). Using source tags in OSM change-sets, it was found that only a small portion of external mappers actually travels to the affected regions, whereas the majority of external mappers relies on desktop mapping instead. Furthermore, the study analyzes the spatio-temporal sequence of posted tweets together with keyword filters to identify a subset of users who most likely traveled to the affected regions for support and rescue operations. It also explores where, geographically, earthquake information spreads within social networks.     

Finding meaningful participation in volunteer geographic information and citizen science: a case comparison in environmental application

ABSTRACT

Volunteer geographic information (VGI) and citizen science (CS) have been used as a way to promote a form of democratization of knowledge. Crowdsourcing facilitates the massive collection of data through nonexpert volunteers. This study compares 57 VGI and CS projects through a cross-case comparison method to explore the level of involvement of participants, based on three main analytic dimensions (project rationale, actor interaction, and data flow) and 12 sub-variables. The primary goal of this research is to analyze the purpose and the role of technology in VGI and CS projects, using three matrices to explore emerging trade-offs. Finally, we synthesized the workflow of VGI and CS project and compared based on project purposes. At the end of this study, five main purposes were found among the selected projects, uncovering that data collection is the most common purpose. A general workflow can be conceptualized, but detail implementation will involve various trade-offs, especially between local involvement and large-area expert-oriented protocol implementations. By exploring diverse relationships among selected variables of analysis in VGI and CS projects, this study pretends to build a foundation for larger, global meta-analysis that can bring some transparency in the epistemic interactions between experts and nonexperts.