Limitations of Majority Agreement in Crowdsourced Image Interpretation

Crowdsourcing can efficiently complete tasks that are difficult to automate, but the quality of crowdsourced data is tricky to evaluate. Algorithms to grade volunteer work often assume that all tasks are similarly difficult, an assumption that is frequently false. We use a cropland identification game with over 2,600 participants and 165,000 unique tasks to investigate how best to evaluate the difficulty of crowdsourced tasks and to what extent this is possible based on volunteer responses alone. Inter‐volunteer agreement exceeded 90% for about 80% of the images and was negatively correlated with volunteer‐expressed uncertainty about image classification. A total of 343 relatively difficult images were independently classified as cropland, non‐cropland or impossible by two experts. The experts disagreed weakly (one said impossible while the other rated as cropland or non‐cropland) on 27% of the images, but disagreed strongly (cropland vs. non‐cropland) on only 7%. Inter‐volunteer disagreement increased significantly with inter‐expert disagreement. While volunteers agreed with expert classifications for most images, over 20% would have been mis‐categorized if only the volunteers’ majority vote was used. We end with a series of recommendations for managing the challenges posed by heterogeneous tasks in crowdsourcing campaigns.     

Beiträge zur Kenntnis von natürlichen und künstlichen Seewasserlagunen

Ohne Zusammenfassung     

The reflection spectrum of liquid sulfur: Implications for Io

The spectral reflectance from 0.38 to 0.75 μm of a column of liquid sulfur has been measured at several temperatures between the melting point (～118°C) and 173°C. Below 160°C the spectral reflectance was observed to vary reversibly as a function of temperature, independent of the previous thermal history of the column. Once the temperature exceeded 160°C, the spectrum would not change given a subsequent decrease in temperature. The spectral reflectance of the liquid-sulfur column at all temperatures was very low (10–19%). Combining this information with Voyager spectrophotometry of Jupiter's satellite Io, it is concluded that liquid sulfur at any temperature on Io's surface would be classified as a “black area” according to the standards used by the Voyager imaging team in their spectrophotometric analysis (L. Soderblom, T. V. Johnson, D. Morrison, E. Danielson, B. L. Smith, J. Veverka, A. Cook, C. Sagan, P. Kupferman, D. Pieri, J. Mosher, C. Avis, J. Gradie, and T. Clancy (1980). Geophys. Res. Lett.7, 963–966).     

Physical properties of the particles composing the Martian dust storm of 1971–1972

Infrared spectra obtained from the Mariner 9 spacecraft during the 1971–1972 dust storm are used to derive information on the composition and particle size distribution of the dust and to study the time evolution of the storm. The dust is not composed of pure granite, basalt, basaltic glass, obsidian, quartz, andesite, or montmorillonite. The infrared spectra suggest that the dust is a mixture of materials, dominated by igneous silicates with >62;60% SiO₂, or weathering products such as clay minerals, but the dust could possibly have a significant component of lower SiO₂ materials such as basalt. Substantial quantities of carbonates, nitrates, or carbon suboxide are excluded from the mixture. All infrared, visible, and ultraviolet data on the Martian surface composition seem consistent with a mixture of basalt and clay minerals or high SiO₂ igneous rocks, with a surface patina of oxides of iron. For all candidate compositions, the data are best matched with a size distribution that approximates a differential power law function of slope ?4. This size distribution is quite similar to terrestial size distributions in regions remote from sources of dust. The relative abundance of particles between 1- and 10-μm radius did not change during the Mariner 9 mission; thus suspended particles did not experience Stokes-Cunningham fallout but instead were supported by turbulence with an eddy diffusion coefficient, $\text{K}{}_{\mathrm{<mtext>e</mtext>}}\text{? 7 × 10}{}^6\text{cm}{}^2\text{sec}{}^{\mathrm{?1}}$. The aerosol optical depth, standardized to 0.3-μm wavelength, varied from about 1.5 early in the mission to about 0.2 at Orbit 200.     

On the temperature dependence of possible S8 infrared bands in planetary atmospheres

Measurements of the temperature dependence between 77 and 333°K of the infrared spectrum of cyclic octatomic sulfur suggest that the 23 μm Jovian feature very tentatively identified by Houck et al. [Science 189, 720–722 (1975)] is not due to S₈; and that the temperature dependence of the frequency of the 835 cm^? band of S₈ may be a useful temperature marker in planetary studies.     

On the nature and visibility of crater-associated streaks on Mars

R. O. Kuzmin has proposed that all crater-associated wind streaks on Mars are depositional and consist of unresolved barchan-like dunes. He claims that any streak can appear either bright or dark relative to its surroundings depending on the azimuth of the Sun relative to the streak axis and on the elevation of the Sun above the horizon. Our studies of the entire Mariner 9 picture collection as well as of available Viking data lend no support to these ideas. We find that the conditions for visibility of bright and dark streaks are identical. In Mariner 9 images both types of streaks are visible for viewing angles ? ? 60°, illumination angles of 15° ? i ? 75°, and over the whole range of phase angles covered (about 15 to 85°). There are numerous examples of dark and light streaks visible at the same azimuth angle of the Sun, contrary to Kuzmin's claim. There is much evidence to indicate that bright and dark streaks differ both in morphology and in character. The common ragged dark streaks are probably erosion scars, while most bright streaks probably represent accumulations of bright dust-storm fallout. There is no evidence at present that these accumulations have a barchan-like texture.