Outlier detection with partial information: application to emergency mapping |
| |
Authors: | Davide D’Alimonte Dan Cornford |
| |
Institution: | (1) NASA/Goddard Space Flight Center, Greenbelt, MD, USA;(2) Neural Computing Research Group, Aston University, Birmingham, UK |
| |
Abstract: | This paper, addresses the problem of novelty detection in the case that the observed data is a mixture of a known ‘background’ process contaminated with an unknown other process, which generates the outliers, or novel observations. The framework we
describe here is quite general, employing univariate classification with incomplete information, based on knowledge of the
distribution (the probability density function, pdf) of the data generated by the ‘background’ process. The relative proportion of this ‘background’ component (the prior ‘background’ probability), the pdf and the prior probabilities of all other components are all assumed unknown. The main contribution is a new classification scheme that
identifies the maximum proportion of observed data following the known ‘background’ distribution. The method exploits the Kolmogorov–Smirnov test to estimate the proportions, and afterwards data are Bayes
optimally separated. Results, demonstrated with synthetic data, show that this approach can produce more reliable results
than a standard novelty detection scheme. The classification algorithm is then applied to the problem of identifying outliers in the SIC2004 data set, in order
to detect the radioactive release simulated in the ‘joker’ data set. We propose this method as a reliable means of novelty
detection in the emergency situation which can also be used to identify outliers prior to the application of a more general
automatic mapping algorithm.
|
| |
Keywords: | |
本文献已被 SpringerLink 等数据库收录! |
|