大规模脉冲星候选体信号的无监督聚类分析研究

随着500 m口径球面射电望远镜(Five-hundred-meter Aperture Spherical radio Telescope, FAST)等大型射电望远镜的建设和使用,脉冲星巡天数据进入PB时代.为解决如此大量高速采样的标量数据挖掘问题,促进新天文现象的发现,提出一种基于无监督聚类的脉冲星候选体筛选方案.该方案采用基于密度层次、划分方法的混合聚类算法,结合MapReduce/Spark并行计算模型和基于滑动窗口的分组策略,进而提高大量候选体信号筛选的效率.通过在脉冲星数据集HTRU2 (High Time Resolution Universe)上的对比实验,结果表明该算法能取得较高的精确度和召回率,分别是0.946和0.905,并且当并行节点足够时,该算法的时间复杂度相比串行执行明显下降.可见,该方法为脉冲星观测大数据的分析挖掘提供一种可行思路.

Research on Unsupervised Clustering Analysis of Large-scale Pulsar Candidate Signals

With the construction and use of large radio telescopes such as Five-hundred-meter Aperture Spherical radio Telescope (FAST), pulsar survey data has entered the PB era. To solve the problem of scalar data mining with such a large number of high-speed sampling and promote the discovery of new astronomical phenomena, this paper proposes a pulsar candidate sifting scheme based on unsupervised clustering. This scheme uses a hybrid clustering algorithm based on density hierarchy and division method, combined with MapReduce/Spark parallel computing model and a sliding window-based grouping strategy, thereby improving the efficiency of screening a large number of candidate signals. Comparative experiments on the data set HTRU2 (High Time Resolution Universe) show that the algorithm can achieve higher accuracy and recall rates, which are 0.946 and 0.905, respectively. And when parallel nodes are sufficient, the time complexity of the algorithm is significantly reduced compared to the serial execution method. It can be seen that this method provides a feasible idea for the analysis and mining of big data pulsar observation.