大规模数据集Spark并行优化谱聚类

针对已有大规模数据集并行谱聚类算法的计算耗时和资源占用巨大等问题，基于当前批处理和图计算兼顾的Spark并行技术，提出了大规模数据集谱聚类的并行优化改进算法，算法通过并行单向迭代避免了相似矩阵计算时的数据重复计算，通过并行位置变换、标量乘法替换及距离缩放优化算法的资源占用，通过近似特征向量替代进一步优化算法的计算量。试验结果验证了算法近特征向量的有效性及在大规模数据集下良好聚类性能和扩展性。

Spark parallel optimization large-scale spectral clustering

To solve the problems of computational time-consuming and resource occupation, which is hard to be prevented in existing spectral clustering on large-scale datasets, based on the Spark technology, an improved parallel optimization algorithm for spectral clustering is proposed. In which, repetitive calculation of data in similar matrix calculations is avoided by parallel one-way iteration, the resource occupancy is optimized by the parallel position transformation, the scalar multiplication replacement and the distance scaling, and the calculation amount is further optimized by the use of the approximate eigenvectors. The experimental results verify the effectiveness of the approximate eigenvectors and the good clustering performance and scalability under large-scale data sets.