基于spark平台的K-means改进算法

iming at the problem that the number of K values and initial clusteringcenter in classical K-means algorithmneed to be artificially specified and that classical serial K-means algorithm in the face of massive data, a canopy- Kmeans algorithm is raised. The algorithm introduces the canopy algorithm as a pre-algorithm of the K-means algorithm to get the initial clustering center point and K value, and combines the Spark framework to parallelize the algorithm. It takes full advantage of Spark\'s memory computing advantages and improves the clustering efficiency. Experiments show that the canopy-Kmeans algorithm has higher accuracy and efficiency than the traditional K-means algorithm and unmodified parallel algorithm.