基于Spark的混合协同过滤算法改进与实现

iming at optimizing and improving a hybrid collaborative filtering based on spark platform for its sparsity, scalability and personalized recommendation by using the method of algorithms integrated. This paper takes the model of Stacking algorithm integrated to integrate multiple weak recommender units in a linearly weighted into a comprehensive recommender. Firstly, this article optimizes the collaborative filtering based on the nearest neighbor by presorting and adjusting the similarity calculation strategy with popularity and praise degree, and improves the rationality and complexity of similarity calculation. It solves the problem of score sparsity to some extent. At the same time, this algorithm integrates closely distributed computing platform, which can make full use of the advantages of distributed platform to design and implement an increment iterative model of recommendation algorithm by using the Spark streaming and distributed storage structure and it solves the problem that collaborative filtering algorithm is hard to expand and make poor real-time performance. The experimental data uses UCI public data set named MovieLens and NetFlix films score, and the experimental results show that the improved algorithm has a good performance and makes great progress in personalized recommendation, accuracy and scalability compared with the previous algorithms. It provides a feasible algorithm integration scheme for the application of the recommended system.