一种基于主题的网页实时分类模型的研究

In this paper, the general classification model is studied firstly，and analyzing the Inadequacies of the general model for real-time classification of the webpage.On this basis,for more suitable for real-time classification,this paper presents a classification model based on the theme.Firstly,constructing the theme of vertical search engine crawlers through Nutch,and the webpage can be crawled all the time,so it can ensure the real-time web.Secondly,removing part of the pages witch has nothing to do with the classification by processing the crawling results of Nutch through theme denoising.In the end,the webpages crawled can be classfried.The experiment show that the speed and accuracy can be improved with the model.For the requirement of big data of the webpage classification on real-time,this model can effectively optimize the input sample and save computing time.