基于可变窗口算法的中文分词应用研究

With the rapid development of the chinese network , real timely processing about the mass chinese data in the network has been more and more focused on . But Automatic Word Segmentation of Chinese Text is an important basic part of the chinese information processing , as is related to efficiency and precision about the system . In the process of word segmentation , there are two big problems which have not been overcome . One is ambiguity identification , and the other is identification of unlisted word . It is a serious obstacle l to deal with the large-scale real texts owing to lots of the variety and the quantity of the Identification of unlisted word. In this paper we first analyzed the two big problems of Chinese Segmentation , and then we analyzed the present methods to solve the problem about the unlisted word . It proposed the algorithm based on the variable window on the Statistics , it showed that the proposed algorithm is effective by the experiments.