一种中文分词的预处理技术

Firstly, this paper analyzes the algorithm based on the vocabulary maximum matching word segmentation, point out its flaws, and then for this defect, proposed a use of high-frequency-words pre-treatment technology, it is based on the characteristics of high-frequency-words, with very little steps to the sentence is divided into as many paragraphs, and then carry out the maximum matching. Lastly through the experimental data proves that the technology will improve the Chinese word segmentation efficiency.