一种基于词义和词频的向量空间模型改进方法

Vector space Model (VSM) is a method of modeling text using Eigenvector, which is widely used in the fields of text categorization and pattern recognition. But when the text content is more, the traditional VSM model may produce the dimension explosion phenomenon, the efficiency is low and the classification effect is difficult to guarantee. Aiming at the phenomenon of VSM, this paper proposes a method to reduce the dimension of text modeling by means of word meaning and frequency, in order to improve efficiency and accuracy. In this paper, we propose a synonym clustering method for polysemy discriminant optimization, combining with the context distinguishing word meaning, weighted by the similarity of the word meaning, and merging the feature items with similar meanings. The new method has greatly reduced the dimension of eigenvector, and polysemy has improved the accuracy of feature extraction. Compared with other text feature extraction and text categorization methods, the results show that the algorithm has a significant improvement in efficiency and accuracy.