基于潜在语义分析和改进的原型算法的跨语言文本分类

With the rapid development and globalization, more and more organizations have to manage a large amount of documents in different languages while suffering from lack of enough labeled data in corresponding target language. Cross-language text classification (CLTC) is to automatically classify documents in target language with the help of labeled documents in source language. Most previous research work adopted translation technology (manually or machine translation) between source and target languages before they applied traditional text classification methods. However, manual translation will cost highly and machine tranlsation will bring a lot of erros. Besides, the marginal distributions of the training data and test data may be non-identical, so the topic drift problem would be another challenge for CLTC. In this paper, we propose an approach to CLTC using latent semantic analysis to build a common intermediate semantic space thus the documents in different languages can be "folded-into" this space without translation. We also present an advanced prototype-based algorithm to make full use of the information behind the target documents to address the topic drifting problem. The preliminary experiments on the Reuters RCV1/RCV2 Multilingual collection with five languages show that our best result on Spanish-English language pair reaches 18.5% improvement in Macro-F over the traditional translation-based SVM.