基于注意力机制的混合神经网络模型的文本分类

ext categorization is one of important tasks in natural language processing(NLP). In text categorization, sentence modeling is critical. In exsisting schemes, local correlation of spatial can be effectively captured by convolutional neural network (CNN), and long-term dependencies, historical information and follwing information can obtained by BiLSTM a variant of recurrent neural network (RNN). For their different modeling capabilities, this paper proposes a hybrid neural network model based on the attention mechanism. It first introduces a contribution rate based on BiLSTM to adjust the different influences on historical information and following information and applies the attention mechanism to combine CNN with BiLSTM to figure out the weight of key information for the relief of the loss of text information. The experimental results show that the introduction of attention mechanism and the contribution rate can effectively improve the accuracy of text categorization.