基于Transformer的中文文本纠错算法

hinese Spelling Correct (CSC) is a task for detecting and correcting spelling errors in Chinese text. At present, most Chinese text error correction work adopts a BERT-based end-to-end language model. Among them, Soft-Masked BERT is one of the mainstream methods. It first regresses the error probability of the character at each position through the detection network, and then the feature vector is encoded by the soft-mask mechanism and sent to the BERT-based error correction network. However, this method uses GRU as an error correction network, which has the problem of not being able to capture long-order dependencies. Therefore, this paper proposes a Transformer-based Chinese text error correction algorithm based on Soft-Masked BERT to solve the above problems. It consists of an error detection network based on stacked Transformer encoders and a BERT-based error correction network. The detection network is connected to the error correction network through a soft-mask mechanism. At the same time, in order to reduce the influence of wrong characters on the results, we removed the jumper structure of the error detection network. The test results on the three benchmarks of SIGHAN13, SIGHAN14 and SIGHAN15 show that the proposed method outperforms Soft-Masked BERT in performance. Among them, the error detection accuracy rate on SIGHAN14 has increased by 2.8%, and the error correction accuracy rate has increased by 2.9%.