局部特征增强的唇读特征提取模型

With the continuous development of human-computer interaction, the challenging lip reading task has become a research hotspot. As a part of the whole lip reading model, feature extraction module determines the upper limit of the overall ability of the lip reading model from the source. The current lip reading feature extraction model is composed of 2D and 3D convolution layers, which can not meet the strong correlation of local features. In order to solve this problem, this paper first analyzes the shortcomings of existing models based on anatomical principles and basic models; then proposes global features to enhance local features through the interaction of global features and local features; finally, based on the above design ideas, designs a lip reading feature extraction model with local feature enhancement. Through the experimental verification on the data set, the lip reading feature extraction model constructed in this paper has stronger representation ability.