基于横向分组查询的多头注意力机制的跨模态检索算法

With the rapid growth of multimedia data, the importance of cross-modal retrieval technology has become increasingly prominent. However, it faces numerous challenges such as complex data processing and low training efficiency. This research focuses on the cross grouped query attention mechanism in cross-modal retrieval, aiming to improve training efficiency and retrieval accuracy. The model utilizes Fast-RCNN to extract image features and BERT to process text information. By calculating the cosine similarity, a relevance matrix is obtained, and a specific pooling strategy is adopted to evaluate the relevance between images and sentences. Its core cross grouped attention mechanism halves the key and value head matrices before entering the linear layer, conducts grouped calculations and then fuses them, reducing the computational load and memory requirements and improving computational efficiency. Moreover, through residual connections, normalization, and processing by the pre-network, the model can operate stably and efficiently. Experiments on the Flickr30K dataset show that although the cross grouped attention model has a slight decrease in image retrieval accuracy, the text retrieval performance is improved, the average accuracy rate has increased by 0.8\%, and the training speed is enhanced by 14.3\%.