基于全景视频内容的空间音频生成方法

With the development of virtual reality technology, 360-degree panoramic video, as a new form of video, provides a more immersive experience to users due to its feature of full-angle surround view. Generally, in order to obtain a deeper sense of immersion on the basis of panoramic video, typically a matching spatial audio is added to the video. However, it is still a technical difficulty to generate realistic spatial audio to integrate into the video scenes. In this paper, firstly we decompose the elements required to generate spatial audio into three main parts: sound objects, room reverberation, and ambient sound. Based on the video content, these elements are produced through multi-target detection and tracking, room parameter regression, and scene classification modules, respectively. And an ambient sounds database is established to facilitate this process. The spatial audio generation system is established using the method proposed in this paper. On this basis we designed and conducted a user study to verify the effectiveness of this algorithm. Through analyzing statistical data, we verified the effectiveness of this algorithm, and we studied the relative importance of each element in the algorithm.