基于面部表情和语音的多模态情感识别研究

his topic is research of multimodal emotion recognition based on facial expression and speech. First, through colleting certain person's facial expression samples, the database of facial expression is established. And through colleting certain person's speech samples, the database of speech is established. Then, the features of facial expression and speech samples respectively are extracted using the principal component analysis (PCA). Considering speech emotion features, the speech time domain features are obtained. And they are fused of expression features. Finally, the features after the fusion are classified using support vector machine (SVM).And the emotional categories of test sample information are given. By the facial expression features and speech time domain features of fusion, the fusion feature of emotion recognition effect is better than only use principal component analysis get the fusion feature of emotion recognition effect. The effect of multimodal emotion recognition based on facial expression and speech is better than the effect of the single modal emotion recognition based on facial expression or speech.