文档介绍：
基于面部表情和语音的多模态情感识别研
究#
张寅,周丽君,周亚同**
5
10
15
20
25
30
(河北工业大学信息工程学院)
摘要:本文对基于面部表情和语音的多模态情感识别进行研究。首先采集特定人面部表情样
本建立面部表情数据库,采集特定人语音样本建立语音数据库。然后利用主成分分析法分别
对表情、语音样本进行特征提取,对提取的特征进行融合;针对语音的情感特性,得出语音
时域特征,并将其与表情特征进行融合。最后利用支持向量机对融合后的特征进行分类,获
得给定测试样本的情感类别信息。由面部表情特征和语音时域特征得到的融合特征,该融合
特征的情感识别效果好于只用主成分分析得到的融合特征的情感识别效果。基于面部表情和
语音的多模态情感识别效果比基于面部表情或语音的单模态情感识别效果好。
关键词:模式识别与智能系统;特征融合;支持向量机;情感识别
中图分类号:
Research on Multimodal Emotion Recognition Based on
Facial Expression and Speech
Zhang Yin, Zhou Lijun, Zhou Yatong
(School of Information Engineering, Hebei University of Technology)
Abstract: This topic is research of multimodal emotion recognition based on facial expression and
speech. First, through colleting certain person’s facial expression samples, the database of facial
expression is established. And through colleting certain person’s speech samples, the database of
speech is established. Then, the features of facial expression and speech samples respectively are
extracted using the ponent analysis (PCA). Considering speech emotion features, the
speech time domain features are obtained. And they are fused of expression features. Finally, the
features after the fusion are classified using support vector machine (SVM).And the emotional
categories of test sample information are given. By the facial expression features and speech time
domain features of fusion, the fusion feature of emotion recognition effect is better than only use
ponent analysis get the fusion feature of emotion recognition effect. The effect of
multimodal emotion recognition based on facial expression and speech is better than the effect of
the single modal emotion recognition based on facial expression or speech.
Key words: pattern recognition and intelligent system;feature fusion;Support Vector Machine
(SVM);emotion recognition
35
0 引言
情感识别在人类的智能活动中起着关键作用,是人类智能的重要标志[1]。随着人工智能
的发展,对人机交互需求也提出新的要求,它包括:能够听懂人所说的、看见人所做的、理
解人的情感、给出适当的反馈等等[2],最终实现人类与机器自然和谐的进行交流。情感识别