中文摘要：

针对视频情感识别中存在运算复杂度高的缺点，提出一种基于时空局部二值模式矩（Temporal-Spatial Local Binary Pattern Moment，TSLBPM）的双模态情感识别方法。首先对视频进行预处理获得表情和姿态序列；然后对表情和姿态序列分别提取TSLBPM特征，计算测试序列与已标记的情感训练集特征间的最小欧氏距离，并将其作为独立证据来构造基本概率分配（Basic Probability Assignment，BPA）；最后使用Dempster-Shafer证据理论联合规则得到情感识别结果。在双模态表情和姿态情感数据库上的实验结果表明，本文提出的时空局部二值模式矩可以快速提取视频图像的时空特征，能有效识别情感状态。与其他方法的对比实验也验证了本文融合方法的优越性。

英文摘要：

To overcome the deficiency of high complexity performance in video emotion recognition, we propose a novel Local Binary Pattern Moment method based on Temporal-Spatial for feature extraction of dual-modality emotion recognition. Firstly, preprocessing is used to obtain the facial expression and posture sequences. Secondly, TSLBPM is utilized to extract the features of the facial expression and posture sequences. The minimum Euclidean distances are selected by calculating the features of the testing sequences and the marked emotion training sets, and they are used as independent evidence to build the Basic Probability Assignment （BPA）. Finally, according to the rules of Dempster-Shafer evidence theory, the expression recognition result is obtained by fused BPA. The experimental results on the FABO expression and posture dual-modality emotion database show the Temporal-Spatial Local Binary Pattern Moment feature of the video image can be extracted quickly and the video emotional state can be effectively identified. What’s more, compared with other methods , the experiments have verified the superiority of fusion.