A Novel Feature Set for Video Emotion Recognition
Abstract
In video recommendation systems, emotions are used along with several other proposed content-based video features. However, such features are independently based on visual or audio signals and the relationship representing the dependencies between the visual and the audio signals is still unexplored. In order to solve this problem, a novel feature set called HHTC features based on the combination of Hilbert—Huang Transform (HHT) based visual features, HHT-based audio features, and cross-correlation features is proposed in this paper. In addition to the dependencies between the visual and the audio signals, the proposed HHTC features have the ability to indicate the time-varying characteristics of these signals. The proposed features are applied to video emotion recognition with the Support Vector Regression (SVR) with potential use in video affective recommendation systems. Experimental results demonstrate that the proposed approach can achieve an improved performance of video affective recognition.
Recommended Citation
S. Mo et al., "A Novel Feature Set for Video Emotion Recognition," Neurocomputing, vol. 291, pp. 11 - 20, Elsevier, May 2018.
The definitive version is available at https://doi.org/10.1016/j.neucom.2018.02.052
Department(s)
Computer Science
Research Center/Lab(s)
Intelligent Systems Center
Second Research Center/Lab
Center for High Performance Computing Research
Keywords and Phrases
Feature extraction; Recommender systems; Speech recognition; Affective recognition; Content analysis; Cross correlations; Emotion recognition; Empirical Mode Decomposition; Huang transform; Support vector regression (SVR); Time-varying characteristics; Mathematical transformations; Article; Content analysis; Correlation analysis; Emotion; Priority journal; Problem solving; Recognition; Support vector machine; Videorecording; Cross-correlation; Hilbert-Huang transform; Video affective content analysis
International Standard Serial Number (ISSN)
0925-2312
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2018 Elsevier, All rights reserved.
Publication Date
01 May 2018
Comments
We would like to extend our heartfelt thanks to all the reviewers, without whose assistance the accomplishment of this thesis would have been impossible. This work was supported by the National Natural Science Foundation of China (61572060, 61772060) and CERNET Innovation Project (NGII20151004, NGII20160316).