A Novel Feature Set for Video Emotion Recognition


In video recommendation systems, emotions are used along with several other proposed content-based video features. However, such features are independently based on visual or audio signals and the relationship representing the dependencies between the visual and the audio signals is still unexplored. In order to solve this problem, a novel feature set called HHTC features based on the combination of Hilbert—Huang Transform (HHT) based visual features, HHT-based audio features, and cross-correlation features is proposed in this paper. In addition to the dependencies between the visual and the audio signals, the proposed HHTC features have the ability to indicate the time-varying characteristics of these signals. The proposed features are applied to video emotion recognition with the Support Vector Regression (SVR) with potential use in video affective recommendation systems. Experimental results demonstrate that the proposed approach can achieve an improved performance of video affective recognition.


Computer Science

Research Center/Lab(s)

Intelligent Systems Center

Second Research Center/Lab

Center for High Performance Computing Research


We would like to extend our heartfelt thanks to all the reviewers, without whose assistance the accomplishment of this thesis would have been impossible. This work was supported by the National Natural Science Foundation of China (61572060, 61772060) and CERNET Innovation Project (NGII20151004, NGII20160316).

Keywords and Phrases

Feature extraction; Recommender systems; Speech recognition; Affective recognition; Content analysis; Cross correlations; Emotion recognition; Empirical Mode Decomposition; Huang transform; Support vector regression (SVR); Time-varying characteristics; Mathematical transformations; Article; Content analysis; Correlation analysis; Emotion; Priority journal; Problem solving; Recognition; Support vector machine; Videorecording; Cross-correlation; Hilbert-Huang transform; Video affective content analysis

International Standard Serial Number (ISSN)


Document Type

Article - Journal

Document Version


File Type





© 2018 Elsevier, All rights reserved.

Publication Date

01 May 2018