Semi-feature level fusion for bimodal affect regression based on facial and bodily expressions

Yang Zhang, Li Zhang

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)

Abstract

Automatic emotion recognition has been widely studied and applied to various computer vision tasks (e.g. health monitoring, driver state surveillance, personalized learning, and security monitoring). As revealed by recent psychological and behavioral research, facial expressions are good in communicating categorical emotions (e.g. happy, sad, surprise, etc.), while bodily expressions could contribute more to the perception of dimensional emotional states (e.g. arousal and valence). In this paper, we propose a semi-feature level fusion framework that incorporates affective information of both the facial and bodily modalities to draw a more reliable interpretation of users’ emotional states in a valence–arousal space. The Genetic Algorithm is also applied to conduct automatic feature optimization. We subsequently propose an ensemble regression model to robustly predict users’ continuous affective dimensions in the valence–arousal space. The empirical findings indicate that by combining the optimal discriminative bodily features and the derived Action Unit intensities as inputs, the proposed system with adaptive ensemble regressors achieves the best performance for the regression of both the arousal and valence dimensions.
Original languageEnglish
Title of host publicationAAMAS '15 Proceedings of the 2015 International Conference on Autonomous Agents and Multiagent Systems
Place of PublicationNew York
PublisherACM
Pages1557-1565
Volume3
ISBN (Print)978-1-4503-3413-6
Publication statusPublished - 2015
Event14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015) - Istanbul, Turkey
Duration: 1 Jan 2015 → …

Conference

Conference14th International Conference on Autonomous Agents and Multiagent Systems (AAMAS 2015)
Period1/01/15 → …

Fingerprint Dive into the research topics of 'Semi-feature level fusion for bimodal affect regression based on facial and bodily expressions'. Together they form a unique fingerprint.

Cite this