Intelligent facial emotion recognition and semantic-based topic detection for a humanoid robot

Li Zhang, Ming Jiang, Dewan Farid, Alamgir Hossain

Research output: Contribution to journalArticlepeer-review

64 Citations (Scopus)

Abstract

Automatic perception of human affective behaviour from facial expressions and recognition of intentions and social goals from dialogue contexts would greatly enhance natural human robot interaction. This research concentrates on intelligent neural network based facial emotion recognition and Latent Semantic Analysis based topic detection for a humanoid robot. The work has first of all incorporated Facial Action Coding System describing physical cues and anatomical knowledge of facial behaviour for the detection of neutral and six basic emotions from real-time posed facial expressions. Feedforward neural networks (NN) are used to respectively implement both upper and lower facial Action Units (AU) analysers to recognise six upper and 11 lower facial actions including Inner and Outer Brow Raiser, Lid Tightener, Lip Corner Puller, Upper Lip Raiser, Nose Wrinkler, Mouth Stretch etc. An artificial neural network based facial emotion recogniser is subsequently used to accept the derived 17 Action Units as inputs to decode neutral and six basic emotions from facial expressions. Moreover, in order to advise the robot to make appropriate responses based on the detected affective facial behaviours, Latent Semantic Analysis is used to focus on underlying semantic structures of the data and go beyond linguistic restrictions to identify topics embedded in the users’ conversations. The overall development is integrated with a modern humanoid robot platform under its Linux C++ SDKs. The work presented here shows great potential in developing personalised intelligent agents/robots with emotion and social intelligence.
Original languageEnglish
Pages (from-to)5160-5168
JournalExpert Systems with Applications
Volume40
Issue number13
DOIs
Publication statusPublished - 1 Oct 2013

Fingerprint Dive into the research topics of 'Intelligent facial emotion recognition and semantic-based topic detection for a humanoid robot'. Together they form a unique fingerprint.

Cite this