Affect detection from text-based virtual improvisation and emotional gesture recognition

Li Zhang, Bryan Yap

Research output: Contribution to journalArticlepeer-review

7 Citations (Scopus)


We have developed an intelligent agent to engage with users in virtual drama improvisation previously. The intelligent agent was able to perform sentence-level affect detection from user inputs with strong emotional indicators. However, we noticed that many inputs with weak or no affect indicators also contain emotional implication but were regarded as neutral expressions by the previous interpretation. In this paper, we employ latent semantic analysis to perform topic theme detection and identify target audiences for such inputs. We also discuss how such semantic interpretation of the dialog contexts is used to interpret affect more appropriately during virtual improvisation. Also, in order to build a reliable affect analyser, it is important to detect and combine weak affect indicators from other channels such as body language. Such emotional body language detection also provides a nonintrusive channel to detect users' experience without interfering with the primary task. Thus, we also make initial exploration on affect detection from several universally accepted emotional gestures.
Original languageEnglish
JournalAdvances in Human-Computer Interaction
Publication statusPublished - 2012


Dive into the research topics of 'Affect detection from text-based virtual improvisation and emotional gesture recognition'. Together they form a unique fingerprint.

Cite this