Synthesizing Expressive Facial and Speech Animation by Text-to-IPA Translation with Emotion Control

Andreea Stef, Kaveen Perera, Hubert P. H. Shum, Edmond Ho

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

17 Downloads (Pure)

Abstract

Given the complexity of human facial anatomy, animating facial expressions and lip movements for speech is a very time-consuming and tedious task. In this paper, a new text-to-animation framework for facial animation synthesis is proposed. The core idea is to improve the expressiveness of lip-sync animation by incorporating facial expressions in 3D animated characters. This idea is realized as a plug-in in Autodesk Maya, one of the most popular animation platforms in the industry, such that professional animators can effectively apply the method in their existing work. We evaluate the proposed system by conducting two sets of surveys, in which both novice and experienced users participate in the user study to provide feedback and evaluations from different perspectives. The results of the survey highlight the effectiveness of creating realistic facial animations with the use of emotion expressions. Video demos of the synthesized animations are available online at https://git.io/fx5U3

Original languageEnglish
Title of host publication2018 12th International Conference on Software, Knowledge, Information Management & Applications (SKIMA)
Subtitle of host publicationPhnom Penh, Cambodia 3 – 5 December 2018.
Place of PublicationPiscataway, NJ
PublisherIEEE
Pages70-78
Number of pages8
ISBN (Electronic)9781538691410
ISBN (Print)9781538691427
DOIs
Publication statusPublished - 4 Feb 2019
Event12th International Conference on Software, Knowledge, Information Management and Applications - Institute of Technology of Cambodia, Phnom Penh, Cambodia
Duration: 3 Dec 20185 Dec 2018
http://skimanetwork.info/

Publication series

NameInternational Conference on Software, Knowledge, Information Management and Applications (SKIMA), Proceedings
ISSN (Print)2373-082X
ISSN (Electronic)2573-3214

Conference

Conference12th International Conference on Software, Knowledge, Information Management and Applications
Abbreviated titleSKIMA 2018
CountryCambodia
CityPhnom Penh
Period3/12/185/12/18
Internet address

Fingerprint

Dive into the research topics of 'Synthesizing Expressive Facial and Speech Animation by Text-to-IPA Translation with Emotion Control'. Together they form a unique fingerprint.

Cite this