Emotion Transfer for 3D Hand Motion using StarGAN

Jacky C. P. Chan; Ana-Sabina Irimia; Edmond S. L. Ho

doi:10.2312/cgvc.20201146

Emotion Transfer for 3D Hand Motion using StarGAN

Jacky C. P. Chan, Ana-Sabina Irimia, Edmond S. L. Ho

Computer and Information Sciences Department

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

2 Citations (Scopus)

122 Downloads (Pure)

Abstract

In this paper, we propose a new data-driven framework for 3D hand motion emotion transfer. Specifically, we first capture high-quality hand motion using VR gloves. The hand motion data is then annotated with the emotion type and converted to images to facilitate the motion synthesis process and the new dataset will be available to the public. To the best of our knowledge, this is the first public dataset with annotated hand motions. We further formulate the emotion transfer for 3D hand motion as an Image-to-Image translation problem, and it is done by adapting the StarGAN framework. Our new framework is able to synthesize new motions, given target emotion type and an unseen input motion. Experimental results show that our framework can produce high quality and consistent hand motions.

Original language	English
Title of host publication	Computer Graphics & Visual Computing (CGVC) 2020
Editors	Panagiotis D. Ritsos, Kai Xu
Place of Publication	Geneve
Publisher	The Eurographics Association
Number of pages	9
ISBN (Print)	9783038681229
DOIs	https://doi.org/10.2312/cgvc.20201146
Publication status	Published - 2020
Event	CGVC 2020: 38th Computer Graphics & Visual Computing Conference - King's College London, London, United Kingdom Duration: 10 Sept 2020 → 11 Sept 2020 https://cgvc.org.uk/CGVC2020/

Conference

Conference	CGVC 2020
Country/Territory	United Kingdom
City	London
Period	10/09/20 → 11/09/20
Internet address	https://cgvc.org.uk/CGVC2020/

Keywords

hand animation
style transfer
emotion
generative adversarial network
motion capture

Access to Document

10.2312/cgvc.20201146

CGVC2020__Final__Emotion_Transfer_for_3D_Hand_Motion_using_StarGAN_lowProof, 1.26 MB

Cite this

@inproceedings{d9d26ca25b7a461f8c2b62fbbb9ee296,

title = "Emotion Transfer for 3D Hand Motion using StarGAN",

abstract = "In this paper, we propose a new data-driven framework for 3D hand motion emotion transfer. Specifically, we first capture high-quality hand motion using VR gloves. The hand motion data is then annotated with the emotion type and converted to images to facilitate the motion synthesis process and the new dataset will be available to the public. To the best of our knowledge, this is the first public dataset with annotated hand motions. We further formulate the emotion transfer for 3D hand motion as an Image-to-Image translation problem, and it is done by adapting the StarGAN framework. Our new framework is able to synthesize new motions, given target emotion type and an unseen input motion. Experimental results show that our framework can produce high quality and consistent hand motions.",

keywords = "hand animation, style transfer, emotion, generative adversarial network, motion capture",

author = "Chan, {Jacky C. P.} and Ana-Sabina Irimia and Ho, {Edmond S. L.}",

year = "2020",

doi = "10.2312/cgvc.20201146",

language = "English",

isbn = "9783038681229",

editor = "Ritsos, {Panagiotis D.} and Kai Xu",

booktitle = "Computer Graphics & Visual Computing (CGVC) 2020",

publisher = "The Eurographics Association",

note = "CGVC 2020 : 38th Computer Graphics & Visual Computing Conference ; Conference date: 10-09-2020 Through 11-09-2020",

url = "https://cgvc.org.uk/CGVC2020/",

}

TY - GEN

T1 - Emotion Transfer for 3D Hand Motion using StarGAN

AU - Chan, Jacky C. P.

AU - Irimia, Ana-Sabina

AU - Ho, Edmond S. L.

PY - 2020

Y1 - 2020

N2 - In this paper, we propose a new data-driven framework for 3D hand motion emotion transfer. Specifically, we first capture high-quality hand motion using VR gloves. The hand motion data is then annotated with the emotion type and converted to images to facilitate the motion synthesis process and the new dataset will be available to the public. To the best of our knowledge, this is the first public dataset with annotated hand motions. We further formulate the emotion transfer for 3D hand motion as an Image-to-Image translation problem, and it is done by adapting the StarGAN framework. Our new framework is able to synthesize new motions, given target emotion type and an unseen input motion. Experimental results show that our framework can produce high quality and consistent hand motions.

AB - In this paper, we propose a new data-driven framework for 3D hand motion emotion transfer. Specifically, we first capture high-quality hand motion using VR gloves. The hand motion data is then annotated with the emotion type and converted to images to facilitate the motion synthesis process and the new dataset will be available to the public. To the best of our knowledge, this is the first public dataset with annotated hand motions. We further formulate the emotion transfer for 3D hand motion as an Image-to-Image translation problem, and it is done by adapting the StarGAN framework. Our new framework is able to synthesize new motions, given target emotion type and an unseen input motion. Experimental results show that our framework can produce high quality and consistent hand motions.

KW - hand animation

KW - style transfer

KW - emotion

KW - generative adversarial network

KW - motion capture

U2 - 10.2312/cgvc.20201146

DO - 10.2312/cgvc.20201146

M3 - Conference contribution

SN - 9783038681229

BT - Computer Graphics & Visual Computing (CGVC) 2020

A2 - Ritsos, Panagiotis D.

A2 - Xu, Kai

PB - The Eurographics Association

CY - Geneve

T2 - CGVC 2020

Y2 - 10 September 2020 through 11 September 2020

ER -

Emotion Transfer for 3D Hand Motion using StarGAN

Abstract

Conference

Keywords

Access to Document

Fingerprint

Cite this