Automatic Sign Dance Synthesis from Gesture-based Sign Language

Naoya Iwamoto; Hubert P. H.  Shum; Wakana Asahina; Shigeo Morishima

doi:10.1145/3359566.3360069

Automatic Sign Dance Synthesis from Gesture-based Sign Language

Naoya Iwamoto, Hubert P. H. Shum, Wakana Asahina, Shigeo Morishima

Computer and Information Sciences Department

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

3 Citations (Scopus)

59 Downloads (Pure)

Abstract

Automatic dance synthesis has become more and more popular due
to the increasing demand in computer games and animations. Existing
research generates dance motions without much consideration
for the context of the music. In reality, professional dancers make
choreography according to the lyrics and music features. In this research, we focus on a particular genre of dance known as sign dance, which combines gesture-based sign language with full body dance motion. We propose a system to automatically generate sign dance from a piece of music and its corresponding sign gesture. The core of the system is a Sign Dance Model trained by multiple regression analysis to represent the correlations between sign dance and sign gesture/music, as well as a set of objective functions to evaluate the quality of the sign dance. Our system can be applied to music
visualization, allowing people with hearing difficulties to understand and enjoy music.

Original language	English
Title of host publication	Motion, Interaction and Games (MIG ’19)
Subtitle of host publication	October 28–30, 2019, Newcastle upon Tyne, United Kingdom
Editors	Hubert P. H. Shum, Edmond S. L. Ho, Marie-Paule Cani, Tiberiu Popa, Daniel Holden, He Wang
Place of Publication	New York, NY, USA
Publisher	ACM
Number of pages	9
ISBN (Electronic)	9781450369947
DOIs	https://doi.org/10.1145/3359566.3360069
Publication status	Published - 28 Oct 2019
Event	MIG 2019: 12th annual ACM/SIGGRAPH conference on Motion, Interaction and Games - Northumbria University, Newcastle upon Tyne, United Kingdom Duration: 28 Oct 2019 → 30 Oct 2019 http://www.mig2019.website/index.html

Conference

Conference	MIG 2019
Country/Territory	United Kingdom
City	Newcastle upon Tyne
Period	28/10/19 → 30/10/19
Internet address	http://www.mig2019.website/index.html

Keywords

Motion Synthesis
Dance
Sign Language
Multiple Regression Analysis

Access to Document

10.1145/3359566.3360069

Iwamoto et al - Automatic Sign Dance Synthesis from Gesture-based Sign Language AAMAccepted author manuscript, 1.71 MB

Cite this

@inproceedings{44b809c61e7c4febbd274d5a323e2411,

title = "Automatic Sign Dance Synthesis from Gesture-based Sign Language",

abstract = "Automatic dance synthesis has become more and more popular dueto the increasing demand in computer games and animations. Existingresearch generates dance motions without much considerationfor the context of the music. In reality, professional dancers makechoreography according to the lyrics and music features. In this research, we focus on a particular genre of dance known as sign dance, which combines gesture-based sign language with full body dance motion. We propose a system to automatically generate sign dance from a piece of music and its corresponding sign gesture. The core of the system is a Sign Dance Model trained by multiple regression analysis to represent the correlations between sign dance and sign gesture/music, as well as a set of objective functions to evaluate the quality of the sign dance. Our system can be applied to musicvisualization, allowing people with hearing difficulties to understand and enjoy music.",

keywords = "Motion Synthesis, Dance, Sign Language, Multiple Regression Analysis",

author = "Naoya Iwamoto and Shum, {Hubert P. H.} and Wakana Asahina and Shigeo Morishima",

year = "2019",

month = oct,

day = "28",

doi = "10.1145/3359566.3360069",

language = "English",

editor = "Shum, {Hubert P. H.} and Ho, {Edmond S. L.} and Marie-Paule Cani and Tiberiu Popa and Daniel Holden and He Wang",

booktitle = "Motion, Interaction and Games (MIG {\textquoteright}19)",

publisher = "ACM",

address = "United States",

note = "MIG 2019 : 12th annual ACM/SIGGRAPH conference on Motion, Interaction and Games ; Conference date: 28-10-2019 Through 30-10-2019",

url = "http://www.mig2019.website/index.html",

}

Iwamoto, N, Shum, HPH, Asahina, W & Morishima, S 2019, Automatic Sign Dance Synthesis from Gesture-based Sign Language. in HPH Shum, ESL Ho, M-P Cani, T Popa, D Holden & H Wang (eds), Motion, Interaction and Games (MIG ’19): October 28–30, 2019, Newcastle upon Tyne, United Kingdom., 18, ACM, New York, NY, USA, MIG 2019, Newcastle upon Tyne, United Kingdom, 28/10/19. https://doi.org/10.1145/3359566.3360069

Automatic Sign Dance Synthesis from Gesture-based Sign Language. / Iwamoto, Naoya; Shum, Hubert P. H. ; Asahina, Wakana et al.
Motion, Interaction and Games (MIG ’19): October 28–30, 2019, Newcastle upon Tyne, United Kingdom. ed. / Hubert P. H. Shum; Edmond S. L. Ho; Marie-Paule Cani; Tiberiu Popa; Daniel Holden; He Wang. New York, NY, USA: ACM, 2019. 18.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Automatic Sign Dance Synthesis from Gesture-based Sign Language

AU - Iwamoto, Naoya

AU - Shum, Hubert P. H.

AU - Asahina, Wakana

AU - Morishima, Shigeo

PY - 2019/10/28

Y1 - 2019/10/28

N2 - Automatic dance synthesis has become more and more popular dueto the increasing demand in computer games and animations. Existingresearch generates dance motions without much considerationfor the context of the music. In reality, professional dancers makechoreography according to the lyrics and music features. In this research, we focus on a particular genre of dance known as sign dance, which combines gesture-based sign language with full body dance motion. We propose a system to automatically generate sign dance from a piece of music and its corresponding sign gesture. The core of the system is a Sign Dance Model trained by multiple regression analysis to represent the correlations between sign dance and sign gesture/music, as well as a set of objective functions to evaluate the quality of the sign dance. Our system can be applied to musicvisualization, allowing people with hearing difficulties to understand and enjoy music.

AB - Automatic dance synthesis has become more and more popular dueto the increasing demand in computer games and animations. Existingresearch generates dance motions without much considerationfor the context of the music. In reality, professional dancers makechoreography according to the lyrics and music features. In this research, we focus on a particular genre of dance known as sign dance, which combines gesture-based sign language with full body dance motion. We propose a system to automatically generate sign dance from a piece of music and its corresponding sign gesture. The core of the system is a Sign Dance Model trained by multiple regression analysis to represent the correlations between sign dance and sign gesture/music, as well as a set of objective functions to evaluate the quality of the sign dance. Our system can be applied to musicvisualization, allowing people with hearing difficulties to understand and enjoy music.

KW - Motion Synthesis

KW - Dance

KW - Sign Language

KW - Multiple Regression Analysis

UR - http://www.scopus.com/inward/record.url?scp=85074814336&partnerID=8YFLogxK

U2 - 10.1145/3359566.3360069

DO - 10.1145/3359566.3360069

M3 - Conference contribution

AN - SCOPUS:85074814336

BT - Motion, Interaction and Games (MIG ’19)

A2 - Shum, Hubert P. H.

A2 - Ho, Edmond S. L.

A2 - Cani, Marie-Paule

A2 - Popa, Tiberiu

A2 - Holden, Daniel

A2 - Wang, He

PB - ACM

CY - New York, NY, USA

T2 - MIG 2019

Y2 - 28 October 2019 through 30 October 2019

ER -

Automatic Sign Dance Synthesis from Gesture-based Sign Language

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this