Prior-less 3D Human Shape Reconstruction with an Earth Mover’s Distance Informed CNN

Jingtian Zhang; Hubert P. H.  Shum; Kevin McCay; Edmond S. L. Ho

doi:10.1145/3359566.3364694

Prior-less 3D Human Shape Reconstruction with an Earth Mover’s Distance Informed CNN

Jingtian Zhang, Hubert P. H. Shum, Kevin McCay, Edmond S. L. Ho

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

1 Citation (Scopus)

32 Downloads (Pure)

Abstract

We propose a novel end-to-end deep learning framework, capable of 3D human shape reconstruction from a 2D image without the need of a 3D prior parametric model. We employ a “prior-less” representation of the human shape using unordered point clouds. Due to the lack of prior information, comparing the generated and ground truth point clouds to evaluate the reconstruction error is challenging. We solve this problem by proposing an Earth Mover’s Distance (EMD) function to find the optimal mapping between point clouds. Our experimental results show that we are able to obtain a visually accurate estimation of the 3D human shape from a single 2D image, with some inaccuracy for heavily occluded parts.

Original language	English
Title of host publication	Proceedings - MIG 2019: ACM Conference on Motion, Interaction, and Games
Subtitle of host publication	Newcastle upon Tyne, England, October 28-30, 2019
Editors	Hubert P. H. Shum, Edmond S. L. Ho, Marie-Paule Cani, Tiberiu Popa, Daniel Holden, He Wang
Place of Publication	New York
Publisher	ACM
Pages	1-2
Number of pages	2
ISBN (Electronic)	9781450369947
DOIs	https://doi.org/10.1145/3359566.3364694
Publication status	Published - 28 Oct 2019
Event	MIG 2019: 12th annual ACM/SIGGRAPH conference on Motion, Interaction and Games - Northumbria University, Newcastle upon Tyne, United Kingdom Duration: 28 Oct 2019 → 30 Oct 2019 http://www.mig2019.website/index.html

Conference

Conference	MIG 2019
Country/Territory	United Kingdom
City	Newcastle upon Tyne
Period	28/10/19 → 30/10/19
Internet address	http://www.mig2019.website/index.html

Keywords

Human Surface Reconstruction
Deep Learning
CNN
Earth Mover’s Distance

Access to Document

10.1145/3359566.3364694

Zhang et al - Prior-less 3D Human Shape Reconstruction with an Earth Mover's Distance Informed CNN AAMAccepted author manuscript, 0.98 MB

Cite this

Zhang, J., Shum, H. P. H., McCay, K., & Ho, E. S. L. (2019). Prior-less 3D Human Shape Reconstruction with an Earth Mover’s Distance Informed CNN. In H. P. H. Shum, E. S. L. Ho, M-P. Cani, T. Popa, D. Holden, & H. Wang (Eds.), Proceedings - MIG 2019: ACM Conference on Motion, Interaction, and Games: Newcastle upon Tyne, England, October 28-30, 2019 (pp. 1-2). Article 44 ACM. https://doi.org/10.1145/3359566.3364694

Zhang, Jingtian ; Shum, Hubert P. H. ; McCay, Kevin et al. / Prior-less 3D Human Shape Reconstruction with an Earth Mover’s Distance Informed CNN. Proceedings - MIG 2019: ACM Conference on Motion, Interaction, and Games: Newcastle upon Tyne, England, October 28-30, 2019. editor / Hubert P. H. Shum ; Edmond S. L. Ho ; Marie-Paule Cani ; Tiberiu Popa ; Daniel Holden ; He Wang. New York : ACM, 2019. pp. 1-2

@inproceedings{1b0bbde24f0c4600a0d0cf5ad0677e05,

title = "Prior-less 3D Human Shape Reconstruction with an Earth Mover{\textquoteright}s Distance Informed CNN",

abstract = "We propose a novel end-to-end deep learning framework, capable of 3D human shape reconstruction from a 2D image without the need of a 3D prior parametric model. We employ a “prior-less” representation of the human shape using unordered point clouds. Due to the lack of prior information, comparing the generated and ground truth point clouds to evaluate the reconstruction error is challenging. We solve this problem by proposing an Earth Mover{\textquoteright}s Distance (EMD) function to find the optimal mapping between point clouds. Our experimental results show that we are able to obtain a visually accurate estimation of the 3D human shape from a single 2D image, with some inaccuracy for heavily occluded parts.",

keywords = "Human Surface Reconstruction, Deep Learning, CNN, Earth Mover{\textquoteright}s Distance",

author = "Jingtian Zhang and Shum, {Hubert P. H.} and Kevin McCay and Ho, {Edmond S. L.}",

year = "2019",

month = oct,

day = "28",

doi = "10.1145/3359566.3364694",

language = "English",

pages = "1--2",

editor = "Shum, {Hubert P. H.} and Ho, {Edmond S. L.} and Marie-Paule Cani and Tiberiu Popa and Daniel Holden and He Wang",

booktitle = "Proceedings - MIG 2019: ACM Conference on Motion, Interaction, and Games",

publisher = "ACM",

address = "United States",

note = "MIG 2019 : 12th annual ACM/SIGGRAPH conference on Motion, Interaction and Games ; Conference date: 28-10-2019 Through 30-10-2019",

url = "http://www.mig2019.website/index.html",

}

Zhang, J, Shum, HPH, McCay, K & Ho, ESL 2019, Prior-less 3D Human Shape Reconstruction with an Earth Mover’s Distance Informed CNN. in HPH Shum, ESL Ho, M-P Cani, T Popa, D Holden & H Wang (eds), Proceedings - MIG 2019: ACM Conference on Motion, Interaction, and Games: Newcastle upon Tyne, England, October 28-30, 2019., 44, ACM, New York, pp. 1-2, MIG 2019, Newcastle upon Tyne, United Kingdom, 28/10/19. https://doi.org/10.1145/3359566.3364694

Prior-less 3D Human Shape Reconstruction with an Earth Mover’s Distance Informed CNN. / Zhang, Jingtian; Shum, Hubert P. H. ; McCay, Kevin et al.
Proceedings - MIG 2019: ACM Conference on Motion, Interaction, and Games: Newcastle upon Tyne, England, October 28-30, 2019. ed. / Hubert P. H. Shum; Edmond S. L. Ho; Marie-Paule Cani; Tiberiu Popa; Daniel Holden; He Wang. New York: ACM, 2019. p. 1-2 44.

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution › peer-review

TY - GEN

T1 - Prior-less 3D Human Shape Reconstruction with an Earth Mover’s Distance Informed CNN

AU - Zhang, Jingtian

AU - Shum, Hubert P. H.

AU - McCay, Kevin

AU - Ho, Edmond S. L.

PY - 2019/10/28

Y1 - 2019/10/28

N2 - We propose a novel end-to-end deep learning framework, capable of 3D human shape reconstruction from a 2D image without the need of a 3D prior parametric model. We employ a “prior-less” representation of the human shape using unordered point clouds. Due to the lack of prior information, comparing the generated and ground truth point clouds to evaluate the reconstruction error is challenging. We solve this problem by proposing an Earth Mover’s Distance (EMD) function to find the optimal mapping between point clouds. Our experimental results show that we are able to obtain a visually accurate estimation of the 3D human shape from a single 2D image, with some inaccuracy for heavily occluded parts.

AB - We propose a novel end-to-end deep learning framework, capable of 3D human shape reconstruction from a 2D image without the need of a 3D prior parametric model. We employ a “prior-less” representation of the human shape using unordered point clouds. Due to the lack of prior information, comparing the generated and ground truth point clouds to evaluate the reconstruction error is challenging. We solve this problem by proposing an Earth Mover’s Distance (EMD) function to find the optimal mapping between point clouds. Our experimental results show that we are able to obtain a visually accurate estimation of the 3D human shape from a single 2D image, with some inaccuracy for heavily occluded parts.

KW - Human Surface Reconstruction

KW - Deep Learning

KW - CNN

KW - Earth Mover’s Distance

UR - http://www.scopus.com/inward/record.url?scp=85074863727&partnerID=8YFLogxK

U2 - 10.1145/3359566.3364694

DO - 10.1145/3359566.3364694

M3 - Conference contribution

AN - SCOPUS:85074863727

SP - 1

EP - 2

BT - Proceedings - MIG 2019: ACM Conference on Motion, Interaction, and Games

A2 - Shum, Hubert P. H.

A2 - Ho, Edmond S. L.

A2 - Cani, Marie-Paule

A2 - Popa, Tiberiu

A2 - Holden, Daniel

A2 - Wang, He

PB - ACM

CY - New York

T2 - MIG 2019

Y2 - 28 October 2019 through 30 October 2019

ER -

Zhang J, Shum HPH, McCay K, Ho ESL. Prior-less 3D Human Shape Reconstruction with an Earth Mover’s Distance Informed CNN. In Shum HPH, Ho ESL, Cani M-P, Popa T, Holden D, Wang H, editors, Proceedings - MIG 2019: ACM Conference on Motion, Interaction, and Games: Newcastle upon Tyne, England, October 28-30, 2019. New York: ACM. 2019. p. 1-2. 44 doi: 10.1145/3359566.3364694

Prior-less 3D Human Shape Reconstruction with an Earth Mover’s Distance Informed CNN

Abstract

Conference

Keywords

Access to Document

Other files and links

Fingerprint

Cite this