3D car shape reconstruction from a single sketch image

Naoiki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho, Shigeo Morishima

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

19 Downloads (Pure)

Abstract

Efficient car shape design is a challenging problem in both the automotive industry and the computer animation/games industry. In this paper, we present a system to reconstruct the 3D car shape from a single 2D sketch image. To learn the correlation between 2D sketches and 3D cars, we propose a Variational Autoencoder deep neural network that takes a 2D sketch and generates a set of multiview depth & mask images, which are more effective representation comparing to 3D mesh, and can be combined to form the 3D car shape. To ensure the volume and diversity of the training data, we propose a feature-preserving car mesh augmentation pipeline for data augmentation. Since deep learning has limited capacity to reconstruct fine-detail features, we propose a lazy learning approach that constructs a small subspace based on a few relevant car samples in the database. Due to the small size of such a subspace, fine details can be represented effectively with a small number of parameters. With a low-cost optimization process, a high-quality car with detailed features is created. Experimental results show that the system performs consistently to create highly realistic cars of substantially different shape and topology, with a very low computational cost.
Original languageEnglish
Title of host publicationProceedings - MIG 2019: ACM Conference on Motion, Interaction, and Games
Subtitle of host publicationNewcastle upon Tyne, England, October 28-30, 2019
EditorsHubert P. H. Shum, Edmond S. L. Ho, Marie-Paule Cani, Tiberiu Popa, Daniel Holden, He Wang
Place of PublicationNew York
PublisherACM
ISBN (Electronic)9781450369947
DOIs
Publication statusPublished - 28 Oct 2019
EventMIG 2019: 12th annual ACM/SIGGRAPH conference on Motion, Interaction and Games - Northumbria University, Newcastle upon Tyne, United Kingdom
Duration: 28 Oct 201930 Oct 2019
http://www.mig2019.website/index.html

Conference

ConferenceMIG 2019
CountryUnited Kingdom
CityNewcastle upon Tyne
Period28/10/1930/10/19
Internet address

Fingerprint Dive into the research topics of '3D car shape reconstruction from a single sketch image'. Together they form a unique fingerprint.

Cite this