Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning

Naoiki Nozawa, Hubert P. H. Shum, Edmond S. L. Ho, Shigeo Morishima

Research output: Chapter in Book/Report/Conference proceedingConference contributionpeer-review

1 Citation (Scopus)
7 Downloads (Pure)

Abstract

Efficient car shape design is a challenging problem in both the automotive industry and the computer animation/games industry. In this paper, we present a system to reconstruct the 3D car shape from a single 2D sketch image. To learn the correlation between 2D sketches and 3D cars, we propose a Variational Autoencoder deep neural network that takes a 2D sketch and generates a set of multi-view depth and mask images, which form a more effective representation comparing to 3D meshes, and can be effectively fused to generate a 3D car shape. Since global models like deep learning have limited capacity to econstruct fine-detail features, we propose a local lazy learning approach that constructs a small subspace based on a few relevant car samples in the database. Due to the small size of such a subspace, fine details can be represented effectively with a small number of parameters. With a low-cost optimization process, a high-quality car shape with detailed features is created. Experimental results show that the system performs consistently to create highly realistic cars of substantially different shape and topology.
Original languageEnglish
Title of host publicationProceedings of the 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2020)
Subtitle of host publicationVolume 1: GRAPP
EditorsKadi Bouatouch, A. Augusto Sousa, Jose Braz
PublisherScitepress
Pages179-190
Number of pages12
Volume1
ISBN (Electronic)9789897584022
DOIs
Publication statusPublished - 20 Mar 2020
EventGRAPP 2020: 15th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Grand Hotel Excelsior in Valletta, Valletta, Malta
Duration: 27 Feb 202029 Feb 2020
http://www.grapp.visigrapp.org/

Conference

ConferenceGRAPP 2020
Abbreviated titleGRAPP
CountryMalta
CityValletta
Period27/02/2029/02/20
Internet address

Fingerprint

Dive into the research topics of 'Single Sketch Image based 3D Car Shape Reconstruction with Deep Learning and Lazy Learning'. Together they form a unique fingerprint.

Cite this