3D Car Shape Reconstruction from a Contour Sketch using GAN and Lazy Learning

Naoiki Nozawa, Hubert Shum*, Qi Feng, Edmond S. L. Ho, Shigeo Morishima

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

3D car models are heavily used in computer games, visual effects, and even automotive designs. As a result, producing such models with minimal labour costs is increasingly more important. To tackle the challenge, we propose a novel system to reconstruct a 3D car using a single sketch image. The system learns from a synthetic database of 3D car models and their corresponding 2D contour sketches and segmentation masks, allowing effective training with minimal data collection cost. The core of the system is a machine learning pipeline that combines the use of a Generative Adversarial Network (GAN) and lazy learning. GAN, being a deep learning method, is capable of modelling complicated data distributions, enabling the effective modelling of a large variety of cars. Its major weakness is that as a global method, modelling the fine details in the local region is challenging. Lazy learning works well to preserve local features by generating a local subspace with relevant data samples. We demonstrate that the combined use of GAN and lazy learning produces is able to produce high-quality results, in which different types of cars with complicated local features can be generated effectively with a single sketch. Our method outperforms existing ones using other machine learning structures such as the variational autoencoder.
Original languageEnglish
Number of pages141
JournalThe Visual Computer
Early online date16 Apr 2021
DOIs
Publication statusE-pub ahead of print - 16 Apr 2021

Fingerprint

Dive into the research topics of '3D Car Shape Reconstruction from a Contour Sketch using GAN and Lazy Learning'. Together they form a unique fingerprint.

Cite this