Evolving and Ensembling Deep CNN Architectures for Image Classification

Benjamin Fielding, Tom Lawrence, Li Zhang

Research output: Contribution to conferencePaperpeer-review

14 Citations (Scopus)
46 Downloads (Pure)

Abstract

Deep Convolutional Neural Networks (CNNs) have traditionally been hand-designed owing to the complexity of their construction and the computational requirements of their training. Recently however, there has been an increase in research interest towards automatically designing deep CNNs for specific tasks. Ensembling has been shown to effectively increase the performance of deep CNNs, although usually with a duplication of work and therefore a large increase in computational resources required. In this paper we present a method for automatically designing and ensembling deep CNN models with a central weight repository to avoid work duplication. The models are trained and optimised together using particle swarm optimisation (PSO), with architecture convergence encouraged. At the conclusion of the joint optimisation and training process a base model nomination method is used to determine the best candidates for the ensemble. Two base model nomination methods are proposed, one using the local best particle positions from the PSO process, and one using the contents of the central weight repository. Once the base model pool has been created, the individual models inherit their parameters from the central weight repository and are then finetuned and ensembled in order to create a final system. We evaluate our system on the CIFAR-10 classification dataset and demonstrate improved results over the single global best model suggested by the optimisation process, with a minor increase in resources required by the finetuning process. Our system achieves an error rate of 4.27% on the CIFAR-10 image classification task with only 36 hours of combined optimisation and training on a single NVIDIA GTX 1080Ti GPU.
Original languageEnglish
Number of pages8
Publication statusPublished - 14 Jul 2019
Event2019 International Joint Conference on Neural Networks - InterContinental Budapest Hotel, Budapest, Hungary
Duration: 14 Jul 201919 Jul 2019
https://www.ijcnn.org/

Conference

Conference2019 International Joint Conference on Neural Networks
Abbreviated titleIJCNN 2019
Country/TerritoryHungary
CityBudapest
Period14/07/1919/07/19
Internet address

Fingerprint

Dive into the research topics of 'Evolving and Ensembling Deep CNN Architectures for Image Classification'. Together they form a unique fingerprint.

Cite this