Evolving filter criteria for randomly initialized network pruning in image classification

Xiangru Chen, Chenjing Liu, Peng Hu, Jie Lin, Yunhong Gong, Yingke Chen, Dezhong Peng*, Xue Geng

*Corresponding author for this work

Research output: Contribution to journalArticlepeer-review

Abstract

Network pruning aims to enhance the performance of deep neural networks by eliminating redundant components from the model. However, existing pruning methods typically require a well-trained model and employ fixed, single pruning criteria throughout the pruning cycles. To address these limitations, we propose a novel method called Evolutionary Filter Criteria (EvoFC). This method enables the automated search for the network pruning ratio and criterion during a population-based heuristic search process. We introduce a unique encoding space that represents the chosen pruning criterion and ratio for each layer, facilitating the acquisition of optimal architecture configurations for candidate networks during iterations. Additionally, we devise a novel weight inheritance mechanism to mitigate the computational burden associated with the population-based nature of the method, resulting in a significant reduction in overall training time. We validate our method by applying it to randomly initialized networks and conducting empirical experiments on CIFAR-10/100, ILSVRC-2012 and Places365 datasets. The results demonstrate that our method effectively reduces the number of FLOPs while striking a fine balance between accuracy and computational efficiency. This underscores the practical value of our method in optimizing performance while efficiently utilizing computational resources, particularly when pruning networks starting from random initialization.

Original languageEnglish
Article number127872
Number of pages22
JournalNeurocomputing
Volume594
Early online date17 May 2024
DOIs
Publication statusE-pub ahead of print - 17 May 2024

Cite this