UCP: Uniform Channel Pruning for Deep Convolutional Neural Networks Compression

UCP: Uniform Channel Pruning for Deep Convolutional Neural Networks Compression and Acceleration

To apply deep CNNs to mobile terminals and portable devices, many scholars have recently worked on the compressing and accelerating deep convolutional neural networks. Based on this, we propose a novel uniform channel pruning (UCP) method to prune deep CNN, and the modified squeeze-and-excitation blocks (MSEB) is used to measure the importance of the channels in the convolutional layers.The unimportant channels, including convolutional kernels related to them, are pruned directly, which greatly reduces the storage cost and the number of calculations. There are two types of residual blocks in ResNet. For ResNet with bottlenecks, we use the pruning method with traditional CNN to trim the 3x3 convolutional layer in the middle of the blocks. For ResNet with basic residual blocks, we propose an approach to consistently prune all residual blocks in the same stage to ensure that the compact network structure is dimensionally correct. Considering that the network loses considerable information after pruning and that the larger the pruning amplitude is, the more information that will be lost, we do not choose fine-tuning but retrain from scratch to restore the accuracy of the network after pruning. Finally, we verified our method on CIFAR-10, CIFAR-100 and ILSVRC-2012 for image classification. The results indicate that the performance of the compact network after retraining from scratch, when the pruning rate is small, is better than the original network. Even when the pruning amplitude is large, the accuracy can be maintained or decreased slightly. On the CIFAR-100, when reducing the parameters and FLOPs up to 82% and 62% respectively, the accuracy of VGG-19 even improved by 0.54% after retraining.

UCP：深度卷积神经网络压缩和加速的统一通道修剪

为了将深层CNN应用于移动终端和便携式设备，最近，许多学者致力于压缩和加速深层卷积神经网络。在此基础上，我们提出了一种对深层CNN进行修剪的新颖的统一通道修剪（UCP）方法，并使用改进的挤压和激励块（MSEB）来测量卷积层中通道的重要性。.. 不重要的通道（包括与之相关的卷积内核）被直接修剪，这大大降低了存储成本和计算量。ResNet中有两种类型的剩余块。对于具有瓶颈的ResNet，我们使用带有传统CNN的修剪方法来修剪块中间的3x3卷积层。对于具有基本残差块的ResNet，我们提出了一种在同一阶段对所有残差块进行连续修剪的方法，以确保紧凑的网络结构在尺寸上是正确的。考虑到网络在修剪后会丢失大量信息，并且修剪幅度越大，丢失的信息就越多，我们不选择微调，而是从头开始重新训练以在修剪后恢复网络的准确性。最后，我们在CIFAR-10，CIFAR-100和ILSVRC-2012上验证了我们的图像分类方法。结果表明，当修剪率较小时，从零开始再训练的紧凑型网络的性能要优于原始网络。即使修剪幅度较大，也可以保持或略微降低精度。在CIFAR-100上，当将参数和FLOP分别降低至82％和62％时，重新训练后VGG-19的精度甚至提高了0.54％。 （阅读更多）