Algorithm-architecture co-design for CNN acceleration exploiting linear association of weights

Abstract

To deploy convolutional neural networks (CNNs) on resource-constrained edge devices and accelerate inference, a hardware-aware CNN optimization algorithm named linear association of weights (LAW) was proposed in this paper. The algorithm reconstructs integer dot-product computations in quantized models, significantly reducing multiplication operations while maintaining accuracy. The implementation works at two granularities. At the single-kernel level, it transforms multiplications into efficient shift-and-add operations by exploiting linear relationships among weights. At the multi-channel level, a cross-channel weight association computation model ensures algorithm generality across different convolutional layers. Based on LAW, a configurable CNN hardware accelerator is designed, featuring an acceleration core that decodes weight association patterns and replaces conventional multipliers with shift-and-add combinations for hardware-level optimization. Experimental results demonstrate that compared with traditional 8-bit integer (int8) quantization, the LAW algorithm reduces 54% of multiplications for 7 × 7 convolution kernels and achieves 63% multiplication reduction for multi-channel 3 × 3 convolutional layers. When evaluated with residual network-18 (ResNet-18) on the ImageNet Large Scale Visual Recognition Challenge 2012 (ILSVRC2012) dataset, the proposed accelerator achieves 40.21 ms inference latency per image with 70.16% top-1 accuracy and 90.12 giga operations per second (GOPS) peak computation amount performance. As an innovative algorithm specifically designed for vision tasks, LAW provides effective hardware acceleration support for vision-oriented deep learning networks, making it particularly suitable for unmanned systems and industrial inspection.

Keywords

accelerated inference / hardware-aware / linear association / shift-and-add

References

[1] LECUN Y, BENGIO Y, HINTON G. Deep learning. Nature, 2015, 521(7553): 436-444.

[2] HUANG A, CAO Z, WANG C R, et al. An FPGA-based on-chip neural network for TDLAS tomography in dynamic flames. IEEE Transactions on Instrumentation and Measurement, 2021, 70: Article 4506911.

[3] SAHANI M, DASH P K . Deep convolutional stack autoencoder of process adaptive VMD data with robust multikernel RVFLN for power quality events recognition. IEEE Transactions on Instrumentation and Measurement, 2021, 70: Article 9001912.

[4] ABBASNEZHAD M, ALIZADEH B . FPGA-based implementation of an artificial neural network for measurement acceleration in BOTDA sensors. IEEE Transactions on Instrumentation and Measurement, 2019, 68(11): 4326-4334.

[5] ZHOU X D, DU Z D, ZHANG S J, et al. Addressing sparsity in deep neural networks. IEEE Transactions on Computer-Aided Design of Integrated Circuits and Systems, 2019, 38(10): 1858-1871.

[6] DENG L, LI G Q, HAN S, et al. Model compression and hardware acceleration for neural networks: a comprehensive survey. Proceedings of the IEEE, 2020, 108(4): 485-532.

[7] LEE J H, KIM J, KWON S J, et al. FlexRound: learnable rounding based on element-wise division for post-training quantization. Proceedings of the 40th International Conference on Machine Learning (ICML'23), 2023, Jul 23-29, Honolulu, HI, USA. 2023: 27p.

[8] NAGEL M, FOURNARAKIS M, AMJAD R A, et al. A white paper on neural network quantization. arXiv Preprint, ArXiv: 2106.08295, 2021.

[9] CHANG S E, LI Y Y, SUN M S, et al. Mix and match: a novel FPGA-centric deep neural network quantization framework. Proceedings of the 2021 IEEE International Symposium on High-Performance Computer Architecture (HPCA'21), 2021, Feb 27-Mar 3, Seoul, Republic of Korea. Piscataway, NJ, USA: IEEE, 2021: 208-220.

[10] WANG K, LIU Z J, LIN Y J, et al. HAQ: hardware-aware automated quantization with mixed precision. Proceedings of the 2019 IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR'19), 2019, Jun 15-20, Long Beach, CA, USA. Piscataway, NJ, USA: IEEE, 2019: 8612-8620.

[11] CHEN W H, WANG P S, CHENG J . Towards mixed-precision quantization of neural networks via constrained optimization. Proceedings of the 2021 IEEE/CVF International Conference on Computer Vision (ICCV'21), 2021, Oct 10-17, Montreal, Canada. Piscataway, NJ, USA: IEEE, 2021: 5350-5359.

[12] XIAO Q C, LIANG Y . Zac: towards automatic optimization and deployment of quantized deep neural networks on embedded devices. Proceedings of the 2019 IEEE/ACM International Conference on Computer-Aided Design (ICCAD'19), 2019, Nov 4-7, Westminster, CO, USA. Piscataway, NJ, USA: IEEE, 2019: 6p.

[13] JIANG T, XING L G, YU J M, et al. A hardware-friendly logarithmic quantization method for CNNs and FPGA implementation. Journal of Real-Time Image Processing, 2024, 21(4): Article 108.