Journal of East China Normal University(Natural Science) ›› 2020, Vol. 2020 ›› Issue (5): 68-82.doi: 10.3969/j.issn.1000-5641.202091001

• Methodology and System of Machine Learning • Previous Articles     Next Articles

Methods and progress in deep neural network model compression

LAI Yejing, HAO Shanfeng, HUANG Dingjiang   

  1. School of Data Science and Engineering, East China Normal University, Shanghai 200062, China
  • Received:2020-08-02 Published:2020-09-24

Abstract: The deep neural network (DNN) model achieves strong performance using substantial memory consumption and high computational power, which can be difficult to deploy on hardware platforms with limited resources. To meet these challenges, researchers have made great strides in this field and have formed a wealth of relevant literature and methods. This paper introduces four representative compression methods for deep neural networks used in recent years: network pruning, quantization, knowledge distillation, and compact network design; in particular, the article focuses on the characteristics of these representative models. Finally, evaluation criteria and research prospects of model compression are summarized.

Key words: deep neural network compression, network pruning, quantification, knowledge distillation, compact neural network

CLC Number: