Abstract:

With the rapid development of deep learning, model size and accuracy have been increasing. However, in the quest for greater accuracy, large training datasets are often necessary for training, which often slows down training and exacerbates carbon emissions. To address these challenges, researchers have proposed a number of approaches, including transfer learning. However, existing transfer learning methods either fine-tune the entire network or only a part of it, such as the final classifier layer. The former often leads to slow migration training, and the latter reduces the accuracy of migration training. To solve these problems, a dual-decision adaptive freezing (DDAF) method is proposed for the transfer learning process. First, a group decision module is used to decide on the layers of the neural network that may require freezing. Subsequently, a layer decision module is used to reach a decision on these layers and determine the layers to eventually freeze, thereby finally freezing the layers that need to be frozen, to minimize the possibility of erroneous freezing, improve the accuracy of training, and accelerate the speed of transfer learning training. Extensive experiments showed that the proposed method improved training speed by 1.97 times with minimal loss of accuracy compared to the traditional method of fine-tuning the entire network and significantly improved the accuracy by 34.52% with minimal loss of training speed compared to fine-tuning only the last layer.

Key words: deep learning, transfer learning, image classification, model acceleration, adaptive freezing