华东师范大学学报(自然科学版) ›› 2022, Vol. 2022 ›› Issue (4): 31-42.doi: 10.3969/j.issn.1000-5641.2022.04.004

• 计算机科学 • 上一篇    下一篇

基于递归结构的神经网络架构搜索算法

李继洲, 林欣*()   

  1. 华东师范大学 计算机科学与技术学院, 上海 200062
  • 收稿日期:2021-01-07 出版日期:2022-07-25 发布日期:2022-07-19
  • 通讯作者: 林欣 E-mail:xlin@cs.ecnu.edu.cn
  • 基金资助:
    中央高校基本科研业务费专项资金; 国防科技大学并行分布式处理重点实验室开放项目

Neural architecture search algorithms based on a recursive structure

Jizhou LI, Xin LIN*()   

  1. School of Computer Science and Technology, East China Normal University, Shanghai 200062, China
  • Received:2021-01-07 Online:2022-07-25 Published:2022-07-19
  • Contact: Xin LIN E-mail:xlin@cs.ecnu.edu.cn

摘要:

神经网络架构搜索算法旨在通过计算机的启发式搜索代替人工搜索, 在巨大的神经网络结构空间中寻找更为高效的神经网络结构. 许多研究通过引入各种对搜索空间的约束来解决早期神经网络结构搜索效率低下、耗时长的问题. 然而, 对于搜索空间的约束虽然能够提升并稳定所搜索到的模型的性能, 但同时也导致了很多潜在的高性能模型结构无法被搜索到. 构建了一种更为关注神经网络宏观结构的递归型搜索空间, 并提出通过分步渐进搜索方案探索这个搜索空间的神经网络架构搜索算法. 实验表明, 该算法能在复杂的搜索空间中高效完成神经网络架构搜索任务, 但与最新的基于受约束搜索空间的神经网络架构搜索算法相比仍稍有差距.

关键词: 神经网络架构搜索, 图像分类, 卷积神经网络

Abstract:

Neural architecture search algorithms aim to find more efficient neural network structures in a huge neural network structure space using computer heuristic search instead of manual search. Previous studies have addressed the problem of inefficient and time-consuming search for early neural network structures by introducing various constraints on the search space. While constraints on the search space can improve and stabilize the performance of the model, they ignore potentially efficient model structures. Hence, in this study, we constructed a recursive model search space that focuses more on the macroscopic structure of neural networks. We proposed a neural architecture search algorithm that explores this search space through a step-by-step incremental search approach. Experiments showed that the algorithm can efficiently perform neural architecture search tasks in complex search spaces, but still fell slightly short of the latest constrained search space-based neural architecture search algorithms.

Key words: neural architecture search, image classification, convolutional neural network

中图分类号: