JISE


  [1] [2] [3] [4] [5] [6] [7] [8] [9] [10] [11] [12] [13] [14]


Journal of Information Science and Engineering, Vol. 39 No. 2, pp. 975-995


Designing Network Design Strategies Through Gradient Path Analysis


CHIEN-YAO WANG1,3,+, HONG-YUAN MARK LIAO1,3
AND I-HAU YEH2,3
1Institute of Information Science
Academia Sinica
Taipei, 115 Taiwan

2Elan Microelectronics Corporation
Hsinchu, 308 Taiwan

3National Taipei University of Technology
Taipei, 106 Taiwan
E-mail: kinyiu@iis.sinica.edu.tw
+; ihyeh@emc.com.tw; liao@iis.sinica.edu.tw


Designing a high-efficiency and high-quality expressive network architecture has always been the most important research topic in the field of deep learning. Most of today’s network design strategies focus on how to integrate features extracted from different layers, and how to design computing units to effectively extract these features, thereby enhancing the expressiveness of the network. This paper proposes a new network design strategy, i.e., to design the network architecture based on gradient path analysis. On the whole, most of today’s mainstream network design strategies are based on feed forward path, that is, the network architecture is designed based on the data path. In this paper, we hope to enhance the expressive ability of the trained model by improving the network learning ability. Due to the mechanism driving the network parameter learning is the backward propagation algorithm, we design network design strategies based on back propagation path. We propose the gradient path design strategies for the layer-level, the stage-level, and the network-level, and the design strategies are proved to be superior and feasible from theoretical analysis and experiments. The source code of this work is at: https://github.com/WongKinYiu/yolov7.


Keywords: network architecture design, gradient path analysis, partial residual networks, cross stage partial networks, efficient layer aggregation networks

  Retrieve PDF document (JISE_202302_02.pdf)