Sorry, you need to enable JavaScript to visit this website.

Adaptive Signal Variances: CNN Initialization Through Modern Architectures

Citation Author(s):
Takahiko Henmi, Esmeraldo Ronnie Rey Zara, Yoshihiro Hirohashi, Tsuyoshi Kato
Submitted by:
Esmeraldo Ronni...
Last updated:
26 September 2021 - 3:01am
Document Type:
Poster
Document Year:
2021
Event:
Presenters:
Esmeraldo Ronnie Rey Zara
Paper Code:
1187
 

Deep convolutional neural networks (CNNs), renowned for their consistent performance, are widely understood by practitioners that the stability of learning depends on the initialization of the model parameters in each layer. Kaiming initialization, the de facto standard, is derived from a much simpler CNN model which consists of only the convolution and fully connected layers. Compared to the current CNN models, the basis CNN model for the Kaiming initialization does not include the max pooling or global average pooling layers. In this study, we derive an new initialization scheme formulated from modern CNN architectures, and empirically investigate the performance of the new initialization methods compared to the standard initialization methods widely used today.

up
0 users have voted: