Sorry, you need to enable JavaScript to visit this website.

Adaptive Signal Variances: CNN Initialization Through Modern Architectures

Primary tabs

Citation Author(s):
Takahiko Henmi, Esmeraldo Ronnie Rey Zara, Yoshihiro Hirohashi, Tsuyoshi Kato
Submitted by:
Esmeraldo Ronni...
Last updated:
26 September 2021 - 3:01am
Document Type:
Document Year:
Presenters Name:
Esmeraldo Ronnie Rey Zara
Paper Code:



Deep convolutional neural networks (CNNs), renowned for their consistent performance, are widely understood by practitioners that the stability of learning depends on the initialization of the model parameters in each layer. Kaiming initialization, the de facto standard, is derived from a much simpler CNN model which consists of only the convolution and fully connected layers. Compared to the current CNN models, the basis CNN model for the Kaiming initialization does not include the max pooling or global average pooling layers. In this study, we derive an new initialization scheme formulated from modern CNN architectures, and empirically investigate the performance of the new initialization methods compared to the standard initialization methods widely used today.

0 users have voted:

Dataset Files