Documents
Poster
Adaptive Signal Variances: CNN Initialization Through Modern Architectures
- Citation Author(s):
- Submitted by:
- Esmeraldo Ronni...
- Last updated:
- 26 September 2021 - 3:01am
- Document Type:
- Poster
- Document Year:
- 2021
- Event:
- Presenters:
- Esmeraldo Ronnie Rey Zara
- Paper Code:
- 1187
- Categories:
- Log in to post comments
Deep convolutional neural networks (CNNs), renowned for their consistent performance, are widely understood by practitioners that the stability of learning depends on the initialization of the model parameters in each layer. Kaiming initialization, the de facto standard, is derived from a much simpler CNN model which consists of only the convolution and fully connected layers. Compared to the current CNN models, the basis CNN model for the Kaiming initialization does not include the max pooling or global average pooling layers. In this study, we derive an new initialization scheme formulated from modern CNN architectures, and empirically investigate the performance of the new initialization methods compared to the standard initialization methods widely used today.