Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Presentation Slides REVE: Regularizing Deep Learning using Variational Entropy Bound

Abstract: 

Studies on generalization performance of machine learning algorithms under the scope of information theory suggest that compressed representations can guarantee good generalization, inspiring many compression-based regularization methods. In this paper, we introduce REVE, a new regularization scheme. Noting that compressing the representation can be sub-optimal, our first contribution is to identify a variable that is directly responsible for the final prediction. Our method aims at compressing the class conditioned entropy of this latter variable. Second, we introduce a variational upper bound on this conditional entropy term. Finally, we propose a scheme to instantiate a tractable loss that is integrated within the training procedure of the neural network and demonstrate its efficiency on different neural networks and datasets.

up
0 users have voted:

Paper Details

Authors:
Antoine Saporta, Yifu Chen, Michael Blot, Matthieu Cord
Submitted On:
20 September 2019 - 12:07am
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Antoine Saporta
Paper Code:
1363
Document Year:
2019
Cite

Document Files

Oral_ICIP19_REVE.pdf

(6)

Subscribe

[1] Antoine Saporta, Yifu Chen, Michael Blot, Matthieu Cord, "Presentation Slides REVE: Regularizing Deep Learning using Variational Entropy Bound", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4757. Accessed: Oct. 18, 2019.
@article{4757-19,
url = {http://sigport.org/4757},
author = {Antoine Saporta; Yifu Chen; Michael Blot; Matthieu Cord },
publisher = {IEEE SigPort},
title = {Presentation Slides REVE: Regularizing Deep Learning using Variational Entropy Bound},
year = {2019} }
TY - EJOUR
T1 - Presentation Slides REVE: Regularizing Deep Learning using Variational Entropy Bound
AU - Antoine Saporta; Yifu Chen; Michael Blot; Matthieu Cord
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4757
ER -
Antoine Saporta, Yifu Chen, Michael Blot, Matthieu Cord. (2019). Presentation Slides REVE: Regularizing Deep Learning using Variational Entropy Bound. IEEE SigPort. http://sigport.org/4757
Antoine Saporta, Yifu Chen, Michael Blot, Matthieu Cord, 2019. Presentation Slides REVE: Regularizing Deep Learning using Variational Entropy Bound. Available at: http://sigport.org/4757.
Antoine Saporta, Yifu Chen, Michael Blot, Matthieu Cord. (2019). "Presentation Slides REVE: Regularizing Deep Learning using Variational Entropy Bound." Web.
1. Antoine Saporta, Yifu Chen, Michael Blot, Matthieu Cord. Presentation Slides REVE: Regularizing Deep Learning using Variational Entropy Bound [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4757