Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

EXTENDED VARIATIONAL INFERENCE FOR PROPAGATING UNCERTAINTY IN CONVOLUTIONAL NEURAL NETWORKS

Abstract: 

Model confidence or uncertainty is critical in autonomous systems as they directly tie to the safety and trustworthiness of
the system. The quantification of uncertainty in the output decisions of deep neural networks (DNNs) is a challenging
problem. The Bayesian framework enables the estimation of the predictive uncertainty by introducing probability distributions
over the (unknown) network weights; however, the propagation of these high-dimensional distributions through
multiple layers and non-linear transformations is mathematically intractable. In this work, we propose an extended variational
inference (eVI) framework for convolutional neural network (CNN) based on tensor Normal distributions (TNDs)
defined over convolutional kernels. Our proposed eVI framework propagates the first two moments (mean and covariance)
of these TNDs through all layers of the CNN. We employ first-order Taylor series linearization to approximate the mean
and covariances passing through the non-linear activations. The uncertainty in the output decision is given by the propagated
covariance of the predictive distribution. Furthermore, we show, through extensive simulations on the MNIST and
CIFAR-10 datasets, that the CNN becomes more robust to Gaussian noise and adversarial attacks.

up
0 users have voted:

Paper Details

Authors:
Dimah Dera, Ghulam Rasool, and Nidhal Bouaynaya
Submitted On:
24 October 2019 - 8:37am
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Nidhal Bouaynaya,
Paper Code:
111
Document Year:
2019
Cite

Document Files

IEEE MLSP presentation 2.pdf

(37)

Subscribe

[1] Dimah Dera, Ghulam Rasool, and Nidhal Bouaynaya, "EXTENDED VARIATIONAL INFERENCE FOR PROPAGATING UNCERTAINTY IN CONVOLUTIONAL NEURAL NETWORKS", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4886. Accessed: Feb. 18, 2020.
@article{4886-19,
url = {http://sigport.org/4886},
author = {Dimah Dera; Ghulam Rasool; and Nidhal Bouaynaya },
publisher = {IEEE SigPort},
title = {EXTENDED VARIATIONAL INFERENCE FOR PROPAGATING UNCERTAINTY IN CONVOLUTIONAL NEURAL NETWORKS},
year = {2019} }
TY - EJOUR
T1 - EXTENDED VARIATIONAL INFERENCE FOR PROPAGATING UNCERTAINTY IN CONVOLUTIONAL NEURAL NETWORKS
AU - Dimah Dera; Ghulam Rasool; and Nidhal Bouaynaya
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4886
ER -
Dimah Dera, Ghulam Rasool, and Nidhal Bouaynaya. (2019). EXTENDED VARIATIONAL INFERENCE FOR PROPAGATING UNCERTAINTY IN CONVOLUTIONAL NEURAL NETWORKS. IEEE SigPort. http://sigport.org/4886
Dimah Dera, Ghulam Rasool, and Nidhal Bouaynaya, 2019. EXTENDED VARIATIONAL INFERENCE FOR PROPAGATING UNCERTAINTY IN CONVOLUTIONAL NEURAL NETWORKS. Available at: http://sigport.org/4886.
Dimah Dera, Ghulam Rasool, and Nidhal Bouaynaya. (2019). "EXTENDED VARIATIONAL INFERENCE FOR PROPAGATING UNCERTAINTY IN CONVOLUTIONAL NEURAL NETWORKS." Web.
1. Dimah Dera, Ghulam Rasool, and Nidhal Bouaynaya. EXTENDED VARIATIONAL INFERENCE FOR PROPAGATING UNCERTAINTY IN CONVOLUTIONAL NEURAL NETWORKS [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4886