Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Linearly Augmented Deep Neural Network

Abstract: 

Deep neural networks (DNN) are a powerful tool for many large vocabulary continuous speech recognition (LVCSR) tasks. Training a very deep network is a challenging problem and pre-training techniques are needed in order to achieve the best results. In this paper, we propose a new type of network architecture, Linear Augmented Deep Neural Network (LA-DNN). This type of network augments each non-linear layer with a linear connection from layer input to layer output. The resulting LA-DNN model eliminates the need for pre-training, addresses the gradient vanishing problem for deep net- works, has higher capacity in modeling linear transformations, trains significantly faster than normal DNN, and produces better acoustic models. The proposed model has been evaluated on TIMIT phoneme recognition and AMI speech recognition tasks. Experimental results show that the LA-DNN models can have 70% fewer parameters than a DNN, while still improving accuracy. On the TIMIT phoneme recognition task, the smaller LA-DNN model improves TIMIT phone accuracy by 2% absolute, and AMI word accuracy by 1.7% absolute.

up
1 user has voted: Pegah Ghahremani

Paper Details

Authors:
Pegah Ghahremani, Jasha Droppo, Michael L. Seltzer
Submitted On:
30 April 2016 - 7:54pm
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Jasha Droppo
Paper Code:
SP-L5.6
Document Year:
2016
Cite

Document Files

LinearlyAugmented-Icassp Presentation.pdf

(167 downloads)

Subscribe

[1] Pegah Ghahremani, Jasha Droppo, Michael L. Seltzer, "Linearly Augmented Deep Neural Network", IEEE SigPort, 2016. [Online]. Available: http://sigport.org/1101. Accessed: May. 24, 2017.
@article{1101-16,
url = {http://sigport.org/1101},
author = {Pegah Ghahremani; Jasha Droppo; Michael L. Seltzer },
publisher = {IEEE SigPort},
title = {Linearly Augmented Deep Neural Network},
year = {2016} }
TY - EJOUR
T1 - Linearly Augmented Deep Neural Network
AU - Pegah Ghahremani; Jasha Droppo; Michael L. Seltzer
PY - 2016
PB - IEEE SigPort
UR - http://sigport.org/1101
ER -
Pegah Ghahremani, Jasha Droppo, Michael L. Seltzer. (2016). Linearly Augmented Deep Neural Network. IEEE SigPort. http://sigport.org/1101
Pegah Ghahremani, Jasha Droppo, Michael L. Seltzer, 2016. Linearly Augmented Deep Neural Network. Available at: http://sigport.org/1101.
Pegah Ghahremani, Jasha Droppo, Michael L. Seltzer. (2016). "Linearly Augmented Deep Neural Network." Web.
1. Pegah Ghahremani, Jasha Droppo, Michael L. Seltzer. Linearly Augmented Deep Neural Network [Internet]. IEEE SigPort; 2016. Available from : http://sigport.org/1101