Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES

Abstract: 

In this paper, we present a pruning technique for maximum en- tropy (MaxEnt) language models. It is based on computing the exact entropy loss when removing each feature from the model, and it ex- plicitly supports backoff features by replacing each removed feature with its backoff. The algorithm computes the loss on the training data, so it is not restricted to models with n-gram like features, al- lowing models with any feature, including long range skips, triggers, and contextual features such as device location.
Results on the 1-billion word corpus show large perplexity im- provements relative for frequency pruned models of comparable size. Automatic speech recognition (ASR) experiments show absolute word error rate improvements in a large-scale cloud based mobile ASR system for Italian.

up
0 users have voted:

Paper Details

Authors:
Tongzhou Chen, Diamantino Caseiro, Pat Rondon
Submitted On:
19 April 2018 - 2:12pm
Short Link:
Type:
Poster
Event:
Presenter's Name:
Tongzhou Chen
Paper Code:
3442
Document Year:
2018
Cite

Document Files

poster.pdf

(84 downloads)

Subscribe

[1] Tongzhou Chen, Diamantino Caseiro, Pat Rondon, "ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES", IEEE SigPort, 2018. [Online]. Available: http://sigport.org/2762. Accessed: Dec. 10, 2018.
@article{2762-18,
url = {http://sigport.org/2762},
author = {Tongzhou Chen; Diamantino Caseiro; Pat Rondon },
publisher = {IEEE SigPort},
title = {ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES},
year = {2018} }
TY - EJOUR
T1 - ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES
AU - Tongzhou Chen; Diamantino Caseiro; Pat Rondon
PY - 2018
PB - IEEE SigPort
UR - http://sigport.org/2762
ER -
Tongzhou Chen, Diamantino Caseiro, Pat Rondon. (2018). ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES. IEEE SigPort. http://sigport.org/2762
Tongzhou Chen, Diamantino Caseiro, Pat Rondon, 2018. ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES. Available at: http://sigport.org/2762.
Tongzhou Chen, Diamantino Caseiro, Pat Rondon. (2018). "ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES." Web.
1. Tongzhou Chen, Diamantino Caseiro, Pat Rondon. ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES [Internet]. IEEE SigPort; 2018. Available from : http://sigport.org/2762