Sorry, you need to enable JavaScript to visit this website.

Language Modeling, for Speech and SLP (SLP-LANG)

Phoneme Level Language Models for Sequence Based Low Resource ASR


Building multilingual and crosslingual models help bring different languages together in a language universal space. It allows models to share parameters and transfer knowledge across languages, enabling faster and better adaptation to a new language. These approaches are particularly useful for low resource languages. In this paper, we propose a phoneme-level language model that can be used multilingually and for crosslingual adaptation to a target language.

Paper Details

Authors:
Siddharth Dalmia, Xinjian Li, Alan W Black, Florian Metze
Submitted On:
14 May 2019 - 10:39am
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

PLMs_ICASSP_Poster (1).pdf

(34)

Subscribe

[1] Siddharth Dalmia, Xinjian Li, Alan W Black, Florian Metze, "Phoneme Level Language Models for Sequence Based Low Resource ASR", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4511. Accessed: Sep. 20, 2019.
@article{4511-19,
url = {http://sigport.org/4511},
author = {Siddharth Dalmia; Xinjian Li; Alan W Black; Florian Metze },
publisher = {IEEE SigPort},
title = {Phoneme Level Language Models for Sequence Based Low Resource ASR},
year = {2019} }
TY - EJOUR
T1 - Phoneme Level Language Models for Sequence Based Low Resource ASR
AU - Siddharth Dalmia; Xinjian Li; Alan W Black; Florian Metze
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4511
ER -
Siddharth Dalmia, Xinjian Li, Alan W Black, Florian Metze. (2019). Phoneme Level Language Models for Sequence Based Low Resource ASR. IEEE SigPort. http://sigport.org/4511
Siddharth Dalmia, Xinjian Li, Alan W Black, Florian Metze, 2019. Phoneme Level Language Models for Sequence Based Low Resource ASR. Available at: http://sigport.org/4511.
Siddharth Dalmia, Xinjian Li, Alan W Black, Florian Metze. (2019). "Phoneme Level Language Models for Sequence Based Low Resource ASR." Web.
1. Siddharth Dalmia, Xinjian Li, Alan W Black, Florian Metze. Phoneme Level Language Models for Sequence Based Low Resource ASR [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4511

Automatic Diagnosis of Alzheimer's Disease Using Neural Network Language Models

Paper Details

Authors:
Julian Fritsch, Sebastian Wankerl, Elmar Nöth
Submitted On:
10 May 2019 - 10:53am
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

ADclassMA_presentation.pdf

(23)

Subscribe

[1] Julian Fritsch, Sebastian Wankerl, Elmar Nöth, "Automatic Diagnosis of Alzheimer's Disease Using Neural Network Language Models", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4344. Accessed: Sep. 20, 2019.
@article{4344-19,
url = {http://sigport.org/4344},
author = {Julian Fritsch; Sebastian Wankerl; Elmar Nöth },
publisher = {IEEE SigPort},
title = {Automatic Diagnosis of Alzheimer's Disease Using Neural Network Language Models},
year = {2019} }
TY - EJOUR
T1 - Automatic Diagnosis of Alzheimer's Disease Using Neural Network Language Models
AU - Julian Fritsch; Sebastian Wankerl; Elmar Nöth
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4344
ER -
Julian Fritsch, Sebastian Wankerl, Elmar Nöth. (2019). Automatic Diagnosis of Alzheimer's Disease Using Neural Network Language Models. IEEE SigPort. http://sigport.org/4344
Julian Fritsch, Sebastian Wankerl, Elmar Nöth, 2019. Automatic Diagnosis of Alzheimer's Disease Using Neural Network Language Models. Available at: http://sigport.org/4344.
Julian Fritsch, Sebastian Wankerl, Elmar Nöth. (2019). "Automatic Diagnosis of Alzheimer's Disease Using Neural Network Language Models." Web.
1. Julian Fritsch, Sebastian Wankerl, Elmar Nöth. Automatic Diagnosis of Alzheimer's Disease Using Neural Network Language Models [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4344

ADVERSARIAL MULTI-TASK DEEP FEATURES AND UNSUPERVISED BACK-END ADAPTATION FOR LANGUAGE RECOGNITION

Paper Details

Authors:
Tan Lee
Submitted On:
8 May 2019 - 9:54pm
Short Link:
Type:
Event:
Presenter's Name:
Document Year:
Cite

Document Files

AdvMTLAndPLDAAdapt4LR.pdf

(35)

Subscribe

[1] Tan Lee, "ADVERSARIAL MULTI-TASK DEEP FEATURES AND UNSUPERVISED BACK-END ADAPTATION FOR LANGUAGE RECOGNITION", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4139. Accessed: Sep. 20, 2019.
@article{4139-19,
url = {http://sigport.org/4139},
author = {Tan Lee },
publisher = {IEEE SigPort},
title = {ADVERSARIAL MULTI-TASK DEEP FEATURES AND UNSUPERVISED BACK-END ADAPTATION FOR LANGUAGE RECOGNITION},
year = {2019} }
TY - EJOUR
T1 - ADVERSARIAL MULTI-TASK DEEP FEATURES AND UNSUPERVISED BACK-END ADAPTATION FOR LANGUAGE RECOGNITION
AU - Tan Lee
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4139
ER -
Tan Lee. (2019). ADVERSARIAL MULTI-TASK DEEP FEATURES AND UNSUPERVISED BACK-END ADAPTATION FOR LANGUAGE RECOGNITION. IEEE SigPort. http://sigport.org/4139
Tan Lee, 2019. ADVERSARIAL MULTI-TASK DEEP FEATURES AND UNSUPERVISED BACK-END ADAPTATION FOR LANGUAGE RECOGNITION. Available at: http://sigport.org/4139.
Tan Lee. (2019). "ADVERSARIAL MULTI-TASK DEEP FEATURES AND UNSUPERVISED BACK-END ADAPTATION FOR LANGUAGE RECOGNITION." Web.
1. Tan Lee. ADVERSARIAL MULTI-TASK DEEP FEATURES AND UNSUPERVISED BACK-END ADAPTATION FOR LANGUAGE RECOGNITION [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4139

EVERY RATING MATTERS: JOINT LEARNING OF SUBJECTIVE LABELS AND INDIVIDUAL ANNOTATORS FOR SPEECH EMOTION CLASSIFICATION


The subjectivity and variability exist in the human emotion perception differs from person to person. In this work, we propose a framework that models the majority of emotion annotation integrated with modeling of subjectivity in improving emotion categorization performances. Our method achieves a promising accuracy of 61.48% on a four-class emotion recognition task. To the best of our knowledge, while there are many works in studying annotator subjectivity, this is one of the first works that have explicitly modeled jointly the consensus with individuality in emotion perception to demonstrate its improvement in classifying emotion in a benchmark corpus.

In our immediate future work, we will evaluate the proposed framework on other public large-scaled emotional database with multiple annotators, e.g., NNIME, to further justify its robustness. We also plan to extend our framework to includ other behavior attributes, e.g., lexical content and body movements. Furthermore, the subjective nature of emotion perception has been shown to be related to the rater personality, a joint modeling of rater’s characteristics with his/her subjectivity in emotion perception may lead to further advancement in robust emotion recognition.

Paper Details

Authors:
Huang-Cheng Chou, Chi-Chun Lee
Submitted On:
30 May 2019 - 2:17am
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

Talk Slides

(30)

Full Paper

(29)

Keywords

Additional Categories

Subscribe

[1] Huang-Cheng Chou, Chi-Chun Lee, "EVERY RATING MATTERS: JOINT LEARNING OF SUBJECTIVE LABELS AND INDIVIDUAL ANNOTATORS FOR SPEECH EMOTION CLASSIFICATION", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4006. Accessed: Sep. 20, 2019.
@article{4006-19,
url = {http://sigport.org/4006},
author = {Huang-Cheng Chou; Chi-Chun Lee },
publisher = {IEEE SigPort},
title = {EVERY RATING MATTERS: JOINT LEARNING OF SUBJECTIVE LABELS AND INDIVIDUAL ANNOTATORS FOR SPEECH EMOTION CLASSIFICATION},
year = {2019} }
TY - EJOUR
T1 - EVERY RATING MATTERS: JOINT LEARNING OF SUBJECTIVE LABELS AND INDIVIDUAL ANNOTATORS FOR SPEECH EMOTION CLASSIFICATION
AU - Huang-Cheng Chou; Chi-Chun Lee
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4006
ER -
Huang-Cheng Chou, Chi-Chun Lee. (2019). EVERY RATING MATTERS: JOINT LEARNING OF SUBJECTIVE LABELS AND INDIVIDUAL ANNOTATORS FOR SPEECH EMOTION CLASSIFICATION. IEEE SigPort. http://sigport.org/4006
Huang-Cheng Chou, Chi-Chun Lee, 2019. EVERY RATING MATTERS: JOINT LEARNING OF SUBJECTIVE LABELS AND INDIVIDUAL ANNOTATORS FOR SPEECH EMOTION CLASSIFICATION. Available at: http://sigport.org/4006.
Huang-Cheng Chou, Chi-Chun Lee. (2019). "EVERY RATING MATTERS: JOINT LEARNING OF SUBJECTIVE LABELS AND INDIVIDUAL ANNOTATORS FOR SPEECH EMOTION CLASSIFICATION." Web.
1. Huang-Cheng Chou, Chi-Chun Lee. EVERY RATING MATTERS: JOINT LEARNING OF SUBJECTIVE LABELS AND INDIVIDUAL ANNOTATORS FOR SPEECH EMOTION CLASSIFICATION [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4006

Gaussian Process LSTM Recurrent Neural Network Language Models for Speech Recognition


Recurrent neural network language models (RNNLMs) have shown superior performance across a range of speech recognition tasks. At the heart of all RNNLMs, the activation functions play a vital role to control the information flows and tracking longer history contexts that are useful for predicting the following words. Long short-term memory (LSTM) units are well known for such ability and thus widely used in current RNNLMs. However, the deterministic parameter estimates in LSTM RNNLMs are prone to over-fitting and poor generalization when given limited training data.

Paper Details

Authors:
Max W. Y. Lam, Xie Chen, Shoukang Hu, Jianwei Yu, Xunying Liu, Helen Meng
Submitted On:
7 May 2019 - 11:49pm
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

GPLSTM ICASSP Poster

(375)

Subscribe

[1] Max W. Y. Lam, Xie Chen, Shoukang Hu, Jianwei Yu, Xunying Liu, Helen Meng, "Gaussian Process LSTM Recurrent Neural Network Language Models for Speech Recognition", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4002. Accessed: Sep. 20, 2019.
@article{4002-19,
url = {http://sigport.org/4002},
author = {Max W. Y. Lam; Xie Chen; Shoukang Hu; Jianwei Yu; Xunying Liu; Helen Meng },
publisher = {IEEE SigPort},
title = {Gaussian Process LSTM Recurrent Neural Network Language Models for Speech Recognition},
year = {2019} }
TY - EJOUR
T1 - Gaussian Process LSTM Recurrent Neural Network Language Models for Speech Recognition
AU - Max W. Y. Lam; Xie Chen; Shoukang Hu; Jianwei Yu; Xunying Liu; Helen Meng
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4002
ER -
Max W. Y. Lam, Xie Chen, Shoukang Hu, Jianwei Yu, Xunying Liu, Helen Meng. (2019). Gaussian Process LSTM Recurrent Neural Network Language Models for Speech Recognition. IEEE SigPort. http://sigport.org/4002
Max W. Y. Lam, Xie Chen, Shoukang Hu, Jianwei Yu, Xunying Liu, Helen Meng, 2019. Gaussian Process LSTM Recurrent Neural Network Language Models for Speech Recognition. Available at: http://sigport.org/4002.
Max W. Y. Lam, Xie Chen, Shoukang Hu, Jianwei Yu, Xunying Liu, Helen Meng. (2019). "Gaussian Process LSTM Recurrent Neural Network Language Models for Speech Recognition." Web.
1. Max W. Y. Lam, Xie Chen, Shoukang Hu, Jianwei Yu, Xunying Liu, Helen Meng. Gaussian Process LSTM Recurrent Neural Network Language Models for Speech Recognition [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4002

NEURAL NETWORK LANGUAGE MODELING WITH LETTER-BASED FEATURES AND IMPORTANCE SAMPLING


In this paper we describe an extension of the Kaldi software toolkit to support neural-based language modeling, intended for use in automatic speech recognition (ASR) and related tasks. We combine the use of subword features (letter ngrams) and one-hot encoding of frequent words so that the models can handle large vocabularies containing infrequent words. We propose a new objective function that allows for training of unnormalized probabilities. An importance sampling based method is supported to speed up training when the vocabulary is large.

Paper Details

Authors:
Hainan Xu, Ke Li, Yiming Wang, Jian Wang, Shiyin Kang, Xie Chen, Daniel Povey, Sanjeev Khudanpur
Submitted On:
19 April 2018 - 11:52pm
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

kaldi-rnnlm-poster.pdf

(137)

Subscribe

[1] Hainan Xu, Ke Li, Yiming Wang, Jian Wang, Shiyin Kang, Xie Chen, Daniel Povey, Sanjeev Khudanpur, "NEURAL NETWORK LANGUAGE MODELING WITH LETTER-BASED FEATURES AND IMPORTANCE SAMPLING", IEEE SigPort, 2018. [Online]. Available: http://sigport.org/3064. Accessed: Sep. 20, 2019.
@article{3064-18,
url = {http://sigport.org/3064},
author = {Hainan Xu; Ke Li; Yiming Wang; Jian Wang; Shiyin Kang; Xie Chen; Daniel Povey; Sanjeev Khudanpur },
publisher = {IEEE SigPort},
title = {NEURAL NETWORK LANGUAGE MODELING WITH LETTER-BASED FEATURES AND IMPORTANCE SAMPLING},
year = {2018} }
TY - EJOUR
T1 - NEURAL NETWORK LANGUAGE MODELING WITH LETTER-BASED FEATURES AND IMPORTANCE SAMPLING
AU - Hainan Xu; Ke Li; Yiming Wang; Jian Wang; Shiyin Kang; Xie Chen; Daniel Povey; Sanjeev Khudanpur
PY - 2018
PB - IEEE SigPort
UR - http://sigport.org/3064
ER -
Hainan Xu, Ke Li, Yiming Wang, Jian Wang, Shiyin Kang, Xie Chen, Daniel Povey, Sanjeev Khudanpur. (2018). NEURAL NETWORK LANGUAGE MODELING WITH LETTER-BASED FEATURES AND IMPORTANCE SAMPLING. IEEE SigPort. http://sigport.org/3064
Hainan Xu, Ke Li, Yiming Wang, Jian Wang, Shiyin Kang, Xie Chen, Daniel Povey, Sanjeev Khudanpur, 2018. NEURAL NETWORK LANGUAGE MODELING WITH LETTER-BASED FEATURES AND IMPORTANCE SAMPLING. Available at: http://sigport.org/3064.
Hainan Xu, Ke Li, Yiming Wang, Jian Wang, Shiyin Kang, Xie Chen, Daniel Povey, Sanjeev Khudanpur. (2018). "NEURAL NETWORK LANGUAGE MODELING WITH LETTER-BASED FEATURES AND IMPORTANCE SAMPLING." Web.
1. Hainan Xu, Ke Li, Yiming Wang, Jian Wang, Shiyin Kang, Xie Chen, Daniel Povey, Sanjeev Khudanpur. NEURAL NETWORK LANGUAGE MODELING WITH LETTER-BASED FEATURES AND IMPORTANCE SAMPLING [Internet]. IEEE SigPort; 2018. Available from : http://sigport.org/3064

ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES


In this paper, we present a pruning technique for maximum en- tropy (MaxEnt) language models. It is based on computing the exact entropy loss when removing each feature from the model, and it ex- plicitly supports backoff features by replacing each removed feature with its backoff. The algorithm computes the loss on the training data, so it is not restricted to models with n-gram like features, al- lowing models with any feature, including long range skips, triggers, and contextual features such as device location.

Paper Details

Authors:
Tongzhou Chen, Diamantino Caseiro, Pat Rondon
Submitted On:
19 April 2018 - 2:12pm
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

poster.pdf

(142)

Subscribe

[1] Tongzhou Chen, Diamantino Caseiro, Pat Rondon, "ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES", IEEE SigPort, 2018. [Online]. Available: http://sigport.org/2762. Accessed: Sep. 20, 2019.
@article{2762-18,
url = {http://sigport.org/2762},
author = {Tongzhou Chen; Diamantino Caseiro; Pat Rondon },
publisher = {IEEE SigPort},
title = {ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES},
year = {2018} }
TY - EJOUR
T1 - ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES
AU - Tongzhou Chen; Diamantino Caseiro; Pat Rondon
PY - 2018
PB - IEEE SigPort
UR - http://sigport.org/2762
ER -
Tongzhou Chen, Diamantino Caseiro, Pat Rondon. (2018). ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES. IEEE SigPort. http://sigport.org/2762
Tongzhou Chen, Diamantino Caseiro, Pat Rondon, 2018. ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES. Available at: http://sigport.org/2762.
Tongzhou Chen, Diamantino Caseiro, Pat Rondon. (2018). "ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES." Web.
1. Tongzhou Chen, Diamantino Caseiro, Pat Rondon. ENTROPY BASED PRUNING OF BACKOFF MAXENT LANGUAGE MODELS WITH CONTEXTUAL FEATURES [Internet]. IEEE SigPort; 2018. Available from : http://sigport.org/2762

LIMITED-MEMORY BFGS OPTIMIZATION OF RECURRENT NEURAL NETWORK LANGUAGE MODELS FOR SPEECH RECOGNITION

Paper Details

Authors:
Xunying Liu, Shansong Liu, Jinze Sha, Jianwei Yu, Zhiyuan Xu, Xie Chen, Helen Meng
Submitted On:
13 April 2018 - 12:02am
Short Link:
Type:

Document Files

ICASSP2018_ShansongLIU_5th.pdf

(275)

Subscribe

[1] Xunying Liu, Shansong Liu, Jinze Sha, Jianwei Yu, Zhiyuan Xu, Xie Chen, Helen Meng, "LIMITED-MEMORY BFGS OPTIMIZATION OF RECURRENT NEURAL NETWORK LANGUAGE MODELS FOR SPEECH RECOGNITION", IEEE SigPort, 2018. [Online]. Available: http://sigport.org/2579. Accessed: Sep. 20, 2019.
@article{2579-18,
url = {http://sigport.org/2579},
author = {Xunying Liu; Shansong Liu; Jinze Sha; Jianwei Yu; Zhiyuan Xu; Xie Chen; Helen Meng },
publisher = {IEEE SigPort},
title = {LIMITED-MEMORY BFGS OPTIMIZATION OF RECURRENT NEURAL NETWORK LANGUAGE MODELS FOR SPEECH RECOGNITION},
year = {2018} }
TY - EJOUR
T1 - LIMITED-MEMORY BFGS OPTIMIZATION OF RECURRENT NEURAL NETWORK LANGUAGE MODELS FOR SPEECH RECOGNITION
AU - Xunying Liu; Shansong Liu; Jinze Sha; Jianwei Yu; Zhiyuan Xu; Xie Chen; Helen Meng
PY - 2018
PB - IEEE SigPort
UR - http://sigport.org/2579
ER -
Xunying Liu, Shansong Liu, Jinze Sha, Jianwei Yu, Zhiyuan Xu, Xie Chen, Helen Meng. (2018). LIMITED-MEMORY BFGS OPTIMIZATION OF RECURRENT NEURAL NETWORK LANGUAGE MODELS FOR SPEECH RECOGNITION. IEEE SigPort. http://sigport.org/2579
Xunying Liu, Shansong Liu, Jinze Sha, Jianwei Yu, Zhiyuan Xu, Xie Chen, Helen Meng, 2018. LIMITED-MEMORY BFGS OPTIMIZATION OF RECURRENT NEURAL NETWORK LANGUAGE MODELS FOR SPEECH RECOGNITION. Available at: http://sigport.org/2579.
Xunying Liu, Shansong Liu, Jinze Sha, Jianwei Yu, Zhiyuan Xu, Xie Chen, Helen Meng. (2018). "LIMITED-MEMORY BFGS OPTIMIZATION OF RECURRENT NEURAL NETWORK LANGUAGE MODELS FOR SPEECH RECOGNITION." Web.
1. Xunying Liu, Shansong Liu, Jinze Sha, Jianwei Yu, Zhiyuan Xu, Xie Chen, Helen Meng. LIMITED-MEMORY BFGS OPTIMIZATION OF RECURRENT NEURAL NETWORK LANGUAGE MODELS FOR SPEECH RECOGNITION [Internet]. IEEE SigPort; 2018. Available from : http://sigport.org/2579

Dialog Context Language Modeling with Recurrent Neural Networks


We propose contextual language models that incorporate dialog level discourse information into language modeling. Previous works on contextual language model treat preceding utterances as a sequence of inputs, without considering dialog interactions. We design recurrent neural network (RNN) based contextual language models that specially track the interactions between speakers in a dialog. Experiment results on Switchboard Dialog Act Corpus show that the proposed model outperforms conventional single turn based RNN language model by 3.3% on perplexity.

Paper Details

Authors:
Bing Liu, Ian Lane
Submitted On:
9 March 2017 - 4:59pm
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

Dialog Context Language Modeling with Recurrent Neural Networks

(271)

Subscribe

[1] Bing Liu, Ian Lane, "Dialog Context Language Modeling with Recurrent Neural Networks", IEEE SigPort, 2017. [Online]. Available: http://sigport.org/1730. Accessed: Sep. 20, 2019.
@article{1730-17,
url = {http://sigport.org/1730},
author = {Bing Liu; Ian Lane },
publisher = {IEEE SigPort},
title = {Dialog Context Language Modeling with Recurrent Neural Networks},
year = {2017} }
TY - EJOUR
T1 - Dialog Context Language Modeling with Recurrent Neural Networks
AU - Bing Liu; Ian Lane
PY - 2017
PB - IEEE SigPort
UR - http://sigport.org/1730
ER -
Bing Liu, Ian Lane. (2017). Dialog Context Language Modeling with Recurrent Neural Networks. IEEE SigPort. http://sigport.org/1730
Bing Liu, Ian Lane, 2017. Dialog Context Language Modeling with Recurrent Neural Networks. Available at: http://sigport.org/1730.
Bing Liu, Ian Lane. (2017). "Dialog Context Language Modeling with Recurrent Neural Networks." Web.
1. Bing Liu, Ian Lane. Dialog Context Language Modeling with Recurrent Neural Networks [Internet]. IEEE SigPort; 2017. Available from : http://sigport.org/1730

CHARACTER-LEVEL LANGUAGE MODELING WITH HIERARCHICAL RECURRENT NEURAL NETWORKS


Recurrent neural network (RNN) based character-level language models (CLMs) are extremely useful for modeling out-of-vocabulary words by nature. However, their performance is generally much worse than the word-level language models (WLMs), since CLMs need to consider longer history of tokens to properly predict the next one. We address this problem by proposing hierarchical RNN architectures, which consist of multiple modules with different timescales.

Paper Details

Authors:
Kyuyeon Hwang, Wonyong Sung
Submitted On:
6 March 2017 - 3:05am
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

poster.pdf

(775)

Subscribe

[1] Kyuyeon Hwang, Wonyong Sung, "CHARACTER-LEVEL LANGUAGE MODELING WITH HIERARCHICAL RECURRENT NEURAL NETWORKS", IEEE SigPort, 2017. [Online]. Available: http://sigport.org/1645. Accessed: Sep. 20, 2019.
@article{1645-17,
url = {http://sigport.org/1645},
author = {Kyuyeon Hwang; Wonyong Sung },
publisher = {IEEE SigPort},
title = {CHARACTER-LEVEL LANGUAGE MODELING WITH HIERARCHICAL RECURRENT NEURAL NETWORKS},
year = {2017} }
TY - EJOUR
T1 - CHARACTER-LEVEL LANGUAGE MODELING WITH HIERARCHICAL RECURRENT NEURAL NETWORKS
AU - Kyuyeon Hwang; Wonyong Sung
PY - 2017
PB - IEEE SigPort
UR - http://sigport.org/1645
ER -
Kyuyeon Hwang, Wonyong Sung. (2017). CHARACTER-LEVEL LANGUAGE MODELING WITH HIERARCHICAL RECURRENT NEURAL NETWORKS. IEEE SigPort. http://sigport.org/1645
Kyuyeon Hwang, Wonyong Sung, 2017. CHARACTER-LEVEL LANGUAGE MODELING WITH HIERARCHICAL RECURRENT NEURAL NETWORKS. Available at: http://sigport.org/1645.
Kyuyeon Hwang, Wonyong Sung. (2017). "CHARACTER-LEVEL LANGUAGE MODELING WITH HIERARCHICAL RECURRENT NEURAL NETWORKS." Web.
1. Kyuyeon Hwang, Wonyong Sung. CHARACTER-LEVEL LANGUAGE MODELING WITH HIERARCHICAL RECURRENT NEURAL NETWORKS [Internet]. IEEE SigPort; 2017. Available from : http://sigport.org/1645

Pages