Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Learning FOFE based FNN-LMs with noise contrastive estimation and part-of-speech features

Abstract: 

A simple but powerful language model called fixed-size
ordinally-forgetting encoding (FOFE) based feedforward neural
network language models (FNN-LMs) has been proposed recently.
Experimental results have shown that FOFE based FNNLMs
can outperform not only the standard FNN-LMs but also
the popular recurrent neural network language models (RNNLMs).
In this paper, we extend FOFE based FNN-LMs from
several aspects. Firstly, we have proposed a new method to
further improve the performance of FOFE based FNN-LMs by
adding transitions of part-of-speech (POS) tags as additional
features. Secondly, we have investigated how to speedup the
learning of FOFE based FNN-LMs by using noise contrastive
estimation (NCE). As a result, we can dramatically speedup
the learning of FOFE based FNN-LMs while we still achieve
very competitive experimental results on Large Text Compression
Benchmark (LTCB).

up
0 users have voted:

Paper Details

Authors:
Junfeng Hou,Shiliang Zhang,Lirong Dai
Submitted On:
14 October 2016 - 4:49am
Short Link:
Type:
Poster
Event:
Presenter's Name:
Junfeng Hou
Paper Code:
P2-8
Document Year:
2016
Cite

Document Files

iscslp2016_poster_jfhou_.pdf

(357)

Subscribe

[1] Junfeng Hou,Shiliang Zhang,Lirong Dai, "Learning FOFE based FNN-LMs with noise contrastive estimation and part-of-speech features", IEEE SigPort, 2016. [Online]. Available: http://sigport.org/1185. Accessed: Feb. 18, 2020.
@article{1185-16,
url = {http://sigport.org/1185},
author = {Junfeng Hou;Shiliang Zhang;Lirong Dai },
publisher = {IEEE SigPort},
title = {Learning FOFE based FNN-LMs with noise contrastive estimation and part-of-speech features},
year = {2016} }
TY - EJOUR
T1 - Learning FOFE based FNN-LMs with noise contrastive estimation and part-of-speech features
AU - Junfeng Hou;Shiliang Zhang;Lirong Dai
PY - 2016
PB - IEEE SigPort
UR - http://sigport.org/1185
ER -
Junfeng Hou,Shiliang Zhang,Lirong Dai. (2016). Learning FOFE based FNN-LMs with noise contrastive estimation and part-of-speech features. IEEE SigPort. http://sigport.org/1185
Junfeng Hou,Shiliang Zhang,Lirong Dai, 2016. Learning FOFE based FNN-LMs with noise contrastive estimation and part-of-speech features. Available at: http://sigport.org/1185.
Junfeng Hou,Shiliang Zhang,Lirong Dai. (2016). "Learning FOFE based FNN-LMs with noise contrastive estimation and part-of-speech features." Web.
1. Junfeng Hou,Shiliang Zhang,Lirong Dai. Learning FOFE based FNN-LMs with noise contrastive estimation and part-of-speech features [Internet]. IEEE SigPort; 2016. Available from : http://sigport.org/1185