Sorry, you need to enable JavaScript to visit this website.

Spoken and Multimodal Dialog Systems and Applications (SLP-SMMD)

End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager


Natural language understanding and dialogue policy learning are both essential in conversational systems that predict the

Paper Details

Authors:
Xuesong Yang, Yun-Nung Chen, Dilek Hakkani-Tur, Paul Crook, Xiujun Li, Jianfeng Gao, Li Deng
Submitted On:
10 March 2017 - 2:14pm
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

E2E_ICASSP17.pdf

(65 downloads)

Keywords

Subscribe

[1] Xuesong Yang, Yun-Nung Chen, Dilek Hakkani-Tur, Paul Crook, Xiujun Li, Jianfeng Gao, Li Deng, "End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager", IEEE SigPort, 2017. [Online]. Available: http://sigport.org/1736. Accessed: Oct. 22, 2017.
@article{1736-17,
url = {http://sigport.org/1736},
author = {Xuesong Yang; Yun-Nung Chen; Dilek Hakkani-Tur; Paul Crook; Xiujun Li; Jianfeng Gao; Li Deng },
publisher = {IEEE SigPort},
title = {End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager},
year = {2017} }
TY - EJOUR
T1 - End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager
AU - Xuesong Yang; Yun-Nung Chen; Dilek Hakkani-Tur; Paul Crook; Xiujun Li; Jianfeng Gao; Li Deng
PY - 2017
PB - IEEE SigPort
UR - http://sigport.org/1736
ER -
Xuesong Yang, Yun-Nung Chen, Dilek Hakkani-Tur, Paul Crook, Xiujun Li, Jianfeng Gao, Li Deng. (2017). End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager. IEEE SigPort. http://sigport.org/1736
Xuesong Yang, Yun-Nung Chen, Dilek Hakkani-Tur, Paul Crook, Xiujun Li, Jianfeng Gao, Li Deng, 2017. End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager. Available at: http://sigport.org/1736.
Xuesong Yang, Yun-Nung Chen, Dilek Hakkani-Tur, Paul Crook, Xiujun Li, Jianfeng Gao, Li Deng. (2017). "End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager." Web.
1. Xuesong Yang, Yun-Nung Chen, Dilek Hakkani-Tur, Paul Crook, Xiujun Li, Jianfeng Gao, Li Deng. End-to-End Joint Learning of Natural Language Understanding and Dialogue Manager [Internet]. IEEE SigPort; 2017. Available from : http://sigport.org/1736

Dialog State Tracking and Action Selection Using Deep Learning Mechanism for Interview Coaching


The best way to prepare for an interview is to review the different types of possible interview questions you will be asked during an interview and practice responding to questions. An interview coaching system tries to simulate an interviewer to provide mock interview practice simulation sessions for the users. The traditional interview coaching systems provide some feedbacks, including facial preference, head nodding, response time, speaking rate, and volume, to let users know their own performance in the mock interview.

Paper Details

Authors:
Ming-Hsiang Su, Kun-Yi Huang, Tsung-Hsien Yang, Kuan-Jung Lai and Chung-Hsien Wu
Submitted On:
22 November 2016 - 11:33pm
Short Link:
Type:
Event:
Presenter's Name:
Document Year:
Cite

Document Files

MingHsiangSu-IALP 2016.pdf

(116 downloads)

Keywords

Subscribe

[1] Ming-Hsiang Su, Kun-Yi Huang, Tsung-Hsien Yang, Kuan-Jung Lai and Chung-Hsien Wu, "Dialog State Tracking and Action Selection Using Deep Learning Mechanism for Interview Coaching", IEEE SigPort, 2016. [Online]. Available: http://sigport.org/1301. Accessed: Oct. 22, 2017.
@article{1301-16,
url = {http://sigport.org/1301},
author = {Ming-Hsiang Su; Kun-Yi Huang; Tsung-Hsien Yang; Kuan-Jung Lai and Chung-Hsien Wu },
publisher = {IEEE SigPort},
title = {Dialog State Tracking and Action Selection Using Deep Learning Mechanism for Interview Coaching},
year = {2016} }
TY - EJOUR
T1 - Dialog State Tracking and Action Selection Using Deep Learning Mechanism for Interview Coaching
AU - Ming-Hsiang Su; Kun-Yi Huang; Tsung-Hsien Yang; Kuan-Jung Lai and Chung-Hsien Wu
PY - 2016
PB - IEEE SigPort
UR - http://sigport.org/1301
ER -
Ming-Hsiang Su, Kun-Yi Huang, Tsung-Hsien Yang, Kuan-Jung Lai and Chung-Hsien Wu. (2016). Dialog State Tracking and Action Selection Using Deep Learning Mechanism for Interview Coaching. IEEE SigPort. http://sigport.org/1301
Ming-Hsiang Su, Kun-Yi Huang, Tsung-Hsien Yang, Kuan-Jung Lai and Chung-Hsien Wu, 2016. Dialog State Tracking and Action Selection Using Deep Learning Mechanism for Interview Coaching. Available at: http://sigport.org/1301.
Ming-Hsiang Su, Kun-Yi Huang, Tsung-Hsien Yang, Kuan-Jung Lai and Chung-Hsien Wu. (2016). "Dialog State Tracking and Action Selection Using Deep Learning Mechanism for Interview Coaching." Web.
1. Ming-Hsiang Su, Kun-Yi Huang, Tsung-Hsien Yang, Kuan-Jung Lai and Chung-Hsien Wu. Dialog State Tracking and Action Selection Using Deep Learning Mechanism for Interview Coaching [Internet]. IEEE SigPort; 2016. Available from : http://sigport.org/1301

Evaluation of a Multimodal 3-D Pronunciation Tutor for Learning Mandarin as a Second Language:An Eye-tracking Study

Paper Details

Authors:
Ying Zhou, Fei Chen, Hui Chen,Nan Yan
Submitted On:
16 October 2016 - 1:06am
Short Link:
Type:
Event:
Presenter's Name:
Document Year:
Cite

Document Files

Eyetracking PPT.ppt

(122 downloads)

Keywords

Subscribe

[1] Ying Zhou, Fei Chen, Hui Chen,Nan Yan, "Evaluation of a Multimodal 3-D Pronunciation Tutor for Learning Mandarin as a Second Language:An Eye-tracking Study", IEEE SigPort, 2016. [Online]. Available: http://sigport.org/1254. Accessed: Oct. 22, 2017.
@article{1254-16,
url = {http://sigport.org/1254},
author = {Ying Zhou; Fei Chen; Hui Chen;Nan Yan },
publisher = {IEEE SigPort},
title = {Evaluation of a Multimodal 3-D Pronunciation Tutor for Learning Mandarin as a Second Language:An Eye-tracking Study},
year = {2016} }
TY - EJOUR
T1 - Evaluation of a Multimodal 3-D Pronunciation Tutor for Learning Mandarin as a Second Language:An Eye-tracking Study
AU - Ying Zhou; Fei Chen; Hui Chen;Nan Yan
PY - 2016
PB - IEEE SigPort
UR - http://sigport.org/1254
ER -
Ying Zhou, Fei Chen, Hui Chen,Nan Yan. (2016). Evaluation of a Multimodal 3-D Pronunciation Tutor for Learning Mandarin as a Second Language:An Eye-tracking Study. IEEE SigPort. http://sigport.org/1254
Ying Zhou, Fei Chen, Hui Chen,Nan Yan, 2016. Evaluation of a Multimodal 3-D Pronunciation Tutor for Learning Mandarin as a Second Language:An Eye-tracking Study. Available at: http://sigport.org/1254.
Ying Zhou, Fei Chen, Hui Chen,Nan Yan. (2016). "Evaluation of a Multimodal 3-D Pronunciation Tutor for Learning Mandarin as a Second Language:An Eye-tracking Study." Web.
1. Ying Zhou, Fei Chen, Hui Chen,Nan Yan. Evaluation of a Multimodal 3-D Pronunciation Tutor for Learning Mandarin as a Second Language:An Eye-tracking Study [Internet]. IEEE SigPort; 2016. Available from : http://sigport.org/1254

Realizing Speech to Gesture Conversion by Keyword Spotting


The paper proposed a method to realize a speech-to-gesture conversion for communication between normal and speech-impaired people. Keyword spotting was employed to recognize the keywords from input speech signals. At the same time, the three dimensional gesture models of keywords were built by 3D modeling technology according to the "Chinese sign language". The speech-to-gesture conversion was finally realized by playing the corresponding 3D gestures with OpenGL from the results of keyword spotting.

Paper Details

Authors:
Na Zhao, Hongwu Yang
Submitted On:
14 October 2016 - 9:54pm
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

keyword spotting, gesture modeling, speech to gesture conversion

(98 downloads)

Keywords

Subscribe

[1] Na Zhao, Hongwu Yang, "Realizing Speech to Gesture Conversion by Keyword Spotting", IEEE SigPort, 2016. [Online]. Available: http://sigport.org/1212. Accessed: Oct. 22, 2017.
@article{1212-16,
url = {http://sigport.org/1212},
author = {Na Zhao; Hongwu Yang },
publisher = {IEEE SigPort},
title = {Realizing Speech to Gesture Conversion by Keyword Spotting},
year = {2016} }
TY - EJOUR
T1 - Realizing Speech to Gesture Conversion by Keyword Spotting
AU - Na Zhao; Hongwu Yang
PY - 2016
PB - IEEE SigPort
UR - http://sigport.org/1212
ER -
Na Zhao, Hongwu Yang. (2016). Realizing Speech to Gesture Conversion by Keyword Spotting. IEEE SigPort. http://sigport.org/1212
Na Zhao, Hongwu Yang, 2016. Realizing Speech to Gesture Conversion by Keyword Spotting. Available at: http://sigport.org/1212.
Na Zhao, Hongwu Yang. (2016). "Realizing Speech to Gesture Conversion by Keyword Spotting." Web.
1. Na Zhao, Hongwu Yang. Realizing Speech to Gesture Conversion by Keyword Spotting [Internet]. IEEE SigPort; 2016. Available from : http://sigport.org/1212

Poster for Unsupervised User Intent Modeling by Feature-Enriched Matrix Factorization


Feature-Enrich Matrix Factorization for SLU at ICASSP16

Spoken language interfaces are being incorporated into various devices such as smart phones and TVs. However, dialogue systems may fail to respond correctly when users’ request functionality is not supported by currently installed apps. This paper proposes a feature-enriched matrix factorization (MF) approach to model open domain intents, which allows a system to dynamically add unexplored domains according to users’ requests.

Paper Details

Authors:
Ming Sun, Alexander I. Rudnicky, Anatole Gershman
Submitted On:
31 March 2016 - 7:51pm
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

FeatureMF_poster.pdf

(228 downloads)

Keywords

Subscribe

[1] Ming Sun, Alexander I. Rudnicky, Anatole Gershman, "Poster for Unsupervised User Intent Modeling by Feature-Enriched Matrix Factorization", IEEE SigPort, 2016. [Online]. Available: http://sigport.org/1079. Accessed: Oct. 22, 2017.
@article{1079-16,
url = {http://sigport.org/1079},
author = {Ming Sun; Alexander I. Rudnicky; Anatole Gershman },
publisher = {IEEE SigPort},
title = {Poster for Unsupervised User Intent Modeling by Feature-Enriched Matrix Factorization},
year = {2016} }
TY - EJOUR
T1 - Poster for Unsupervised User Intent Modeling by Feature-Enriched Matrix Factorization
AU - Ming Sun; Alexander I. Rudnicky; Anatole Gershman
PY - 2016
PB - IEEE SigPort
UR - http://sigport.org/1079
ER -
Ming Sun, Alexander I. Rudnicky, Anatole Gershman. (2016). Poster for Unsupervised User Intent Modeling by Feature-Enriched Matrix Factorization. IEEE SigPort. http://sigport.org/1079
Ming Sun, Alexander I. Rudnicky, Anatole Gershman, 2016. Poster for Unsupervised User Intent Modeling by Feature-Enriched Matrix Factorization. Available at: http://sigport.org/1079.
Ming Sun, Alexander I. Rudnicky, Anatole Gershman. (2016). "Poster for Unsupervised User Intent Modeling by Feature-Enriched Matrix Factorization." Web.
1. Ming Sun, Alexander I. Rudnicky, Anatole Gershman. Poster for Unsupervised User Intent Modeling by Feature-Enriched Matrix Factorization [Internet]. IEEE SigPort; 2016. Available from : http://sigport.org/1079

Poster for Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models


CDSSM for Zero-Shot Intent Modeling at ICASSP16

The recent surge of intelligent personal assistants motivates spoken language understanding of dialogue systems. However, the domain constraint along with the inflexible intent schema remains a big issue. This paper focuses on the task of intent expansion, which helps remove the domain limit and make an intent schema flexible. A convolutional deep structured semantic model (CDSSM) is applied to jointly learn the representations for human intents and associated utterances.

Paper Details

Authors:
Dilek Hakkani-Tur, Xiaodong He
Submitted On:
31 March 2016 - 7:30pm
Short Link:
Type:
Event:
Presenter's Name:
Paper Code:
Document Year:
Cite

Document Files

ZeroShot_poster.pdf

(225 downloads)

Keywords

Subscribe

[1] Dilek Hakkani-Tur, Xiaodong He, "Poster for Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models", IEEE SigPort, 2016. [Online]. Available: http://sigport.org/1078. Accessed: Oct. 22, 2017.
@article{1078-16,
url = {http://sigport.org/1078},
author = {Dilek Hakkani-Tur; Xiaodong He },
publisher = {IEEE SigPort},
title = {Poster for Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models},
year = {2016} }
TY - EJOUR
T1 - Poster for Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models
AU - Dilek Hakkani-Tur; Xiaodong He
PY - 2016
PB - IEEE SigPort
UR - http://sigport.org/1078
ER -
Dilek Hakkani-Tur, Xiaodong He. (2016). Poster for Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models. IEEE SigPort. http://sigport.org/1078
Dilek Hakkani-Tur, Xiaodong He, 2016. Poster for Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models. Available at: http://sigport.org/1078.
Dilek Hakkani-Tur, Xiaodong He. (2016). "Poster for Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models." Web.
1. Dilek Hakkani-Tur, Xiaodong He. Poster for Zero-Shot Learning of Intent Embeddings for Expansion by Convolutional Deep Structured Semantic Models [Internet]. IEEE SigPort; 2016. Available from : http://sigport.org/1078