Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

DEEP MULTIMODAL LEARNING FOR EMOTION RECOGNITION IN SPOKEN LANGUAGE

Abstract: 

In this paper, we present a novel deep multimodal framework to predict human emotions based on sentence-level spoken language. Our architecture has two distinctive characteristics. First, it extracts the high-level features from both text and audio via a hybrid deep multimodal structure, which considers the spatial information from text, temporal information from audio, and high-level associations from low-level handcrafted features. Second, we fuse all features by using a three-layer deep neural network to learn the correlations across modalities and train the feature extraction and fusion modules together, allowing optimal global fine-tuning of the entire structure. We evaluated the proposed framework on the IEMOCAP dataset. Our result shows promising performance, achieving 60.4% in weighted accuracy for five emotion categories.

up
0 users have voted:

Paper Details

Authors:
Yue Gu, Shuhong Chen, Ivan Marsic
Submitted On:
13 April 2018 - 3:30pm
Short Link:
Type:
Poster
Event:
Presenter's Name:
Yue Gu
Paper Code:
3738
Document Year:
2018
Cite

Document Files

ICASSP_2018_POSTER.pdf

(110 downloads)

Subscribe

[1] Yue Gu, Shuhong Chen, Ivan Marsic, "DEEP MULTIMODAL LEARNING FOR EMOTION RECOGNITION IN SPOKEN LANGUAGE", IEEE SigPort, 2018. [Online]. Available: http://sigport.org/2752. Accessed: Aug. 21, 2018.
@article{2752-18,
url = {http://sigport.org/2752},
author = {Yue Gu; Shuhong Chen; Ivan Marsic },
publisher = {IEEE SigPort},
title = {DEEP MULTIMODAL LEARNING FOR EMOTION RECOGNITION IN SPOKEN LANGUAGE},
year = {2018} }
TY - EJOUR
T1 - DEEP MULTIMODAL LEARNING FOR EMOTION RECOGNITION IN SPOKEN LANGUAGE
AU - Yue Gu; Shuhong Chen; Ivan Marsic
PY - 2018
PB - IEEE SigPort
UR - http://sigport.org/2752
ER -
Yue Gu, Shuhong Chen, Ivan Marsic. (2018). DEEP MULTIMODAL LEARNING FOR EMOTION RECOGNITION IN SPOKEN LANGUAGE. IEEE SigPort. http://sigport.org/2752
Yue Gu, Shuhong Chen, Ivan Marsic, 2018. DEEP MULTIMODAL LEARNING FOR EMOTION RECOGNITION IN SPOKEN LANGUAGE. Available at: http://sigport.org/2752.
Yue Gu, Shuhong Chen, Ivan Marsic. (2018). "DEEP MULTIMODAL LEARNING FOR EMOTION RECOGNITION IN SPOKEN LANGUAGE." Web.
1. Yue Gu, Shuhong Chen, Ivan Marsic. DEEP MULTIMODAL LEARNING FOR EMOTION RECOGNITION IN SPOKEN LANGUAGE [Internet]. IEEE SigPort; 2018. Available from : http://sigport.org/2752