Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Speech Emotion Recognition Using Multi-hop Attention Mechanism

Abstract: 

In this paper, we are interested in exploiting textual and acoustic data of an utterance for the speech emotion classification task. The baseline approach models the information from audio and text independently using two deep neural networks (DNNs). The outputs from both the DNNs are then fused for classification. As opposed to using knowledge from both the modalities separately, we propose a framework to exploit acoustic information in tandem with lexical data. The proposed framework uses two bi-directional long short-term memory (BLSTM) for obtaining hidden representations of the utterance. Furthermore, we propose an attention mechanism, referred to as the multi-hop, which is trained to automatically infer the correlation between the modalities.
The multi-hop attention first computes the relevant segments of the textual data corresponding to the audio signal. The relevant textual data is then applied to attend parts of the audio signal. To evaluate the performance of the proposed system, experiments are performed in the IEMOCAP dataset. Experimental results show that the proposed technique outperforms the state-of-the-art system by 6.5% relative improvement in terms of weighted accuracy.

up
0 users have voted:

Paper Details

Authors:
Seunghyun Yoon,Seokhyun Byun,Subhadeep Dey,Kyomin Jung
Submitted On:
23 May 2019 - 5:27am
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Seunghyun Yoon
Paper Code:
MLSP-L3.2
Document Year:
2019
Cite

Document Files

presentation slide

(78)

Subscribe

[1] Seunghyun Yoon,Seokhyun Byun,Subhadeep Dey,Kyomin Jung, "Speech Emotion Recognition Using Multi-hop Attention Mechanism", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4559. Accessed: Oct. 15, 2019.
@article{4559-19,
url = {http://sigport.org/4559},
author = {Seunghyun Yoon;Seokhyun Byun;Subhadeep Dey;Kyomin Jung },
publisher = {IEEE SigPort},
title = {Speech Emotion Recognition Using Multi-hop Attention Mechanism},
year = {2019} }
TY - EJOUR
T1 - Speech Emotion Recognition Using Multi-hop Attention Mechanism
AU - Seunghyun Yoon;Seokhyun Byun;Subhadeep Dey;Kyomin Jung
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4559
ER -
Seunghyun Yoon,Seokhyun Byun,Subhadeep Dey,Kyomin Jung. (2019). Speech Emotion Recognition Using Multi-hop Attention Mechanism. IEEE SigPort. http://sigport.org/4559
Seunghyun Yoon,Seokhyun Byun,Subhadeep Dey,Kyomin Jung, 2019. Speech Emotion Recognition Using Multi-hop Attention Mechanism. Available at: http://sigport.org/4559.
Seunghyun Yoon,Seokhyun Byun,Subhadeep Dey,Kyomin Jung. (2019). "Speech Emotion Recognition Using Multi-hop Attention Mechanism." Web.
1. Seunghyun Yoon,Seokhyun Byun,Subhadeep Dey,Kyomin Jung. Speech Emotion Recognition Using Multi-hop Attention Mechanism [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4559