Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Expression Guided EEG Representation Learning for Emotion Recognition

Abstract: 

Learning a joint and coordinated representation between different modalities can improve multimodal emotion recognition. In this paper, we propose a deep representation learning approach for emotion recognition from electroencephalogram (EEG) signals guided by facial electromyogram (EMG) and electrooculogram (EOG) signals. We recorded EEG, EMG and EOG signals from 60 participants who watched 40 short videos and self-reported their emotions. A cross-modal encoder that jointly learns the features extracted from facial and ocular expressions and EEG responses was designed and evaluated on our recorded data and MAHOB-HCI, a publicly available database. We demonstrate that the proposed representation is able to improve emotion recognition performance. We also show that the learned representation can be transferred to a different database without EMG and EOG and achieve superior performance. Methods that fuse behavioral and neural responses can be deployed in wearable emotion recognition solutions, practical in situations in which computer vision expression recognition is not feasible.

up
0 users have voted:

Paper Details

Authors:
Soheil Rayatdoost, David Rudrauf, Mohammad Soleymani
Submitted On:
16 May 2020 - 12:55am
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Mohammad Soleymani
Paper Code:
5658
Document Year:
2020
Cite

Document Files

rayatdoost_ICASSP_17_04.pdf

(58)

Subscribe

[1] Soheil Rayatdoost, David Rudrauf, Mohammad Soleymani, "Expression Guided EEG Representation Learning for Emotion Recognition", IEEE SigPort, 2020. [Online]. Available: http://sigport.org/5368. Accessed: Sep. 19, 2020.
@article{5368-20,
url = {http://sigport.org/5368},
author = {Soheil Rayatdoost; David Rudrauf; Mohammad Soleymani },
publisher = {IEEE SigPort},
title = {Expression Guided EEG Representation Learning for Emotion Recognition},
year = {2020} }
TY - EJOUR
T1 - Expression Guided EEG Representation Learning for Emotion Recognition
AU - Soheil Rayatdoost; David Rudrauf; Mohammad Soleymani
PY - 2020
PB - IEEE SigPort
UR - http://sigport.org/5368
ER -
Soheil Rayatdoost, David Rudrauf, Mohammad Soleymani. (2020). Expression Guided EEG Representation Learning for Emotion Recognition. IEEE SigPort. http://sigport.org/5368
Soheil Rayatdoost, David Rudrauf, Mohammad Soleymani, 2020. Expression Guided EEG Representation Learning for Emotion Recognition. Available at: http://sigport.org/5368.
Soheil Rayatdoost, David Rudrauf, Mohammad Soleymani. (2020). "Expression Guided EEG Representation Learning for Emotion Recognition." Web.
1. Soheil Rayatdoost, David Rudrauf, Mohammad Soleymani. Expression Guided EEG Representation Learning for Emotion Recognition [Internet]. IEEE SigPort; 2020. Available from : http://sigport.org/5368