Sorry, you need to enable JavaScript to visit this website.

Multi-view variational recurrent neural network for human emotion recognition using multi-modal biological signals

Citation Author(s):
Yuya Moroto, Keisuke Maeda, Takahiro Ogawa, Miki Haseyama
Submitted by:
Yuya Moroto
Last updated:
17 November 2023 - 12:05pm
Document Type:
Presentation Slides
Yuya Moroto
Paper Code:

In this paper, the Multi-view Variational Recurrent Neural Network (MvVRNN) is proposed for multi-modal human emotion recognition with gaze and brain activity data while humans view images. For realizing accurate emotion recognition, we focus on the following three characteristics of biological signals: 1) the relationship between implicit and explicit information such as gaze and brain activity data, 2) the temporal changes related to human emotions and 3) the effects of noises that can be included during data acquisition. For treating these characteristics, the proposed MvVRNN has several mechanisms including 1) the integration of multi-modal information including implicit and explicit states of humans, 2) the recurrent module for sequential data and 3) the variational approximation based on the Gaussian distribution. The experimental results show that emotion recognition based on the MvVRNN outperforms several existing methods.

0 users have voted: