Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

WHAT MAKES THE SOUND?: A DUAL-MODALITY INTERACTING NETWORK FOR AUDIO-VISUAL EVENT LOCALIZATION

Abstract: 

The presence of auditory and visual senses enables humans to obtain a profound understanding of the real-world scenes. While audio and visual signals are capable of providing scene knowledge individually, the combination of both offers a better insight about the underlying event. In this paper, we address the problem of audio-visual event localization where the goal is to identify the presence of an event that is both audible and visible in a video, using fully or weakly supervised learning. For this, we propose a novel Audio-Visual Interacting Network (AVIN) that enables inter as well as intra modality interactions by exploiting the local and global information of the two modalities. Our empirical evaluations confirm the superiority of our proposed model over the existing state-of-the art methods, in both fully as well as weakly supervised learning tasks, thus asserting the efficacy of our joint-modeling.

up
0 users have voted:

Paper Details

Authors:
Janani Ramaswamy
Submitted On:
16 May 2020 - 1:09am
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Janani Ramaswamy
Paper Code:
5164
Document Year:
2020
Cite

Document Files

What_makes_the_sound_ICASSP2020.pdf

(16)

Subscribe

[1] Janani Ramaswamy, "WHAT MAKES THE SOUND?: A DUAL-MODALITY INTERACTING NETWORK FOR AUDIO-VISUAL EVENT LOCALIZATION", IEEE SigPort, 2020. [Online]. Available: http://sigport.org/5369. Accessed: Jul. 13, 2020.
@article{5369-20,
url = {http://sigport.org/5369},
author = {Janani Ramaswamy },
publisher = {IEEE SigPort},
title = {WHAT MAKES THE SOUND?: A DUAL-MODALITY INTERACTING NETWORK FOR AUDIO-VISUAL EVENT LOCALIZATION},
year = {2020} }
TY - EJOUR
T1 - WHAT MAKES THE SOUND?: A DUAL-MODALITY INTERACTING NETWORK FOR AUDIO-VISUAL EVENT LOCALIZATION
AU - Janani Ramaswamy
PY - 2020
PB - IEEE SigPort
UR - http://sigport.org/5369
ER -
Janani Ramaswamy. (2020). WHAT MAKES THE SOUND?: A DUAL-MODALITY INTERACTING NETWORK FOR AUDIO-VISUAL EVENT LOCALIZATION. IEEE SigPort. http://sigport.org/5369
Janani Ramaswamy, 2020. WHAT MAKES THE SOUND?: A DUAL-MODALITY INTERACTING NETWORK FOR AUDIO-VISUAL EVENT LOCALIZATION. Available at: http://sigport.org/5369.
Janani Ramaswamy. (2020). "WHAT MAKES THE SOUND?: A DUAL-MODALITY INTERACTING NETWORK FOR AUDIO-VISUAL EVENT LOCALIZATION." Web.
1. Janani Ramaswamy. WHAT MAKES THE SOUND?: A DUAL-MODALITY INTERACTING NETWORK FOR AUDIO-VISUAL EVENT LOCALIZATION [Internet]. IEEE SigPort; 2020. Available from : http://sigport.org/5369