Sorry, you need to enable JavaScript to visit this website.

Multimodal Speech Recognition

End-to-End Multimodal Speech Recognition

Transcription or sub-titling of open-domain videos is still a chal- lenging domain for Automatic Speech Recognition (ASR) due to the data’s challenging acoustics, variable signal processing and the essentially unrestricted domain of the data. In previous work, we have shown that the visual channel – specifically object and scene features – can help to adapt the acoustic model (AM) and language model (LM) of a recognizer, and we are now expanding this work to end-to-end approaches.

Paper Details

Ramon Sanabria, Florian Metze
Submitted On:
12 April 2018 - 8:02pm
Short Link:
Presenter's Name:
Paper Code:
Document Year:

Document Files




Additional Categories


[1] Ramon Sanabria, Florian Metze, "End-to-End Multimodal Speech Recognition", IEEE SigPort, 2018. [Online]. Available: Accessed: Jul. 20, 2019.
url = {},
author = {Ramon Sanabria; Florian Metze },
publisher = {IEEE SigPort},
title = {End-to-End Multimodal Speech Recognition},
year = {2018} }
T1 - End-to-End Multimodal Speech Recognition
AU - Ramon Sanabria; Florian Metze
PY - 2018
PB - IEEE SigPort
UR -
ER -
Ramon Sanabria, Florian Metze. (2018). End-to-End Multimodal Speech Recognition. IEEE SigPort.
Ramon Sanabria, Florian Metze, 2018. End-to-End Multimodal Speech Recognition. Available at:
Ramon Sanabria, Florian Metze. (2018). "End-to-End Multimodal Speech Recognition." Web.
1. Ramon Sanabria, Florian Metze. End-to-End Multimodal Speech Recognition [Internet]. IEEE SigPort; 2018. Available from :