Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

Motion Dynamics Improve Speaker-Independent Lipreading

Abstract: 

We present a novel lipreading system that improves on the task of speaker-independent word recognition by decoupling motion and content dynamics. We achieve this by implementing a deep learning architecture that uses two distinct pipelines to process motion and content and subsequently merges them, implementing an end-to-end trainable system that performs fusion of independently learned representations. We obtain a average relative word accuracy improvement of ≈6.8% on unseen speakers and of ≈3.3% on known speakers, with respect to a baseline which uses a standard architecture.

up
0 users have voted:

Paper Details

Authors:
Matteo Riva, Michael Wand, Jürgen Schmidhuber
Submitted On:
19 April 2020 - 6:19pm
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Matteo Riva
Paper Code:
4996
Document Year:
2020
Cite

Document Files

Presentation PDF slides

(55)

Subscribe

[1] Matteo Riva, Michael Wand, Jürgen Schmidhuber, "Motion Dynamics Improve Speaker-Independent Lipreading", IEEE SigPort, 2020. [Online]. Available: http://sigport.org/5108. Accessed: Jul. 14, 2020.
@article{5108-20,
url = {http://sigport.org/5108},
author = {Matteo Riva; Michael Wand; Jürgen Schmidhuber },
publisher = {IEEE SigPort},
title = {Motion Dynamics Improve Speaker-Independent Lipreading},
year = {2020} }
TY - EJOUR
T1 - Motion Dynamics Improve Speaker-Independent Lipreading
AU - Matteo Riva; Michael Wand; Jürgen Schmidhuber
PY - 2020
PB - IEEE SigPort
UR - http://sigport.org/5108
ER -
Matteo Riva, Michael Wand, Jürgen Schmidhuber. (2020). Motion Dynamics Improve Speaker-Independent Lipreading. IEEE SigPort. http://sigport.org/5108
Matteo Riva, Michael Wand, Jürgen Schmidhuber, 2020. Motion Dynamics Improve Speaker-Independent Lipreading. Available at: http://sigport.org/5108.
Matteo Riva, Michael Wand, Jürgen Schmidhuber. (2020). "Motion Dynamics Improve Speaker-Independent Lipreading." Web.
1. Matteo Riva, Michael Wand, Jürgen Schmidhuber. Motion Dynamics Improve Speaker-Independent Lipreading [Internet]. IEEE SigPort; 2020. Available from : http://sigport.org/5108