Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

3D VISUAL SPEECH ANIMATION USING 2D VIDEOS

Abstract: 

In visual speech animation, lip motion accuracy is of paramount importance for speech intelligibility, especially for the hard of hearing or foreign language learners. We present an approach for visual speech animation that uses tracked lip motion in front-view 2D videos of a real speaker to drive the lip motion of a synthetic 3D head. This makes use of a 3D morphable model (3DMM), built using 3D synthetic head poses, with corresponding landmarks identified in the 2D videos and the 3DMM. We show that using a wider range of synthetic head poses for different phoneme intensities to create a 3DMM, as well as a combination of front and side photographs of the real speakers rather than just front photographs to produce initial neutral 3D synthetic head poses, gives better animation results when compared to ground truth data consisting of front-view 2D videos of real speakers.

up
0 users have voted:

Paper Details

Authors:
Rabab Algadhy , Yoshihiko Gotoh , Steve Maddock
Submitted On:
8 May 2019 - 8:26am
Short Link:
Type:
Poster
Event:
Presenter's Name:
Rabab Algadhy
Paper Code:
2318
Document Year:
2019
Cite

Document Files

3D Visual Speech Animation Using 2D Videos.pdf

(27)

Subscribe

[1] Rabab Algadhy , Yoshihiko Gotoh , Steve Maddock, "3D VISUAL SPEECH ANIMATION USING 2D VIDEOS", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4084. Accessed: Dec. 06, 2019.
@article{4084-19,
url = {http://sigport.org/4084},
author = {Rabab Algadhy ; Yoshihiko Gotoh ; Steve Maddock },
publisher = {IEEE SigPort},
title = {3D VISUAL SPEECH ANIMATION USING 2D VIDEOS},
year = {2019} }
TY - EJOUR
T1 - 3D VISUAL SPEECH ANIMATION USING 2D VIDEOS
AU - Rabab Algadhy ; Yoshihiko Gotoh ; Steve Maddock
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4084
ER -
Rabab Algadhy , Yoshihiko Gotoh , Steve Maddock. (2019). 3D VISUAL SPEECH ANIMATION USING 2D VIDEOS. IEEE SigPort. http://sigport.org/4084
Rabab Algadhy , Yoshihiko Gotoh , Steve Maddock, 2019. 3D VISUAL SPEECH ANIMATION USING 2D VIDEOS. Available at: http://sigport.org/4084.
Rabab Algadhy , Yoshihiko Gotoh , Steve Maddock. (2019). "3D VISUAL SPEECH ANIMATION USING 2D VIDEOS." Web.
1. Rabab Algadhy , Yoshihiko Gotoh , Steve Maddock. 3D VISUAL SPEECH ANIMATION USING 2D VIDEOS [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4084