Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

A COMPARATIVE STUDY OF ESTIMATING ARTICULATORY MOVEMENTS FROM PHONEME SEQUENCES AND ACOUSTIC FEATURES

Abstract: 

Unlike phoneme sequences, movements of speech articulators (lips, tongue, jaw, velum) and the resultant acoustic signal are known to encode not only the linguistic message but also carry para-linguistic information. While several works exist for estimating articulatory movement from acoustic signals, little is known to what extent articulatory movements can be predicted only from linguistic information, i.e., phoneme sequence. In this work, we estimate articulatory movements from three different input representations: R1) acoustic signal, R2) phoneme sequence, R3) phoneme sequence with timing information. While an attention network is used for estimating articulatory movement in the case of R2, BLSTM network is used for R1 and R3. Experiments with ten subjects’ acoustic-articulatory data reveal that the estimation techniques achieve an average correlation coefficient of 0.85, 0.81, and 0.81 in the case of R1, R2, and R3 respectively. This indicates that attention network, although uses only phoneme sequence (R2) without any timing information, results in an estimation performance similar to that using rich acoustic signal (R1), suggesting that articulatory motion is primarily driven by the linguistic message. The correlation coefficient is further improved to 0.88 when R1 and R3 are used together for estimating articulatory movements.

up
0 users have voted:

Paper Details

Authors:
Abhayjeet Singh, Aravind Illa, Prasanta Kumar Ghosh
Submitted On:
26 May 2020 - 5:45am
Short Link:
Type:
Presentation Slides
Event:
Presenter's Name:
Abhayjeet Singh
Paper Code:
3859
Document Year:
2020
Cite

Document Files

Presentation slides

(13)

Subscribe

[1] Abhayjeet Singh, Aravind Illa, Prasanta Kumar Ghosh, "A COMPARATIVE STUDY OF ESTIMATING ARTICULATORY MOVEMENTS FROM PHONEME SEQUENCES AND ACOUSTIC FEATURES", IEEE SigPort, 2020. [Online]. Available: http://sigport.org/5391. Accessed: Jul. 08, 2020.
@article{5391-20,
url = {http://sigport.org/5391},
author = {Abhayjeet Singh; Aravind Illa; Prasanta Kumar Ghosh },
publisher = {IEEE SigPort},
title = {A COMPARATIVE STUDY OF ESTIMATING ARTICULATORY MOVEMENTS FROM PHONEME SEQUENCES AND ACOUSTIC FEATURES},
year = {2020} }
TY - EJOUR
T1 - A COMPARATIVE STUDY OF ESTIMATING ARTICULATORY MOVEMENTS FROM PHONEME SEQUENCES AND ACOUSTIC FEATURES
AU - Abhayjeet Singh; Aravind Illa; Prasanta Kumar Ghosh
PY - 2020
PB - IEEE SigPort
UR - http://sigport.org/5391
ER -
Abhayjeet Singh, Aravind Illa, Prasanta Kumar Ghosh. (2020). A COMPARATIVE STUDY OF ESTIMATING ARTICULATORY MOVEMENTS FROM PHONEME SEQUENCES AND ACOUSTIC FEATURES. IEEE SigPort. http://sigport.org/5391
Abhayjeet Singh, Aravind Illa, Prasanta Kumar Ghosh, 2020. A COMPARATIVE STUDY OF ESTIMATING ARTICULATORY MOVEMENTS FROM PHONEME SEQUENCES AND ACOUSTIC FEATURES. Available at: http://sigport.org/5391.
Abhayjeet Singh, Aravind Illa, Prasanta Kumar Ghosh. (2020). "A COMPARATIVE STUDY OF ESTIMATING ARTICULATORY MOVEMENTS FROM PHONEME SEQUENCES AND ACOUSTIC FEATURES." Web.
1. Abhayjeet Singh, Aravind Illa, Prasanta Kumar Ghosh. A COMPARATIVE STUDY OF ESTIMATING ARTICULATORY MOVEMENTS FROM PHONEME SEQUENCES AND ACOUSTIC FEATURES [Internet]. IEEE SigPort; 2020. Available from : http://sigport.org/5391