Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

EFFECTS OF F0 ESTIMATION ALGORITHMS ON ULTRASOUND-BASED SILENT SPEECH INTERFACES

Abstract: 

This paper shows recent progresses on our Silent Speech Interface (SSI) that translates tongue motions into audible speech. In our previous work and also in the current study, the prediction of fundamental frequency (F0) from Ultra-sound Tongue Images (UTI) was achieved using articulatory-to-acoustic mapping methods based on deep learning. Here we investigated several traditional discontinuous speech-based F0 estimation algorithms for the target of UTI-based SSI system. Besides, the vocoder parameters (F0, Maximum Voiced Frequency and Mel-Generalized Cepstrum) are predicted using deep neural networks, with UTI as input. We found that during the articulatory-to-acoustic mapping experiments, those discontinuous F0 algorithms are predicted with lower error, and they result in slightly more natural synthesized speech than the Idiap baseline continuous F0 algorithm. Experimental results confirmed that discontinuous algorithms (e.g. Yin) are closest to original speech in objective metrics and subjective listening test.

a.pdf

PDF icon a.pdf (15)
up
0 users have voted:

Comments

updated abstract

checked

Paper Details

Authors:
Submitted On:
21 October 2020 - 11:11am
Short Link:
Type:
Poster
Event:
Presenter's Name:
Pengyu Dai
Document Year:
2020
Cite

Document Files

a.pdf

(15)

Subscribe

[1] , "EFFECTS OF F0 ESTIMATION ALGORITHMS ON ULTRASOUND-BASED SILENT SPEECH INTERFACES", IEEE SigPort, 2020. [Online]. Available: http://sigport.org/5464. Accessed: Nov. 29, 2020.
@article{5464-20,
url = {http://sigport.org/5464},
author = { },
publisher = {IEEE SigPort},
title = {EFFECTS OF F0 ESTIMATION ALGORITHMS ON ULTRASOUND-BASED SILENT SPEECH INTERFACES},
year = {2020} }
TY - EJOUR
T1 - EFFECTS OF F0 ESTIMATION ALGORITHMS ON ULTRASOUND-BASED SILENT SPEECH INTERFACES
AU -
PY - 2020
PB - IEEE SigPort
UR - http://sigport.org/5464
ER -
. (2020). EFFECTS OF F0 ESTIMATION ALGORITHMS ON ULTRASOUND-BASED SILENT SPEECH INTERFACES. IEEE SigPort. http://sigport.org/5464
, 2020. EFFECTS OF F0 ESTIMATION ALGORITHMS ON ULTRASOUND-BASED SILENT SPEECH INTERFACES. Available at: http://sigport.org/5464.
. (2020). "EFFECTS OF F0 ESTIMATION ALGORITHMS ON ULTRASOUND-BASED SILENT SPEECH INTERFACES." Web.
1. . EFFECTS OF F0 ESTIMATION ALGORITHMS ON ULTRASOUND-BASED SILENT SPEECH INTERFACES [Internet]. IEEE SigPort; 2020. Available from : http://sigport.org/5464