Sorry, you need to enable JavaScript to visit this website.

facebooktwittermailshare

END-TO-END FEEDBACK LOSS IN SPEECH CHAIN FRAMEWORK VIA STRAIGHT-THROUGH ESTIMATOR

Abstract: 

The speech chain mechanism integrates automatic speech recognition (ASR) and text-to-speech synthesis (TTS) modules into a single cycle during training. In our previous work, we applied a speech chain mechanism as a semi-supervised learning. It provides the ability for ASR and TTS to assist each other when they receive unpaired data and let them infer the missing pair and optimize the model with reconstruction loss. If we only have speech without transcription, ASR generates the most likely transcription from the speech data, and then TTS uses the generated transcription to reconstruct the original speech features. However, in previous papers, we just limited our back-propagation to the closest module, which is the TTS part. One reason is that back-propagating the error through the ASR is challenging due to the output of the ASR being discrete tokens, creating non-differentiability between the TTS and ASR. In this paper, we address this problem and describe how to thoroughly train a speech chain end-to-end for reconstruction loss using a straight-through estimator (ST). Experimental results revealed that, with sampling from ST-Gumbel-Softmax, we were able to update ASR parameters and improve the ASR performances by 11\% relative CER reduction compared to the baseline.

up
0 users have voted:

Paper Details

Authors:
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura
Submitted On:
14 May 2019 - 8:26pm
Short Link:
Type:
Poster
Event:
Presenter's Name:
Andros Tjandra
Paper Code:
SLP-P7.8
Document Year:
2019
Cite

Document Files

ICASSP19_Poster_V1.pdf

(45)

Subscribe

[1] Andros Tjandra, Sakriani Sakti, Satoshi Nakamura, "END-TO-END FEEDBACK LOSS IN SPEECH CHAIN FRAMEWORK VIA STRAIGHT-THROUGH ESTIMATOR", IEEE SigPort, 2019. [Online]. Available: http://sigport.org/4519. Accessed: Nov. 13, 2019.
@article{4519-19,
url = {http://sigport.org/4519},
author = {Andros Tjandra; Sakriani Sakti; Satoshi Nakamura },
publisher = {IEEE SigPort},
title = {END-TO-END FEEDBACK LOSS IN SPEECH CHAIN FRAMEWORK VIA STRAIGHT-THROUGH ESTIMATOR},
year = {2019} }
TY - EJOUR
T1 - END-TO-END FEEDBACK LOSS IN SPEECH CHAIN FRAMEWORK VIA STRAIGHT-THROUGH ESTIMATOR
AU - Andros Tjandra; Sakriani Sakti; Satoshi Nakamura
PY - 2019
PB - IEEE SigPort
UR - http://sigport.org/4519
ER -
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura. (2019). END-TO-END FEEDBACK LOSS IN SPEECH CHAIN FRAMEWORK VIA STRAIGHT-THROUGH ESTIMATOR. IEEE SigPort. http://sigport.org/4519
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura, 2019. END-TO-END FEEDBACK LOSS IN SPEECH CHAIN FRAMEWORK VIA STRAIGHT-THROUGH ESTIMATOR. Available at: http://sigport.org/4519.
Andros Tjandra, Sakriani Sakti, Satoshi Nakamura. (2019). "END-TO-END FEEDBACK LOSS IN SPEECH CHAIN FRAMEWORK VIA STRAIGHT-THROUGH ESTIMATOR." Web.
1. Andros Tjandra, Sakriani Sakti, Satoshi Nakamura. END-TO-END FEEDBACK LOSS IN SPEECH CHAIN FRAMEWORK VIA STRAIGHT-THROUGH ESTIMATOR [Internet]. IEEE SigPort; 2019. Available from : http://sigport.org/4519