Sorry, you need to enable JavaScript to visit this website.

SPEECH MODELING WITH A HIERARCHICAL TRANSFORMER DYNAMICAL VAE

Citation Author(s):
Xiaoyu Lin, Xiaoyu Bie, Simon Leglaive, Laurent Girin, Xavier Alameda-Pineda
Submitted by:
Xiaoyu Lin
Last updated:
31 May 2023 - 5:35pm
Document Type:
Poster
Document Year:
2023
Event:
Presenters:
Xiaoyu Lin
 

The dynamical variational autoencoders (DVAEs) are a family of latent-variable deep generative models that extends the VAE to model a sequence of observed data and a corresponding sequence of latent vectors. In almost all the DVAEs of the literature, the temporal dependencies within each sequence and across the two sequences are modeled with recurrent neural networks. In this paper, we propose to model speech signals with the Hierarchical Transformer DVAE (HiT-DVAE), which is a DVAE with two levels of latent variable (sequence-wise and frame-wise) and in which the temporal dependencies are implemented with the Transformer architecture. We show that HiT-DVAE outperforms several other DVAEs for speech spectrogram modeling, while enabling a simpler training procedure, revealing its high potential for downstream low-level speech processing tasks such as speech enhancement.

up
0 users have voted: