Sorry, you need to enable JavaScript to visit this website.

Punctuation Prediction for Streaming On-Device Speech Recognition

Citation Author(s):
Zhikai Zhou, Tian Tan, Yanmin Qian
Submitted by:
Zhikai Zhou
Last updated:
5 May 2022 - 4:52am
Document Type:
Research Manuscript
Document Year:
2022
Event:
Presenters:
Zhikai Zhou
Paper Code:
7552
 

Punctuation prediction is essential for automatic speech recognition (ASR). Although many works have been proposed for punctuation prediction, the on-device scenarios are rarely discussed with an end-to-end ASR. The punctuation prediction task is often treated as a post-processing of ASR outputs, but the mismatch between natural language in training input and ASR hypotheses in testing is ignored. Besides, language models built with deep neural networks are too large for edge devices. In this paper, we discuss one-pass models for both ASR and punctuation prediction to replace the conventional two-pass post-processing pipeline. Then the joint ASR-punctuation model is proposed to utilize multi-task learning to decouple the recognition and punctuation on the ASR decoder. Experimental results show that the proposed joint model not only outperforms the traditional post-processing method with limited extra parameters, but also achieves better accuracy in comparison to the direct ASR modeling on transcripts with punctuation.

up
0 users have voted: