Efficient Punctuation Restoration via Weighted Lookahead Scoring Method for Streaming ASR Systems

Researchers have developed a non-autoregressive punctuation restoration method for streaming automatic speech recognition (ASR) systems that uses a weighted lookahead scoring approach with bounded future context. The method, which compares punctuation insertion hypotheses against a no-insertion baseline under a K-subword-token lookahead, achieved a macro F1 score of 0.893 without fine-tuning and 0.937 after fine-tuning on the IWSLT 2017 dataset, outperforming existing baselines. This approach addresses latency and alignment failures in streaming ASR by making incremental punctuation decisions at each word boundary without requiring free-form generation.

arXiv:2606.05179v1 Announce Type: new Abstract: Punctuation restoration improves ASR Automatic Speech Recognition readability. However streaming ASR requires online decisions with limited future context. In streaming ASR, the system predicts punctuation incrementally, which makes generation-based approaches prone to latency and alignment failures under boundary-wise evaluation. This paper proposes a non-autoregressive scoring method no free-form generation that preserves the input transcript and makes a decision at each word boundary. Our method compares punctuation insertion hypotheses against a no-insertion baseline under a bounded K-subword-token lookahead, and calibrates decisions using a weight {\alpha} and a validation-calibrated threshold {\tau} no parameter updates during inference . On IWSLT 2017, our scoring method achieves a 4-class macro F1 of 0.893 in the no fine-tuning setting validation-calibrated, K=2 and 0.937 after fine-tuning K=2 , outperforming the prompt-based baseline 0.566 and a fine-tuned ELECTRA baseline 0.913 under the same lookahead budget. We analyze the impact of the lookahead budget through ablation studies on K.