{"slug": "the-parser-already-knows-lightweight-bias-correction-in-constrained-decoding", "title": "The Parser Already Knows: Lightweight Bias Correction in Constrained Decoding", "summary": "Researchers propose a lightweight, offline-trained logit correction for Grammar Constrained Decoding (GCD) that uses the parser and lexer states already computed during incremental parsing to restore the language model's true probability distribution, closing the gap between masked and true distributions while adding negligible overhead. The method outperforms both standard masking and online sampling across several grammars, with even its lightest variant matching or exceeding both baselines.", "body_md": "arXiv:2608.10137v1 Announce Type: new\nAbstract: Grammar Constrained Decoding (GCD) forces Language Models (LMs) to produce syntactically valid outputs by masking out non-conforming tokens at each step. However, rigid masking distorts the model's underlying probability distribution, often biasing generation toward valid but suboptimal outputs. While online sampling restores this distribution, it requires computationally expensive iterative resampling. As a result, existing methods force a compromise between output quality and inference latency. Our key insight is that the internal parser and lexer states inherently maintained during incremental parsing already encode future grammatical validity -- exactly the information required to restore the LM's true distribution. We propose a lightweight, offline-trained logit correction conditioned on this syntactic and lexical state together with candidate next tokens. Because these states are already computed as a necessary part of incremental parsing for masking, extracting them adds negligible overhead while leaving the base LM's weights completely untouched. Across several grammars, this correction substantially closes the gap between the masked distribution and the LM's true distribution, consistently outperforming both masking and online sampling. Even its lightest variant, which relies on the candidate next token alone, still matches or exceeds both baselines: the next token itself carries an implicit lookahead, much like how parsers commonly use a lookahead token to resolve ambiguous decisions. By restoring the probability mass that masking removes, it reconciles the LM's probabilistic integrity with grammar conformance.", "url": "https://wpnews.pro/news/the-parser-already-knows-lightweight-bias-correction-in-constrained-decoding", "canonical_source": "https://arxiv.org/abs/2608.10137", "published_at": "2026-08-12 04:00:00+00:00", "updated_at": "2026-08-12 04:09:47.690136+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "large-language-models", "natural-language-processing"], "entities": ["arXiv"], "alternates": {"html": "https://wpnews.pro/news/the-parser-already-knows-lightweight-bias-correction-in-constrained-decoding", "markdown": "https://wpnews.pro/news/the-parser-already-knows-lightweight-bias-correction-in-constrained-decoding.md", "text": "https://wpnews.pro/news/the-parser-already-knows-lightweight-bias-correction-in-constrained-decoding.txt", "jsonld": "https://wpnews.pro/news/the-parser-already-knows-lightweight-bias-correction-in-constrained-decoding.jsonld"}}