cd /news/large-language-models/show-hn-production-grammar-guided-de… · home › topics › large-language-models › article
[ARTICLE · art-171660] src=github.com ↗ pub= topic=large-language-models verified=true sentiment=↑ positive

Show HN: Production Grammar-Guided Decoding in vLLM with Xgrammar Benchmarks

A Show HN cookbook published by Snowden SAY Lab reports that grammar-guided decoding with xgrammar in vLLM achieves 99.97% valid JSON schema compliance, versus roughly 92% with prompt instructions alone, and cuts invalid schema errors from 7.8% to 0.01% and invented fields from 4.2% to 0.00%. The cookbook masks prohibited-token logits to -inf before sampling and reports under 2.1 ms p99 throughput overhead and 180 MB of VRAM per 64 active schemas. The code and benchmark script are available in the snowden-systems/vllm-constrained-decoding-cookbook repository.

recommendation

Enforce JSON schemas with xgrammar guided decoding in vLLM instead of prompt instructions alone; the source measured 0.01% invalid schema errors versus 7.8% prompt-only, at under 2.1 ms p99 overhead.

more in playbook: Verifying model output
read1 min views5 publishedNov 18, 2026
Show HN: Production Grammar-Guided Decoding in vLLM with Xgrammar Benchmarks
Image: Michielbdejong (auto-discovered)

⚡ Maintained by Snowden SAY Lab: Deep production post-mortems, GPU kernel optimization, and LLM runtime architecture in our Telegram channel: @snowdensay.

📖 Full Engineering Monograph & Complete Codebase: available in Telegram bot @Snowden_book_bot

This cookbook provides drop-in, zero-hallucination structured decoding patterns for high-throughput LLM serving via vLLM and xgrammar.

  • 99.97% Valid JSON Schema compliance (compared to ~92% with prompt instructions alone).
  • Zero Token Waste: Model physically cannot generate illegal keys or invalid types.
  • Logit Masking Pre-Softmax: Logits of prohibited tokens are set to-inf before sampling.
git clone https://github.com/snowden-systems/vllm-constrained-decoding-cookbook.git
cd vllm-constrained-decoding-cookbook
pip install -r requirements.txt
python benchmarks/run_benchmark.py
python
import json
from pydantic import BaseModel, Field
from typing import Literal

class AgentToolCall(BaseModel):
    action: Literal["read_file", "write_file", "run_test"]
    path: str = Field(..., pattern=r"^[\w\-/\.]+$")
    timeout_sec: int = Field(default=30, ge=1, le=300)

schema = AgentToolCall.model_json_schema()

print(json.dumps(schema, indent=2))
Metric Prompt-Only (GPT-4o / Llama-3.3) Guided Decoding (xgrammar)
Invalid Schema Errors 7.8% 0.01%
Invented Fields (e.g. user_id) 4.2% 0.00%
Throughput Penalty 0 ms < 2.1 ms p99 overhead
VRAM per 64 Active Schemas 0 MB 180 MB
── more in #large-language-models 4 stories · sorted by recency
── more on @snowden say lab 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/show-hn-production-g…] indexed:0 read:1min 2026-11-18 · —