# Show HN: Production Grammar-Guided Decoding in vLLM with Xgrammar Benchmarks

> Source: <https://github.com/grfdsasdvef/vllm-constrained-decoding-cookbook>
> Published: 2026-11-18 08:16:29+00:00

**Recommendation:** Enforce JSON schemas with xgrammar guided decoding in vLLM instead of prompt instructions alone; the source measured 0.01% invalid schema errors versus 7.8% prompt-only, at under 2.1 ms p99 overhead.
More in the playbook "Verifying model output": <https://wpnews.pro/playbooks/output-verification>

⚡ **Maintained by Snowden SAY Lab:**
Deep production post-mortems, GPU kernel optimization, and LLM runtime architecture in our Telegram channel: **[@snowdensay](https://t.me/snowdensay)**.

📖 **Full Engineering Monograph & Complete Codebase:** available in Telegram bot [@Snowden_book_bot](https://t.me/Snowden_book_bot?start=production-ai-agent-infrastructure)

This cookbook provides drop-in, zero-hallucination structured decoding patterns for high-throughput LLM serving via **vLLM** and **xgrammar**.

- **99.97% Valid JSON Schema** compliance (compared to ~92% with prompt instructions alone).
- **Zero Token Waste:** Model physically cannot generate illegal keys or invalid types.
- **Logit Masking Pre-Softmax:** Logits of prohibited tokens are set to`-inf` before sampling.

```
git clone https://github.com/snowden-systems/vllm-constrained-decoding-cookbook.git
cd vllm-constrained-decoding-cookbook
pip install -r requirements.txt
python benchmarks/run_benchmark.py
python
import json
from pydantic import BaseModel, Field
from typing import Literal

class AgentToolCall(BaseModel):
    action: Literal["read_file", "write_file", "run_test"]
    path: str = Field(..., pattern=r"^[\w\-/\.]+$")
    timeout_sec: int = Field(default=30, ge=1, le=300)

schema = AgentToolCall.model_json_schema()

# Ready to pass to vLLM GuidedDecodingParams
print(json.dumps(schema, indent=2))
```

| Metric | Prompt-Only (GPT-4o / Llama-3.3) | Guided Decoding (xgrammar) | 
|---|---|---|
| **Invalid Schema Errors** | 7.8% | **0.01%** | 
| **Invented Fields (e.g. user_id)** | 4.2% | **0.00%** | 
| **Throughput Penalty** | 0 ms | **< 2.1 ms p99 overhead** | 
| **VRAM per 64 Active Schemas** | 0 MB | **180 MB** | 

- 📡 **Telegram Engineering Channel:**[@snowdensay](https://t.me/snowdensay)
- 📖 **Monograph Repository Bot:**[@Snowden_book_bot](https://t.me/Snowden_book_bot)
