MLC AI released XGrammar-2, a major upgrade to its constrained-decoding library that introduces Structural Tag, a composable JSON DSL for expressing OpenAI Harmony format, tool calling, reasoning chan…
Guided decoding, a technique that constrains large language model token sampling to a formal grammar such as a JSON schema, has moved from hand-built logit manipulation to a near-ubiquitous API featur…
XGrammar uses bitmasking for efficient constrained decoding in AI, ensuring structurally sound outputs by setting logits of disallowed tokens to negative infinity. The method precomputes context-indep…
XGrammar, a grammar-constrained decoding library used by vLLM, SGLang, TensorRT-LLM, and MLC-LLM, masks invalid tokens before the model samples, guaranteeing valid JSON output and eliminating retry lo…