XGrammar

XGrammar uses bitmasking for efficient constrained decoding in AI, ensuring structurally sound outputs by setting logits of disallowed tokens to negative infinity. The method precomputes context-indep…

XGrammar, a grammar-constrained decoding library used by vLLM, SGLang, TensorRT-LLM, and MLC-LLM, masks invalid tokens before the model samples, guaranteeing valid JSON output and eliminating retry lo…