Fixing JSON Output from GPT: A Pattern That Actually Works

A developer spent three days debugging malformed JSON output from GPT-4 before discovering that constrained decoding, rather than prompt engineering or post-generation parsing, reliably produces valid structured data. Using the Outlines library, the engineer forced the model to generate only tokens that conform to a predefined JSON schema, eliminating edge cases where the model returned invalid dates, extra fields, or nested objects. The technique works by compiling the schema into a finite state machine that masks disallowed tokens during generation, producing valid JSON every time without requiring post-hoc validation or manual fixes.