Your LLM returns JSON that crashes json.loads about 1 time in 20. Here is how to handle it

A developer published a step-by-step Python approach for handling malformed JSON returned by LLMs, which fails roughly one time in 20 in production. The technique strips markdown code fences and surrounding prose, extracts the outermost object or array, repairs trailing commas, validates required keys and types, and feeds the exact error back to the model for a single retry. The author recommends structured outputs where providers support them, framing defensive parsing as the fallback.

If you call an LLM and ask for JSON, it works in the demo and then fails in production at the worst moment. The usual culprits are a markdown code fence around the JSON, a sentence before it "Sure Here is the JSON:" , or a trailing comma. Here is a small, step-by-step way to deal with all three in plain Python. python import json import re def extract json text: str : text = text.strip 1. remove {% endraw %} json ... {% raw %} fences fence = re.search r" ?:json ?\s . ? ", text, re.DOTALL if fence: text = fence.group 1 .strip 2. keep only the outermost {...} or ... starts = i for i in text.find "{" , text.find " " if i = -1 if not starts: raise ValueError "no JSON found" start = min starts end = max text.rfind "}" , text.rfind " " return text start : end + 1 python def parse llm json text: str : raw = extract json text try: return json.loads raw except json.JSONDecodeError: remove trailing commas before } or fixed = re.sub r",\s }\ ", r"\1", raw return json.loads fixed let it raise if still broken Valid JSON can still have the wrong keys. Check the fields you rely on: python def require data: dict, schema: dict : for key, typ in schema.items : if key not in data: raise KeyError f"missing key: {key}" if not isinstance data key , typ : raise TypeError f"{key} should be {typ. name }" return data result = require parse llm json reply , {"title": str, "score": int} Feed the exact error back to the model. One retry fixes most failures, and more than two retries usually just burns tokens: python def ask json call llm, prompt, schema, retries=1 : reply = call llm prompt for attempt in range retries + 1 : try: return require parse llm json reply , schema except ValueError, KeyError, TypeError as e: if attempt == retries: raise reply = call llm f"{prompt}\n\nYour previous answer was invalid: {e}. " "Return only the corrected JSON, no extra text." Parsing defensively is the fallback. Where the provider supports it, use structured outputs so the model is constrained to your schema in the first place. See OpenAI's Structured Outputs guide https://platform.openai.com/docs/guides/structured-outputs and Python's json module docs https://docs.python.org/3/library/json.html for the error types you can catch json.JSONDecodeError carries the line and column . What is the weirdest broken-JSON output you have gotten from a model? I'd like to add it to the test cases.