{"slug": "your-llm-returns-json-that-crashes-json-loads-about-1-time-in-20-here-is-how-to", "title": "Your LLM returns JSON that crashes json.loads about 1 time in 20. Here is how to handle it", "summary": "A developer published a step-by-step Python approach for handling malformed JSON returned by LLMs, which fails roughly one time in 20 in production. The technique strips markdown code fences and surrounding prose, extracts the outermost object or array, repairs trailing commas, validates required keys and types, and feeds the exact error back to the model for a single retry. The author recommends structured outputs where providers support them, framing defensive parsing as the fallback.", "body_md": "If you call an LLM and ask for JSON, it works in the demo and then fails in production at the worst moment. The usual culprits are a markdown code fence around the JSON, a sentence before it (\"Sure! Here is the JSON:\"), or a trailing comma.\n\nHere is a small, step-by-step way to deal with all three in plain Python.\n\n``` python\nimport json\nimport re\n\ndef extract_json(text: str):\n    text = text.strip()\n    # 1. remove ```\n{% endraw %}\njson ...\n{% raw %}\n ``` fences\n    fence = re.search(r\"```\n\n(?:json)?\\s*(.*?)\n\n```\", text, re.DOTALL)\n    if fence:\n        text = fence.group(1).strip()\n    # 2. keep only the outermost {...} or [...]\n    starts = [i for i in (text.find(\"{\"), text.find(\"[\")) if i != -1]\n    if not starts:\n        raise ValueError(\"no JSON found\")\n    start = min(starts)\n    end = max(text.rfind(\"}\"), text.rfind(\"]\"))\n    return text[start : end + 1]\npython\ndef parse_llm_json(text: str):\n    raw = extract_json(text)\n    try:\n        return json.loads(raw)\n    except json.JSONDecodeError:\n        # remove trailing commas before } or ]\n        fixed = re.sub(r\",\\s*([}\\]])\", r\"\\1\", raw)\n        return json.loads(fixed)  # let it raise if still broken\n```\n\nValid JSON can still have the wrong keys. Check the fields you rely on:\n\n``` python\ndef require(data: dict, schema: dict):\n    for key, typ in schema.items():\n        if key not in data:\n            raise KeyError(f\"missing key: {key}\")\n        if not isinstance(data[key], typ):\n            raise TypeError(f\"{key} should be {typ.__name__}\")\n    return data\n\nresult = require(parse_llm_json(reply), {\"title\": str, \"score\": int})\n```\n\nFeed the exact error back to the model. One retry fixes most failures, and more than two retries usually just burns tokens:\n\n``` python\ndef ask_json(call_llm, prompt, schema, retries=1):\n    reply = call_llm(prompt)\n    for attempt in range(retries + 1):\n        try:\n            return require(parse_llm_json(reply), schema)\n        except (ValueError, KeyError, TypeError) as e:\n            if attempt == retries:\n                raise\n            reply = call_llm(\n                f\"{prompt}\\n\\nYour previous answer was invalid: {e}. \"\n                \"Return only the corrected JSON, no extra text.\"\n            )\n```\n\nParsing defensively is the fallback. Where the provider supports it, use structured outputs so the model is constrained to your schema in the first place. See OpenAI's [Structured Outputs guide](https://platform.openai.com/docs/guides/structured-outputs) and Python's [json module docs](https://docs.python.org/3/library/json.html) for the error types you can catch (`json.JSONDecodeError` carries the line and column).\n\nWhat is the weirdest broken-JSON output you have gotten from a model? I'd like to add it to the test cases.", "url": "https://wpnews.pro/news/your-llm-returns-json-that-crashes-json-loads-about-1-time-in-20-here-is-how-to", "canonical_source": "https://dev.to/renev3408/your-llm-returns-json-that-crashes-jsonloads-about-1-time-in-20-here-is-how-to-handle-it-4n2p", "published_at": "2026-10-15 22:38:54+00:00", "updated_at": "2026-10-15 22:58:55.114443+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "developer-tools", "structured-data"], "entities": ["OpenAI"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/your-llm-returns-json-that-crashes-json-loads-about-1-time-in-20-here-is-how-to", "markdown": "https://wpnews.pro/news/your-llm-returns-json-that-crashes-json-loads-about-1-time-in-20-here-is-how-to.md", "text": "https://wpnews.pro/news/your-llm-returns-json-that-crashes-json-loads-about-1-time-in-20-here-is-how-to.txt", "jsonld": "https://wpnews.pro/news/your-llm-returns-json-that-crashes-json-loads-about-1-time-in-20-here-is-how-to.jsonld"}}