{"slug": "openai-won-t-let-you-escape-freely-in-json-mode", "title": "OpenAI won't let you \"escape\" freely in JSON mode", "summary": "OpenAI and Azure OpenAI's JSON mode endpoints cannot correctly emit escaped Unicode characters like é, instead producing NUL bytes that break systems such as PostgreSQL. The constraint is undocumented and affects any UTF-8 characters the model attempts to escape. The issue is avoidable by using raw UTF-8 in prompt examples rather than \\uXXXX escapes.", "body_md": "# OpenAI won't let you “escape” freely in JSON mode\n\nAccented characters like é may be escaped in JSON as \\u00e9. We found that OpenAI's and Azure OpenAI's endpoints can't emit these correctly in JSON mode: after the prefix \\u00, the decoder allows only control-character completions (\\u0000–\\u001f). The output stays valid JSON but holds the wrong bytes — typically a NUL plus literal e9. Once parsed, those control bytes could break some systems and lead to unexpected errors. This is not a JSON limitation but an undocumented endpoint constraint. We describe the failure mode, reproduce it experimentally, and offer practical mitigations.\n\n### Authors\n\n- Weixuan Xiao\n\n### Published\n\nJune 23, 2026\n\n## TL;DR\n\nAccented characters like `é`\n\nmay be escaped in JSON as `\\u00e9`\n\n. We found that **OpenAI’s** and **Azure OpenAI’s endpoints** can’t emit these correctly in JSON mode: after the prefix `\\u00`\n\n, the decoder allows only control-character completions (`\\u0000`\n\n- `\\u001f`\n\n). So `é`\n\ncannot form. The output stays valid JSON but holds the wrong bytes — typically a `NUL`\n\nplus literal `e9`\n\n(`\\u0000e9`\n\n).\n\nOnce parsed, those control bytes could break production systems: for exmample, PostgreSQL rejects `NUL`\n\nin text, logs and indexes corrupt silently. This is not a JSON limitation — RFC 8259 does permit any `\\uXXXX`\n\nescape. It is an undocumented endpoint constraint from OpenAI and Azure OpenAI.\n\nThis applies not only to the accented characters, but also to any UTF-8 characters that the models want to generate in the escaped sequence `\\uXXXX`\n\nin JSON, including CJK, Hindi, Cyrillic, etc.\n\nThe constraint doesn’t limit what the model can express: JSON can accept raw UTF-8, so `é`\n\nor the other UTF-8 characters don’t necessarily need to be escaped. The trouble starts when prompt examples use `\\uXXXX`\n\nescapes (the default in Python!): the model imitates them, attempts the escape, and hits the blocked path. Show raw characters in your examples and the failure never arises.\n\n## Introduction\n\nSince the announcement of **JSON mode** at OpenAI DevDay in 2023, and the broader push toward **Structured Outputs** (schema-constrained generation) ([OpenAI, 2024](#bib-openai2024structuredoutputs)), many teams have increasingly relied on LLMs to act as *trustable* serializers: “it can just return valid JSON as requested and we can use the JSON directly in production.”\n\nIn practice, these features do a great job at enforcing **outer structure**—objects vs arrays, required keys, field types, enums, and “no extra fields.” But they do not automatically guarantee **the semantic correctness of the content**.\n\nThis article describes a failure mode we observed under OpenAI’s JSON mode, where the output can be **syntactically valid JSON** yet contain **corrupted strings**, with downstream consequences — such as database errors (especially for modern solutions using PostgreSQL as backend, such as Neon and Supabase), confusing system logs due to invisible and unexpected characters, frontend failure due to serialization, etc.\n\nWe first present what we observed in production and explain Structured Outputs and the JSON grammar. Then, we demonstrate our experiments and the results to reproduce the failures under JSON mode on OpenAI endpoints. Finally, we show our findings and conclude with mitigations and takeaways.\n\n## Observations\n\nAt Giskard, we have some Python tooling that uses a **PostgreSQL** database and OpenAI endpoints. The application:\n\n- sends requests to OpenAI endpoints and enables JSON mode\n- parses the JSON results\n- saves the parsed contents in the database.\n\nIn backend logs, we occasionally saw errors where PostgreSQL rejected an insert of a string field because it contained a null byte (`\\x00`\n\n, which PostgreSQL does not accept in text fields):\n\n```\ninvalid byte sequence for encoding \"UTF8\": 0x00\n```\n\nWe found that these invalid strings were coming from LLM outputs and had mismatches in accented characters:\n\n- What we expected: French characters with accents in the words, e.g.\n`é`\n\n- What we actually got: some fragments like\n`Cr\\x00e9dit`\n\nin Python backend\n\nA simplified illustration in the Python string:\n\n```\nexpected:  \"é\"           ( U+00E9 as unicode codepoint)\ngot:       \"\\\\x00e9\"     ( a NUL byte followed by 2 ASCII letters \"e9\" )\n```\n\nIntuitively, it looks like the model meant to produce `\\u00e9`\n\nbut “with extra zeros”, yielding `\\u0000e9`\n\n. After the JSON parsing in Python, the `\\u0000`\n\nbecomes a `\\x00`\n\n(`NUL`\n\nbyte) and the `e9`\n\nliterally becomes 2 letters — `e`\n\nand `9`\n\nin Python string. The PostgreSQL database actually validates the UTF-8 string to be saved, which led to the failure due to the existing `NUL`\n\n.\n\nOn the Internet, we also find related reports — the similar issue also occurs for German letters, and is unresolved on the [OpenAI community forum](https://community.openai.com/t/gpt-4-1-character-encoding-issues/1236017/23).\n\nThe JSON mode, that we enabled, is part of the implementation of Structured Outputs. It could guarantee that the generated contents follow the required JSON structure.\n\nIn JSON specification, RFC 8259 §7 (Strings) specifies that JSON strings can include Unicode characters (UTF‑8), *or* escape sequences as follows:\n\n- A Unicode escape sequence has the form:\n`\\u`\n\n+**exactly four** hex digits, representing one**UTF‑16 code unit**. - ASCII control characters\n`U+0000`\n\n–`U+001F`\n\ncannot appear raw inside JSON strings, but they are allowed**when escaped**(e.g.,`\\u0000`\n\n,`\\u001a`\n\n,`\\u001f`\n\n). - Characters above\n`U+FFFF`\n\ncan be represented as a**surrogate pair**(e.g.,`\\uD83D\\uDE00`\n\nfor 😀).\n\nTherefore, both `é`\n\nand `\\u00e9`\n\ncould be used to represent `é`\n\n(U+00E9) in a JSON string.\n\nNotice that the LLM output presented in the example above is also a valid JSON string: `\"\\u0000e9\"`\n\n. However, it does not represent `é`\n\n. Instead, it represents:\n\n`\\u0000`\n\n(NUL)- followed by the literal characters\n`e`\n\nand`9`\n\nJSON has no semantic notion of “you typed too many zeros.” The parser will accept it as a normal string, and the downstream system will receive a string that actually contains a null byte control character in this case.\n\n## Validations\n\nTo reproduce and understand the issue, we prompted the LLM to directly generate JSON strings containing non-ASCII characters.\n\nThe template is:\n\n```\n<system>\nYou are a JSON generator.\nRespond with a single JSON object that includes only:\n- \"encoded\": a single-entry object whose key is the exact surface word (Unicode) and whose value is a JSON string for the same word using \\\\uXXXX escapes for non-ASCII codepoints.\nDo not add any other top-level keys.\nHere is an example for the text \"©opyright\":\n{\n    \"encoded\": {\n        \"©opyright\": \"\\\\u00a9opyright\"\n    }\n}\n</system>\n\n<user>\nGive me the escaped unicode JSON representation for \"<LETTER>\":\n</user>\n```\n\nwhere `<LETTER>`\n\nis a non-ASCII letter in French or other European languages (e.g. `à`\n\n, `œ`\n\n).\n\nWe first ran experimentations on Azure ** gpt-4o** endpoint, without enforcing JSON grammar (JSON mode is not enabled) for 2380 tests (10 repetitions for 17 non-ASCII letters with uppercase and lowercase, and varied temperatures). In around\n\n**99.8%** tests, it generated the escaped forms of the letters without any issues. The 5 wrong generations are\n\n**caused by the higher temperatures**— one error under 1.7 and four for 2.0. Then, we turned on JSON mode. All the tests are failing with wrong generations or corrupted contents:\n\nHere are some representative causes of the corruptions:\n\n- leading redundant 0:\n- À is U+00C0\n`{\"encoded\": {\"À\": \"\\\\u0000c0\"}}`\n\n- Æ is U+00C6\n`{\"encoded\": {\"Æ\": \"\\\\u0006c\"}}`\n\n- À is U+00C0\n- wrong values:\n- Æ is U+00C6\n`{\"encoded\": {\"Æ\": \"\\\\u001e\"}}`\n\n- Ä is U+00C4\n`{\"encoded\": {\"Ä\": \"\\\\u001c\"}}`\n\n- Ü is U+00DC\n`{\"encoded\": {\"Ü\": \"\\\\u001c\"}}`\n\n- Ÿ is U+0178\n`{\"encoded\": {\"Ÿ\": \"\\\\u00178\"}}`\n\n- Æ is U+00C6\n- or both of the above:\n- Æ is U+00C6\n`{\"encoded\": {\"Æ\": \"\\\\u00066\"}}`\n\n- Ä is U+00C4\n`{\"encoded\": {\"Ä\": \"\\\\u000041\"}}`\n\n`{\"encoded\": {\"Ä\": \"\\\\u000046\"}}`\n\n- Æ is U+00C6\n\nSome discussions and debates warn that format constraints can probably hurt model performance ([Tam & others, 2024](#bib-tam2024speakfreely)), though others argue constraints are necessary in production when implemented correctly ([Kurt, 2024](#bib-kurt2024saywhatyoumean)). Recent work on the format tax ([Lee et al., 2026](#bib-lee2026formattax)) and on separating deliberation from the payload ([Banerjee et al., 2025](#bib-banerjee2025crane); [Nguyen et al., 2026](#bib-nguyen2026thinking)) motivates adding a `reasoning`\n\nfield so the model can think before outputting the `encoded`\n\nvalue, e.g.:\n\n```\n\"© is U+00A9. In JSON I represent it as \\\\\\\\u00a9 before the rest of the letters.\"\n```\n\nThe results remain similar — all 2380 generations failed with the corrupted contents. The `reasoning`\n\nfield often states the correct escape, yet `encoded`\n\nstill diverges. We can see that the LLM does know what to output, but it cannot output correctly with `\\\\u`\n\n(the single backslash `u`\n\n, which is parsed as an escaped character):\n\n```\n{\n    \"reasoning\": \"Ô is U+00D4. In JSON I represent it as \\\\\\\\u00d4.\",\n    \"encoded\": {\"Ô\": \"\\\\u0000d4\"}\n}\n```\n\nHere are detailed results of the failures:\n\nWith the results above, we can observe that, once the JSON mode is turned on with Azure OpenAI endpoints (or OpenAI official endpoints), the model is only capable of outputting:\n\n- either the double-backslash\n`\\\\\\\\u`\n\nin JSON string, which can not be decoded to the original letter, but to a string with 6 literal characters —`\"\\u00e0\"`\n\n- or the single-backslash with the corrupted hex value.\n\nWe also did a supplementary run with a subset of the same task on our self-hosted vLLM OpenAI’s `gpt-oss-20b`\n\nmodel (which shares the same base tokenizer as Azure OpenAI’s `gpt-4o`\n\n, despite that `gpt-4o`\n\ncontains some appended new tokens in the vocabulary, but they will not affect the cases of single letters). There were no such corruption issues at all. This may suggest that the issue is not caused by the models or the tokenizers themselves, but by the JSON mode on the endpoints side.\n\n## Hypothesis: OpenAI’s JSON mode constrains `\\u00`\n\nescaped sequence to control-character only\n\nBased on the results of the experiments, we believe the failures are not from the LLM predicting the wrong tokens, but from the systems implementing the JSON grammar constraints in **Azure OpenAI** and **OpenAI’s official endpoints**.\n\nIn endpoints from both providers, we observed that in **Structured Outputs JSON mode**, once the model has generated the prefix `\\u00`\n\n, the top-10 candidates for the next token only contain control-character range. This could indicate that the *endpoint’s constrained decoder* only permits completions in this range. There is a stricter grammar in JSON mode: only control characters, including `\\u00[0,1][0-9a-f]`\n\n(i.e., `\\u0000`\n\n… `\\u001f`\n\n) and `\\u007f`\n\n(`DEL`\n\n) are permitted.\n\nThis restriction is **not imposed by JSON** (RFC 8259 allows any `\\uXXXX`\n\nescape) or claimed in OpenAI’s documentation ([OpenAI, 2024](#bib-openai2024structuredoutputs)); it appears to be specific to these endpoints’ JSON-mode decoding.\n\nThe corruptions are happening 100%; we managed to reproduce this consistently across the whole OpenAI model family (from `gpt-4o`\n\nup to `gpt-5.2`\n\n) on Azure AI, and `gpt-3.5-turbo`\n\nand `gpt-4o`\n\non OpenAI. Such corruptions do not happen to both OpenAI’s `gpt-oss-20b`\n\nmodel and third-party `qwen3.5-4B`\n\nmodel, hosted by vLLM.\n\nBut our hypothesis still needs to be confirmed by OpenAI, since we do not have enough information about their systems.\n\n### Why this breaks escapes like `\\u00e9`\n\nLet’s take the French letter `é`\n\nagain, which is normally written as `\\u00e9`\n\n.\n\nUnder the observed restriction:\n\n- The model can generate the prefix\n`\\u00`\n\n. - At the next step, tokens corresponding to\n`e`\n\n/`e9`\n\nare masked and excluded; the decoder only allows`00`\n\n,`01`\n\n, …`1f`\n\n. - After leaving that constrained region, we’ve observed three common outcomes:\n**The model insists on the intended value**→ it emits`\\u0000`\n\n(NUL) and then literal`e`\n\nand`9`\n\n, producing**three characters**: NUL +`e`\n\n+`9`\n\n.**The model drifts into corruption**→ it emits`\\u000e`\n\n+ literal`9`\n\n(two characters), or another control escape like`\\u0001`\n\n+ literal`9`\n\n.**The model believes it succeeded**→ the decoded text contains control characters (e.g.,`\\u0002`\n\n), matching patterns like`r\\x02f\\x02rence`\n\n.\n\n- After JSON parsing in Python, these control characters materialize as\n`\\x0X`\n\nor`\\x1X`\n\nbytes, which can trigger downstream failures that we mentioned earlier.\n\n## Mitigations — practical guidance\n\nRecommended approach, if you are using the LLM endpoints from OpenAI or Azure OpenAI:\n\n**Do not demonstrate or strongly encourage**; models often mimic the formatting style you show.`\\uXXXX`\n\n-style escapes in prompts**Do not proactively ask for ASCII-only** output; notice that in Python, the default parameter`ensure_ascii`\n\nof`json.dumps`\n\nis`True`\n\n, which generates`\\uXXXX`\n\nfor non-ASCII characters. This might encourage the models to use escaped sequences in its output. Explicitly setting`ensure_ascii=False`\n\nis highly recommended.**Add checks for anomalies in the output, on top of schema validation**:- Reject or sanitize control characters (especially\n`U+0000`\n\nand generally`U+0000`\n\n–`U+001F`\n\nplus`U+007F`\n\n)\n\n- Reject or sanitize control characters (especially\n\n## Takeaways\n\nFollowing the trend of open-weight models and LLM agents, LLM API providers are emerging. They might have different implementations and considerations in the different components — without documenting every detail. Some behaviors are by-design and expected. For example, OpenAI’s or Azure OpenAI’s endpoints won’t let you “escape” freely in JSON mode — the constraint only allows control characters to be escaped:\n\n**This could be by-design and on-purpose from OpenAI**, because UTF-8 string in JSON can already represent anything except control characters.** But this behavior appears undocumented**in OpenAI’s documentation ([OpenAI, 2024](#bib-openai2024structuredoutputs)), which has been causing confusion for developers for years.\n\nAlso, for JSON, XML, SQL literals, shell escaping, HTML entities, URL encoding, and similar tasks:\n\n- LLMs can express their own intents,\n- but escaping and serialization are handled by the libraries besides the LLMs.\n\nThe main points are simple in practice:\n\n- log system behaviors and proactively find out the subtle failures;\n**keep evaluating and testing your LLMs and agents, before it causes real harms in production.** If you need help in securing your agents, reach out to the[Giskard team](https://www.giskard.ai/contact).\n\n## Bibliography\n\n*Proceedings of ICML*.\n\n*Say What You Mean: A Response to “Let Me Speak Freely.”*https://blog.dottxt.ai/say-what-you-mean.html\n\n*arXiv Preprint arXiv:2604.03616*.\n\n*arXiv Preprint arXiv:2601.07525*.\n\n*Structured outputs*. https://platform.openai.com/docs/guides/structured-outputs\n\n*Proceedings of EMNLP (Industry Track)*.", "url": "https://wpnews.pro/news/openai-won-t-let-you-escape-freely-in-json-mode", "canonical_source": "https://research.giskard.ai/blog/structured-output/", "published_at": "2026-06-25 12:54:36+00:00", "updated_at": "2026-06-25 13:15:37.843105+00:00", "lang": "en", "topics": ["large-language-models", "ai-safety", "ai-products", "ai-tools"], "entities": ["OpenAI", "Azure OpenAI", "Giskard", "PostgreSQL", "Neon", "Supabase"], "alternates": {"html": "https://wpnews.pro/news/openai-won-t-let-you-escape-freely-in-json-mode", "markdown": "https://wpnews.pro/news/openai-won-t-let-you-escape-freely-in-json-mode.md", "text": "https://wpnews.pro/news/openai-won-t-let-you-escape-freely-in-json-mode.txt", "jsonld": "https://wpnews.pro/news/openai-won-t-let-you-escape-freely-in-json-mode.jsonld"}}