I Fixed LLM Formatting by Stopping the Prompt Obsession

A developer solved LLM formatting errors by separating content generation from formatting using a Jinja2 templating engine. Instead of prompting the LLM to output Markdown directly, the system now outputs structured JSON, which is then rendered deterministically with Jinja2 templates. This approach reduced formatting errors to 0% and cut manual editing time from 30 minutes per article to instant generation.

Dealing with rendering crashes caused by unstable LLM outputs? Instead of fighting with prompts, I handed over control to a Jinja2 templating engine. By separating content generation from formatting, I reduced formatting errors to 0% and cut manual editing time from 30 minutes per article to instant generation. In a production environment, relying on LLMs to generate Markdown directly is a nightmare. We frequently encountered missing code block closing tags and broken table syntax, causing frontend rendering to crash. The core issue is that LLM token generation is inherently probabilistic. No matter how detailed your prompt is, you cannot guarantee strict syntax adherence—especially with nested code blocks or complex tables. If left unchecked, this requires engineers to spend 30 minutes formatting each article. With 10 articles daily, that’s 200 hours a month wasted on non-automatable fixes. LLMs operate on Next Token Prediction. They don't adhere to syntax like a compiler. For example, a model might output: python def func : return True Missing the closing triple backticks Even if your System Prompt screams "You MUST close code blocks," the instruction's weight gets diluted during long-context generation. By the time the model reaches the end of a long response, the structural integrity often loosens. Asking the LLM to output the final text directly means you give up control. You can't validate or sanitize the data before it hits the renderer. The shift was simple but powerful: Treat the LLM as a pure data provider. Instead of asking for Markdown, the LLM now outputs structured JSON or XML. Deterministic code Jinja2 handles the Markdown stitching. Before: Relying on LLM for Markdown response = client.chat.completions.create model="gpt-4", messages= {"role": "user", "content": "Output a Markdown article with Python code"} markdown content = response.choices 0 .message.content Probabilistic, high risk After: LLM outputs JSON, Jinja2 handles formatting prompt = """ Return the article content in JSON format, including title, sections list , and code snippets list . Do NOT include Markdown syntax. """ llm response = client.chat.completions.create model="gpt-4", messages= ... article data = json.loads llm response.choices 0 .message.content Deterministic rendering env = Environment loader=FileSystemLoader 'templates' template = env.get template 'article layout.jinja2' final markdown = template.render article data 100% format correct Before rendering, I added a "Format Sanitizer" layer. This performs strong type checking on JSON fields to filter out potential XSS characters or syntax-breaking strings. | Decision | Alternative | Rationale | |---|---|---| Jinja2 Templating | Prompt Engineering | Prompts are soft constraints; templates are hard constraints. Absolute correctness is required. | Structured JSON | Regex Post-processing | Patching probability with regex is complex and error-prone. Structured data isolates content from format at the source. | Backend Template Layer | Frontend JS Fixes | Processing format on the backend ensures clean data storage and avoids repetitive logic across clients App/Web . | The refactor paid off immediately: This post was automatically generated by Agent Daily Publisher