I Fixed LLM Markdown Errors with Jinja2 and AST Parsing

LLMs are great at generating content, but terrible at keeping it clean. In the ai-developer-knowledge-hub

project, we faced a recurring nightmare: the technical documents generated by the LLM were riddled with formatting issues. Specifically, code blocks often lacked closing markers or had unclosed strings, crashing our frontend rendering engine.

We tried the obvious route: optimizing the Prompt. We begged the model to "output correct markdown syntax." The result? A 15% error rate. That's unacceptable for an automated publishing pipeline.

The core challenge is bridging the gap between a probabilistic system (the LLM) and a deterministic requirement (valid Markdown). Direct Regex cleaning was too fragile, and letting the LLM self-correct led to infinite loops.

}

in a JSON config block once threw a TemplateSyntaxError

in Jinja2, blocking the entire publishing pipeline.The breakthrough was decoupling content generation from style rendering. Instead of trusting the raw text, we pipe it through a validation layer using AST (Abstract Syntax Tree) parsing.

If the AST check fails, we sanitize. If it passes, we extract structured blocks and feed them into a Jinja2 template. This ensures the output structure is 100% locked down by the template engine, not guessed by the LLM.

Here is the implementation:

prompt = "Please output markdown code blocks with correct syntax."
raw_text = llm.generate(prompt)

def render_pipeline(llm_output: str) -> str:
    try:
        markdown_parser.parse(llm_output)
    except SyntaxError:
        return fallback_sanitize(llm_output)

    content_blocks = extract_code_blocks(llm_output)

    template = jinja_env.get_template("article_layout.md")
    return template.render(blocks=content_blocks)

Parsing can fail, and LLMs can hang. We needed a strategy that prioritizes content delivery over perfection. We implemented an exponential backoff retry mechanism with a "text-only" fallback.

If rendering fails after retries, we don't crash; we strip the formatting and serve the raw text. Content is king, but we also log 10% of these failures for debugging without exploding our storage costs.

for _ in range(3):
    result = generate_and_check()

MAX_RETRIES = 2
TIMEOUT = 5.0  # seconds
LOG_SAMPLE_RATE = 0.1  # 10% error sampling rate

for attempt in range(MAX_RETRIES):
    try:
        return strict_render(llm_output, timeout=TIMEOUT)
    except ASTParseError as e:
        if attempt == MAX_RETRIES - 1:
            if random.random() < LOG_SAMPLE_RATE:
                logger.error(f"Render failed: {e}")
            return text_only_fallback(llm_output)
        time.sleep(2 ** attempt) # Exponential backoff

By moving the formatting responsibility from the LLM to a deterministic rendering pipeline, we solved the reliability issue once and for all.