Finding the Bugs I Built an AI Agent to Prevent: Building Deterministic Guardrails with LangGraph & Python

A developer built Ahmad-AI, a local LangGraph/Python/Ollama agent with live SQL and Islamic knowledge base access, and hardened it with deterministic Python guardrails after six rounds of adversarial testing exposed fabrication bugs. The agent invented fake item IDs and stock levels alongside an honest "no results" message, and cited a real but irrelevant Quran verse (5:51) because the citation guard's regex didn't match the model's output format. The system now ships with a 22-test unit suite, a live end-to-end harness, a pre-verified citation whitelist, and bounded tool rounds.

A real debugging log: fabrication bugs, deployment gaps, and false test passes found and closed across 6 rounds of adversarial testing. System prompts are a leash, not a lobotomy. A system prompt can be ignored, misread, or drift under pressure. Deterministic code cannot. When I built Ahmad-AI — a local AI agent using LangGraph, Python, and Ollama with access to live SQL databases and Islamic knowledge bases — I established one non-negotiable design rule: It must never state a specific citation or data claim unless that claim is independently verified. Verified, or it says nothing. Because general-purpose LLMs fabricate citations confidently and often, "confidently wrong" is worse than "honestly uncertain." I spent weeks trying to prove the system would break its own rule anyway. It did. Repeatedly. Here is what I learned across six rounds of adversarial testing against a live agent. I ran a set of test questions against the live agent — empty database queries, Quran & Hadith citation requests, and ambiguous sorting queries. Two critical bugs immediately surfaced: When querying a table with no matching rows, the agent correctly output: "The query did not return any results." In the exact same sentence, it invented two fake item IDs with fake stock levels. When asked about usury riba , the agent cited verse 5:51 a real Quran verse, but entirely about interfaith alliances . My citation guard failed to fire because its regex expected Surah name X:Y, while the model wrote Quran X:Y . A safety mechanism that doesn't match the shape of the output is functionally non-existent. My first instinct was to strengthen the system prompt. I resisted. Instead of writing tighter rules for the LLM, I built a deterministic backstop in Python that overrides the model’s prose when known failure patterns appear in state: python python: def apply tool result guards state, response : """ Deterministic backstop: if the last DB query came back empty or hard-errored, override entirely with a fixed, honest message regardless of what the model wrote. """ if is no data last tool message : return "No matching data was found for that query. The data may not exist yet." ALLOWED CITATIONS = {"20:114"} Pre-verified whitelist def enforce citation guard text : for match in CITATION PATTERN.finditer text : if not any allowed in match.group 0 for allowed in ALLOWED CITATIONS : return SAFE ISLAMIC FALLBACK Blocked even if the citation is real return text Manually retyping test prompts in a terminal doesn't scale. To keep the agent honest, I split automated testing into two distinct layers: Unit Tests 22 tests : Direct execution of Python guard functions against synthetic inputs fast, no live LLM required . End-to-End Evaluation Harness: Drives the live LangGraph execution against Ollama and Postgres. Treating fuzzy model outputs as binary pass/fail either hides real regressions behind noisy false failures or downgrades hard security checks to "close enough." A whitelist beats a correctness checker: Verifying "is this citation accurate?" is hard. Checking "is this citation on the pre-approved whitelist?" is simple and reliable. Print a Build Stamp at Startup: A version stamp printed at agent startup costs nothing and prevents testing against stale code. Distinguish Transient from Permanent Errors: Retrying a connection drop makes sense. Retrying a bad SQL column name is just a slower way to fail. Don't ask LLMs to do arithmetic or row matching: Stating query answers explicitly in tool outputs prevents models from misreading JSON rows. Read the Full Engineering Log This architecture is live, tested against a 22-unit-test suite and a 5/5 E2E hard check harness. For the complete technical breakdown — including deployment gap fixes, bounded execution limits MAX TOOL ROUNDS , and verbatim adversarial outputs: 📖 Read the full interactive log on my portfolio: 👉 Engineering Log 01: Guardrails & Hallucination Prevention https://emadaed.github.io/digisuite/engineering-log.html Muhammad Ahmad is an AI Agent Engineer and Digital Supply Chain Architect PMP®, RMP® with 15+ years of operational leadership experience.