The Agentic Code That Compiles but Crashes in Production

A developer outlined a set of defensive patterns for catching agent-generated Python code that passes CI but fails at runtime, including try-except guards around imports like `requests`, retry wrappers with exponential backoff and limits for external API calls, and explicit environment-variable checks that raise clear configuration errors at startup. The writeup also recommends running agentic output through the same GitHub Actions validation pipeline as hand-written code so failures surface early rather than in production.

You see a pull request where the agent generated a function that imports requests and calls an external API. The CI passes, but the service crashes in staging because the container lacks the library. This mismatch between sandbox and production is why good developers stay wary of agentic development. You will learn how to spot hallucinated imports, add safety wrappers around external calls, and integrate agentic output into a CI pipeline so it fails fast instead of at runtime. php Generated by an agent – looks fine locally def fetch data url: str - dict: import requests resp = requests.get url, timeout=5 return resp.json The snippet imports requests without checking if the package is installed. In a production container that only has requests in a dev-only extra, the import fails at runtime. You can guard against this by adding a try-except around the import and falling back to a safe stub: php def fetch data url: str - dict: try: import requests except ImportError: class MissingPackage: def get self, args, kwargs : raise RuntimeError "The 'requests' package is missing in production." requests = MissingPackage resp = requests.get url, timeout=5 return resp.json The guard makes the failure explicit early, so you can add the missing dependency to your production image before the service starts. Generated code often omits rate limiting or circuit breaking. An agent may produce a function that blindly retries on failure, exhausting a third-party quota. php Agent output – no throttling def query service payload: dict - str: return httpx.post "https://api.example.com/endpoint", json=payload .text Wrap it with a simple retry with exponential backoff and a limit: python import httpx import time def query service payload: dict, max retries: int = 3, base delay: float = 0.5 - str: for attempt in range max retries + 1 : try: resp = httpx.post "https://api.example.com/endpoint", json=payload, timeout=10 resp.raise for status return resp.text except httpx.HTTPStatusError, httpx.NetworkError as exc: if attempt == max retries: raise RuntimeError f"Service call failed after {max retries} attempts" from exc delay = base delay 2 attempt time.sleep delay The wrapper ensures you do not hammer the external API and you get a clear error after the final attempt. Agents frequently reference environment variables without verifying they exist. A generated function might read a secret from os.environ that was never set in the deployment environment, causing an immediate crash on startup. php Generated code – assumes the env var exists def get api key - str: return os.environ "API KEY" If API KEY is absent, this raises a KeyError with no context about whether the variable is misconfigured or genuinely missing. Replace the bare lookup with a helper that surfaces configuration errors clearly: php def get api key - str: key = os.environ.get "API KEY" if key is None: raise RuntimeError "Configuration error: 'API KEY' is not set. Check your environment variables." return key This pattern catches missing configuration at startup rather than failing later when the feature is actually used. Even well-checked generated code should be validated by the same pipeline as hand-written code. A minimal GitHub Actions workflow looks like this: name: Validate Agentic Code on: push, pull request jobs: validate: runs-on: ubuntu-latest steps: - uses: actions/checkout@v4 - name: Set up Python uses: actions/setup-python@v5 with: python-version: "3.11" - name: Install dependencies run: | python -m pip install --upgrade pip pip install -r requirements.txt - name: Run unit tests run: pytest - name: Run static type check run: mypy src/ The workflow installs the exact dependencies you declare, runs unit tests, and runs a static type checker. This catches missing imports and type mismatches before the code reaches production. Even with guards, agentic output can introduce subtle bugs: Mitigate these by treating generated code as a draft. Run a linter, enforce code-review checklists, and keep a log of which files were auto-generated. Why are good developers so wary of agentic development? https://martinfowler.com/rachels-ramblings/good-devs-wary.html – I added concrete code examples for import guards, retry wrappers, and environment-variable handling, plus a full CI workflow, to show how you can safely incorporate agentic output into production-ready systems. These write-ups are researched and published with no paywall, sponsor, or tracking. If one saved you an afternoon, a small tip keeps them coming. USDT, USDC or USDD · TRC-20 Tron TFTNsfyomKrnUutRjBTGVULp19ByW29KbY