# My Demo Script Found a Production Bug on Its First Run: A Tiny Post-Mortem

> Source: <https://dev.to/xbill/my-demo-script-found-a-production-bug-on-its-first-run-a-tiny-post-mortem-k35>
> Published: 2026-07-15 20:36:09+00:00

I built a demo script to show off a project. It failed on step 2 — and in doing so, it found a real bug that had been sitting in the codebase the whole time, invisible to a fully green test suite.

This is a short technical report on what happened, why the tests missed it, and what it taught me about validating against remote APIs. Total incident cost: about ten minutes. Lessons: worth writing down.

The project is an [MCP (Model Context Protocol) server](https://hub.docker.com/r/xbill9/nb2lite-mcp) that wraps Google's `gemini-3.1-flash-lite-image`

model — image generation and stateful editing exposed as four tools that any MCP client (Claude Code, a Google ADK agent, a Rust CLI) can call. I've written up [the architecture separately](https://dev.to/xbill/build-one-ai-tool-server-call-it-from-three-different-agents-mcp-explained-22l2); this post is only about the bug.

The relevant detail: the server validates tool arguments *locally* before calling the API. One of those arguments is `thinking_level`

, the model's latency-vs-quality dial:

```
# server.py — as originally written
SUPPORTED_THINKING_LEVELS = {"minimal", "low", "medium", "high"}

@mcp.tool()
def generate_image(
    prompt: str, aspect_ratio: str = "1:1", thinking_level: str = "medium"
) -> str:
    ...
```

Four allowed values, defaulting to `medium`

. The test suite — 10 unit tests, all mocked — passed. Lint passed. The server had been demoed through agents and worked. Ship it, right?

I wrote a `demo.sh`

that walks through the stack live: list the tools, generate an image, then do a stateful edit. To keep the demo cheap I picked the *lowest* thinking level:

```
cargo run --quiet -- generate "a tiny robot chef cooking ramen" 16:9 minimal
```

First run, step 2:

```
🔴 Image generation failed: Error code: 400 - {'error': {'message':
"'minimal' is not a supported thinking level for this model.
Allowed values are: low, high.", 'code': 'invalid_request'}}
```

The live API accepts exactly **two** thinking levels for this model: `low`

and `high`

. Not four.

Read that against the code above and the real bug jumps out — and it's much worse than a demo flag being wrong:

The server's

defaultwas`medium`

. Every live call that didn't explicitly override`thinking_level`

was a guaranteed HTTP 400.

The local validation layer was happily approving values the API would reject, and rejecting nothing that mattered. It wasn't validating the contract; it was validating a *memory* of the contract.

The test suite mocks the API client:

``` python
@patch("server._get_client")
def test_generate_image_success(self, mock_get_client):
    mock_client.interactions.create.return_value = mock_interaction
    ...
    result = generate_image(prompt="test", thinking_level="medium")
    self.assertIn("🟢 Image successfully saved!", result)
```

This is a *good* test — it verifies the server's own logic: argument handling, base64 decoding, file naming, error shaping. Mocked tests are supposed to isolate you from the network, and they do.

But that isolation cuts both ways: **a mocked test can never detect that the remote contract changed** (or was never what you thought). The mock returns whatever you tell it to, including for requests the real API would reject. My suite effectively asserted "the server correctly forwards `medium`

to the API" — which it did. Correctly forwarding an invalid value is still a bug, just not one visible from inside the mock boundary.

Two things had to line up for this to reach production:

`SUPPORTED_THINKING_LEVELS`

is a local copy of a fact the API owns. Local copies drift — whether because the docs were wrong, the model changed, or the set was written for a different model.`high`

(quality) — masking the broken default and the two phantom values.Mechanically boring, which is the point — the hard part was *knowing*:

```
-SUPPORTED_THINKING_LEVELS = {"minimal", "low", "medium", "high"}
+SUPPORTED_THINKING_LEVELS = {"low", "high"}

-    prompt: str, aspect_ratio: str = "1:1", thinking_level: str = "medium"
+    prompt: str, aspect_ratio: str = "1:1", thinking_level: str = "low"
```

Plus the sweep that actually takes the time: three tool signatures and docstrings, the server's self-describing `get_help`

text, the README, two articles, the API cheat-sheet doc — and a **regression test that locks the new knowledge in**:

```
# The live API only accepts low/high for this model; medium must be rejected
result = generate_image(prompt="test", thinking_level="medium")
self.assertIn("Unsupported thinking level 'medium'", result)
```

Then verification against the real thing: a live call with pure default parameters — the exact case that was broken — returning `🟢 Image successfully saved!`

. And because the server ships as a Docker image, a rebuild and push, since the published image contained the bug too.

One design decision earned its keep during all this: the server's tools never raise — they catch everything and return human-readable `🔴 ...`

strings. The 400 came back as legible text an agent (or a demo script, or me) could read and act on, instead of a stack trace tearing down the MCP session.

**1. Mocked tests verify your code. They cannot verify the contract.** You need at least one test that touches the real API — even a single cheap smoke call. Mine now lives in the demo script and a `/verify-stack`

routine: generate one tiny image with *default parameters*, because defaults are the values nobody passes explicitly and therefore nobody tests.

**2. A local allowlist of remote-owned values is a drift time bomb.** If you must pre-validate (it does give agents faster, clearer errors than a round-trip 400), treat the list as a cache of someone else's truth: comment where it came from, and pin a regression test to the values you've *observed* the API reject.

**3. Test your defaults, specifically.** The bug survived every live interaction before the demo because humans and agents kept overriding the broken default. `f(x)`

being called a hundred times tells you nothing about `f()`

.

**4. A demo script is the cheapest end-to-end test you'll ever write.** It exercises the happy path a real user takes, with real credentials, against the real API — precisely the layer unit tests can't reach. Mine found a production bug on its first execution, before any audience did. Write the demo *before* you think you need one; run it in fast mode (`DEMO_FAST=1`

) whenever the API-facing code changes.

**5. Return errors agents can read.** Tool calls that fail as readable text keep the conversation alive — the calling LLM can see `Allowed values are: low, high`

and retry correctly on its own. That same property is what made this bug a ten-minute fix instead of a debugging session.

| T+0 |
`demo.sh` first run: step 2 fails with HTTP 400 |
| T+1 min | Root cause identified from the error body: API allows `low` /`high` only |
| T+4 min | Server validation + defaults fixed; regression test added |
| T+6 min | Docs swept (README, articles, cheat-sheet, skill); 10/10 tests green |
| T+8 min | Live verification with default params: 🟢 |
| T+10 min | Fixed image rebuilt and pushed to Docker Hub |

The demo, incidentally, works great now — the stateful edit produces the same cyberpunk kitchen with a new neon RAMEN sign, pixel-for-pixel continuity intact. But the bug report turned out to be the better story.

*Have your own "the demo found it" story? I'd love to hear it in the comments.* 👇