{"slug": "smash-story-the-demo-script-that-out-debugged-my-test-suite", "title": "Smash Story: The Demo Script That Out-Debugged My Test Suite", "summary": "A developer discovered a production bug in their MCP server for Google's gemini-3.1-flash-lite-image model when a demo script revealed that the server's validation layer accepted thinking levels 'minimal' and 'medium' that the live API rejects. The unit tests, which mocked the Gemini client, passed because the mock returned success for any input, masking the bug. The fix involved updating the allowed values to only 'low' and 'high' and adding a regression test.", "body_md": "*This is a Smash Stories submission for the DEV Summer Bug Smash: a debugging story about the gap between \"all tests pass\" and \"it actually works\" — and the unlikely hero that closed it.*\n\nThe project is a small [MCP (Model Context Protocol) server](https://hub.docker.com/r/xbill9/nb2lite-mcp) that wraps Google's `gemini-3.1-flash-lite-image`\n\nmodel. It exposes image generation and *stateful* image editing as four tools that any MCP-speaking agent can call — Claude Code, a Google ADK agent, and a Rust CLI all consume the same ~300-line Python server. (Full architecture write-up [here](https://dev.to/xbill/build-one-ai-tool-server-call-it-from-three-different-agents-mcp-explained-22l2).)\n\nBy every signal a developer normally trusts, it was healthy:\n\nThen I wrote a demo script. It found a production bug in under a minute of runtime.\n\n`demo.sh`\n\nwalks the stack live: discover the tools, generate an image, then do a stateful edit. To keep the demo cheap, step 2 requested the lowest quality tier the server documents:\n\n```\ncargo run --quiet -- generate \"a tiny robot chef cooking ramen\" 16:9 minimal\n```\n\nFirst run:\n\n```\n🔴 Image generation failed: Error code: 400 - {'error': {'message':\n\"'minimal' is not a supported thinking level for this model.\nAllowed values are: low, high.\", 'code': 'invalid_request'}}\n```\n\nWait. The server's own validation had *approved* `minimal`\n\nbefore sending it. Here's that validation:\n\n```\n# server.py — as shipped\nSUPPORTED_THINKING_LEVELS = {\"minimal\", \"low\", \"medium\", \"high\"}\n\n@mcp.tool()\ndef generate_image(\n    prompt: str, aspect_ratio: str = \"1:1\", thinking_level: str = \"medium\"\n) -> str:\n    ...\n```\n\nFour allowed values. The live API accepts **two**: `low`\n\nand `high`\n\n. And look at the default — `medium`\n\n. That's the real smash-worthy find:\n\nEvery live call that didn't explicitly overrideThe validation layer wasn't validating the API's contract — it was validating a stale memory of it.`thinking_level`\n\nwas a guaranteed HTTP 400.\n\nThe suite mocks the Gemini client, as unit tests should:\n\n``` python\n@patch(\"server._get_client\")\ndef test_generate_image_success(self, mock_get_client):\n    mock_client.interactions.create.return_value = mock_interaction\n    result = generate_image(prompt=\"test\", thinking_level=\"medium\")\n    self.assertIn(\"🟢 Image successfully saved!\", result)\n```\n\nThe mock returns success for *any* input — including inputs the real API rejects. The tests correctly proved \"the server forwards `medium`\n\nfaithfully.\" Faithfully forwarding an invalid value is still a bug; it's just invisible from inside the mock boundary.\n\nTwo conditions had to align for this to ship:\n\n`SUPPORTED_THINKING_LEVELS`\n\nwas a cached copy of a fact only the API owns. Cached copies drift.`high`\n\nfor quality — so the broken default and the two phantom values were never exercised. `f(x)`\n\nbeing called a hundred times tells you nothing about `f()`\n\n.Two lines of production code, plus the part that actually takes discipline — locking the discovery in so it can't regress:\n\n```\n-SUPPORTED_THINKING_LEVELS = {\"minimal\", \"low\", \"medium\", \"high\"}\n+SUPPORTED_THINKING_LEVELS = {\"low\", \"high\"}\n\n-    prompt: str, aspect_ratio: str = \"1:1\", thinking_level: str = \"medium\"\n+    prompt: str, aspect_ratio: str = \"1:1\", thinking_level: str = \"low\"\n# New regression test: the live API only accepts low/high for this\n# model; medium must now be rejected locally with a readable error.\nresult = generate_image(prompt=\"test\", thinking_level=\"medium\")\nself.assertIn(\"Unsupported thinking level 'medium'\", result)\n```\n\nThen the sweep (three tool signatures, docstrings, the server's self-describing `get_help`\n\n, every doc that repeated the wrong values) and a rebuild + push of the published Docker image, which had been shipping the bug to anyone who pulled it.\n\n| Before | After | |\n|---|---|---|\n| Live call with default params | HTTP 400, every time | 🟢 image saved |\n`thinking_level=\"minimal\"` / `\"medium\"`\n|\nApproved locally, rejected remotely | Rejected locally with the allowed values named |\n| Test suite | 10/10 green (bug invisible) | 11 assertions incl. contract regression test |\nPublished image `xbill9/nb2lite-mcp`\n|\nShipped the broken default | Rebuilt, pushed, verified live |\n\nElapsed time from first failure to fixed-image-on-Docker-Hub: about ten minutes — because the failing tool call came back as readable text (`Allowed values are: low, high`\n\n) instead of a stack trace. Error messages that name the fix are half the debugging.\n\n`DEMO_FAST=1 ./demo.sh`\n\n).The whole project is built on Google AI, end to end:\n\n`gemini-3.1-flash-lite-image`\n\n`store=True`\n\n+ `previous_interaction_id`\n\n) are what make multi-turn image editing work: the demo's edit step adds a neon RAMEN sign to `LlmAgent`\n\non `gemini-2.5-flash`\n\n`MCPToolset`\n\n— Gemini calling Gemini, with the bug fix sitting in between.`Allowed values are: low, high`\n\n) is what made this a ten-minute smash.`xbill9/nb2lite-mcp`", "url": "https://wpnews.pro/news/smash-story-the-demo-script-that-out-debugged-my-test-suite", "canonical_source": "https://dev.to/gde/smash-story-the-demo-script-that-out-debugged-my-test-suite-430k", "published_at": "2026-07-15 23:58:42+00:00", "updated_at": "2026-07-16 00:06:33.992208+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "large-language-models", "ai-tools"], "entities": ["Google", "Gemini", "Claude Code", "Rust", "MCP"], "alternates": {"html": "https://wpnews.pro/news/smash-story-the-demo-script-that-out-debugged-my-test-suite", "markdown": "https://wpnews.pro/news/smash-story-the-demo-script-that-out-debugged-my-test-suite.md", "text": "https://wpnews.pro/news/smash-story-the-demo-script-that-out-debugged-my-test-suite.txt", "jsonld": "https://wpnews.pro/news/smash-story-the-demo-script-that-out-debugged-my-test-suite.jsonld"}}