{"slug": "your-ai-had-questions-it-answered-them-in-the-code", "title": "Your AI had questions. It answered them in the code.", "summary": "A developer at knopfdruck.ai reports that AI coding assistants silently resolve ambiguity in specifications, picking one rule from a family of rules that fit the provided samples and never surfacing the choice. Because tests generated from those same samples pass for any rule in the family, the mismatch only appears in production on inputs unlike the samples. The proposed fix is to have the AI restate its interpretation in prose, with no code, until it matches the intended rule, then derive tests from that agreed rule.", "body_md": "If you build with AI every day, you know the pattern: the AI delivers, it runs, the tests pass, you nod. A few weeks in, nobody asks why it built things the way it did. With a UI you would notice, because you can see it. With logic in the background you see nothing.\n\nWe caught ourselves doing it. This post is about the mechanism behind it, and what we changed.\n\nSay a quote price comes together like this: material cost and setup time, plus a surcharge that depends on quantity. You write that down in one paragraph, attach a few old quotes as samples, and hand both to two people: a new colleague and an AI.\n\nThe colleague comes back an hour later with questions. What about a material that appears in none of the samples? At what quantity does the surcharge kick in, and does it jump or grow? And a third one you hadn't thought of.\n\nThe AI comes back a minute later with a calculation that runs and hits every sample to the cent.\n\n\"Looks good.\"\n\nThe AI had the same questions. It asked none of them and answered all of them, in the code, without a word.\n\nA paragraph plus samples doesn't define one rule. It defines a family of rules: every rule that reproduces the sample results. A stepped surcharge and a linear one can agree on every quote you attached. So can a fallback that prices an unknown material like steel and one that throws an error.\n\nThe AI has to pick one member of that family. It picks one that fits the samples, and it doesn't mention that there was a choice.\n\nNow look at where your tests come from. If they are built from the same samples, they check exactly what the AI already optimised for, and every rule in the family passes them. Green means the code fits the samples. Whether it implements the rule you meant, those tests can't say.\n\nThe difference shows up with the first input that isn't like the samples: the unusual material, the odd quantity. It shows up in production, not in the test run.\n\nWe saw this ourselves. When we asked the AI how exactly it calculates, its answer described a different rule from the one we meant. The tests had passed anyway, because the samples fit both rules.\n\nThe fix isn't less AI. It's a different start: the AI has to surface its decisions before it writes any code.\n\nSo before anything gets built, the AI answers in prose, with no code:\n\nWe answer, and it restates again. Round by round, until its restatement and our rule are the same rule. Only then does it build.\n\nThat changes two things downstream.\n\nThe tests come from the agreed rule, not just from the samples. Every question that came up in the rounds becomes a test case, so the suite finally holds inputs the samples never had. A passing test now confirms a rule both sides know.\n\nAnd we still read the code, even when it runs. The quickest thing to look for is a constant or a special case that appears in the code and nowhere in the agreed rule. Each one is a decision nobody asked for.\n\nSkip the rounds, and the decisions still get made. You've just handed them to someone who doesn't ask.\n\n*This report was written in German for decision-makers at [knopfdruck.ai](https://knopfdruck.ai/stories/beifahrer-oder-fahrer/).*", "url": "https://wpnews.pro/news/your-ai-had-questions-it-answered-them-in-the-code", "canonical_source": "https://dev.to/knopfdruck/your-ai-had-questions-it-answered-them-in-the-code-1jo3", "published_at": "2026-09-22 14:16:31+00:00", "updated_at": "2026-09-22 14:22:57.544460+00:00", "lang": "en", "topics": ["ai-tools", "developer-tools", "ai-agents", "ai-products"], "entities": ["knopfdruck.ai"], "alternates": {"html": "https://wpnews.pro/news/your-ai-had-questions-it-answered-them-in-the-code", "markdown": "https://wpnews.pro/news/your-ai-had-questions-it-answered-them-in-the-code.md", "text": "https://wpnews.pro/news/your-ai-had-questions-it-answered-them-in-the-code.txt", "jsonld": "https://wpnews.pro/news/your-ai-had-questions-it-answered-them-in-the-code.jsonld"}}