# The bug that passes every test and does nothing. I let an AI write more than I read, and the tests that would have caught all of it. > Source: > Published: 2026-08-10 10:27:11+00:00 Six real silent failures from 14 months of shipping alone, the week I let an AI write more than I read, and the tests that would have caught all of it. A crash is honest. It tells you something went wrong. Silence is also a claim. It says everything went fine. That claim is far more expensive when it is false, and it is the one we are now producing at scale. **I have shipped a Windows system monitor in public for fourteen months,** alone, in the evenings after work. Everything is on GitHub from the first commit, so this is a record I cannot edit. Here are six failures from it, none of which crashed, all of which passed every test I owned. ❤️ [If you want to support me, click here](https://buycoffee.to/hcklabs) :) ❤️ A fan curve editor. Drag points, click Apply, green message. The message appeared. The file was never written. Two releases of users setting a curve, seeing a confirmation, restarting a week later and finding defaults back. Zero reports, because **user cannot tell the difference between "it saved" and "it said it saved."** A success message is not evidence of success. It is a string. ``` temps = psutil.sensors_temperatures() # {} on Windows, always cpu = temps.get("coretemp", []) # [] if cpu and cpu[0].current > 80: # never true warn() ``` `psutil.sensors_temperatures()` returns an empty dict on Windows. Not an error. So the thermal monitor ran on schedule, found nothing to warn about, and reported all clear. For months. The test asserted `warn()` was not called when temperatures were normal. **An empty reading is indistinguishable from a normal reading if you never assert a reading exists.** ``` python # Weak: passes when temps is empty, which IS the bug def test_no_false_alarm(): assert not monitor.check(temps={}).warned # Stronger: the reading itself is the subject def test_temperature_source_returns_data(): reading = sensors.read_cpu_temp() assert reading is not None assert 0 < reading < 150 ``` If a test would still pass on a machine where the feature is switched off, it is not testing the feature. `except: pass` and four months of a dead subsystem ``` try: ctx = build_learning_context() except: pass ``` Two lines above, a `NameError` on every single call. The bare `except` swallowed it. No crash, no log. Learning engines had been running and producing correct numbers for four months while the layer that consumes them never received a thing. A bare `except: pass` does not handle an error. It deletes the evidence that one occurred. Go grep your project for `except:` followed by `pass` . I will wait. Rule now: ``` except Exception as e: log_event("learning_context_failed", repr(e)) # never silent ctx = None ``` One line. Alternative cost four months. A TURBO toggle on the dashboard. ** Clickable**. I tested the feature by calling the function directly. I tested the button by looking at it. The two halves of a feature can both be correct and still not be a feature. ** Packaged Windows apps live in a folder users cannot right-click into**, so app creates a desktop shortcut for them. It has to point at an identifier built from the package family name plus the Application Id in the manifest. ``` manifest: Application Id="App" shortcut code: ...PCWorkman_4hekbcs2ddfbc!PCWorkmanHCK ``` Every Store user who used that feature got a shortcut that launched nothing. No error. Zero bug reports, because **nobody files a ticket about a shortcut that does nothing.** They double-click twice, shrug, and never use it again. Identifiers that must agree across two files will eventually disagree. Test the agreement, not either side of it. A process-inspection engine with signature, typosquat and masquerade checks. It correctly catches `svch0st.exe` impersonating `svchost.exe` . It also flagged `spoolsv.exe` , with a valid Microsoft signature from the correct System32 path. Cause: the process library carries a note meaning "heavy, watch resource use". That note was raising the **security** verdict. Two different kinds of truth in one field. When one field carries two kinds of truth, one of them will eventually answer a question it was never asked. | What happened | What the system reported | |---|---| | Settings never written | Applied successfully | | Temperature never read | All clear | | Learning never called | A confident answer | | Feature never connected | A working button | | Shortcut never valid | A shortcut on the desktop | | Advisory note misread | A security verdict | **A silent failure is a false claim of success.** Not an absence of output. A wrong output that happens to be reassuring. I work with an AI assistant and say so on every post. Normally that means a conversation: ask, read properly, argue with a third of it, keep what survives. Then came a release week. Store submission, a version bump across 42 files, a build, a package, and day shifts behind a steering wheel. I started accepting more and reading less. In one week: **Plausible data written into a database.** Thirty-five entries added to the known-process library. Vendor fields looked entirely reasonable. `bash.exe` was attributed to "*Git Development Community*". **Real Authenticode signer** is a person's name, and the engine compares expected vendor to actual signature, so a mismatch raises a warning. Result: **three processes that were merely unrecognised became flagged as suspicious.** Confident, plausible, and worse than writing nothing. **A regex that passed tests and failed in production.** ``` # Passed every unit test. Never matched in the running app. pos = text_widget.search(r'\[-> [^\]]+\]', idx, regexp=True) ``` The unit tests used Python's `re` . The widget's search is evaluated by Tcl, whose bracket expressions do not treat `[^\]]` same way. **Two engines, one string, no error message.** Every link rendered as plain text. Fix: search for a literal prefix, parse with the engine the pattern was written for. ``` php pos = text_widget.search('[-> ', idx) # literal, engine-agnostic m = re.match(r'\[-> ([^\]]+)\]', line_text) # Python parses Python ``` **A git reset --hard that wiped a day of uncommitted work.** **A cleanup script that ate 38 commas.** A punctuation pass across 15 HTML files removed the comma after 38 closing tags. `"Driver conflicts, leftover GPU packages"` became `"Driver conflictsleftover GPU packages"` . Nothing crashed. Every page rendered perfectly. None of these threw an exception. **All of them produced output that looked right.** This is not an argument against working this way. Project moves faster because of it. It is an argument about **where review has to happen**. Generated code is fluent by construction: it compiles, reads well, uses the right function names. **Fluency is not correctness, and fluency is exactly what makes the difference invisible.** **Verify facts, never accept plausible ones.** If a value can be read from the system, read it. A plausible fact is more dangerous than a missing one, because a missing one gets checked. **Ask which engine actually runs this.** Before trusting a green test, ask whether it exercises the same runtime the user hits. **Never let a destructive command through unread.** Anything with `--hard` , `--force` , `rm` , `DROP` or `reset` gets read character by character. Back up first. **Check the output, not the exit code.** The comma script "*succeeded*". The vendor entries "*succeeded*". All six bugs above "*succeeded*". **Write a ratchet the same day.** The fix is half the work. The other half is a test that fails the build if it comes back. They only turn one way. Every bug here has one now. **Click the thing.** After a refactor that split one module into seven, 96 tests stayed green while every sidebar page silently fell back to the dashboard. Not one test built the real window. Five minutes of human clicking caught what the whole suite could not. **Instrument the first divergence, not the damage at the end.** Three days on a replay bug, measuring how far apart two runs ended up. That number tells you the size of the damage and nothing else. Logging first tick where they stopped agreeing turned evenings into minutes. **Trust probes, not names.** Detecting a read-only install folder by checking whether the path contains `WindowsApps` is a guess about the world. A write probe is a fact about it. I build alone. No reviewer, nobody to ask "did you check that it actually saved?" These bugs did not survive because they were subtle. Several were obvious. They survived because exactly one person could have caught them, and that person had already decided the feature worked. **You do not write a test for a feature you already believe works.** That is not a technical problem. It is the problem of being the only witness. I am 22, in Poland, self-taught after a technical school, and twelve projects died before this one. The laptop most of it was built on is from 2014 and hits 94 degrees. The day job has been a warehouse, then welding plastic, now a taxi. You do not do careful review at midnight after a twelve-hour shift. **You do the thing that feels finished.** Everything in the first half of this article is what "feels finished" looks like six months later. Three things help, and none is discipline: **write in public** (the difference between how you describe a feature and what it does is where these live), **ship to people who owe you nothing** (a tester refused to accept "it works on my machine" about a console that would not close, and he was right), and **keep a log** for the version of you in six months. We are getting better at producing code that reads correctly and faster at producing it. Neither makes code more likely to do what you meant. **Do not accept an outcome as proof of an action.** Not from your code, not from your tools, not from anything that generates text for you, and not from yourself at midnight. Look for the receipt. *I build PC Workman, a free Windows system monitor with a fully offline assistant. 331 automated tests and a public list of everything above.*