{"slug": "the-bug-that-passes-every-test-and-does-nothing-i-let-an-ai-write-more-than-i-of", "title": "The bug that passes every test and does nothing. I let an AI write more than I read, and the tests that would have caught all of it.", "summary": "A developer who shipped a Windows system monitor alone for 14 months documented six silent failures where bugs passed all tests and produced no errors, including a fan curve editor that showed a success message without writing the file and a thermal monitor that never detected temperatures because psutil returns an empty dict on Windows. The developer emphasizes that success messages and empty readings are not evidence of correctness and recommends stronger tests that assert the reading itself exists.", "body_md": "Six real silent failures from 14 months of shipping alone, the week I let an AI write more than I read, and the tests that would have caught all of it.\n\nA crash is honest. It tells you something went wrong.\n\nSilence is also a claim. It says everything went fine.\n\nThat claim is far more expensive when it is false, and it is the one we are now producing at scale.\n\n**I have shipped a Windows system monitor in public for fourteen months,** alone, in the evenings after work. Everything is on GitHub from the first commit, so this is a record I cannot edit. Here are six failures from it, none of which crashed, all of which passed every test I owned.\n\n❤️ [If you want to support me, click here](https://buycoffee.to/hcklabs) :) ❤️\n\nA fan curve editor. Drag points, click Apply, green message.\n\nThe message appeared. The file was never written. Two releases of users setting a curve, seeing a confirmation, restarting a week later and finding defaults back.\n\nZero reports, because **user cannot tell the difference between\n\"it saved\" and \"it said it saved.\"**\n\nA success message is not evidence of success. It is a string.\n\n```\ntemps = psutil.sensors_temperatures()      # {} on Windows, always\ncpu = temps.get(\"coretemp\", [])            # []\nif cpu and cpu[0].current > 80:            # never true\n    warn()\n```\n\n`psutil.sensors_temperatures()`\n\nreturns an empty dict on Windows.\n\nNot an error. So the thermal monitor ran on schedule, found nothing to warn about, and reported all clear. For months.\n\nThe test asserted `warn()`\n\nwas not called when temperatures were normal.\n\n**An empty reading is indistinguishable from a normal reading if you never\nassert a reading exists.**\n\n``` python\n# Weak: passes when temps is empty, which IS the bug\ndef test_no_false_alarm():\n    assert not monitor.check(temps={}).warned\n\n# Stronger: the reading itself is the subject\ndef test_temperature_source_returns_data():\n    reading = sensors.read_cpu_temp()\n    assert reading is not None\n    assert 0 < reading < 150\n```\n\nIf a test would still pass on a machine where the feature is switched off, it is not testing the feature.\n\n`except: pass`\n\nand four months of a dead subsystem\n\n```\ntry:\n    ctx = build_learning_context()\nexcept:\n    pass\n```\n\nTwo lines above, a `NameError`\n\non every single call. The bare `except`\n\nswallowed it. No crash, no log. Learning engines had been running and producing correct numbers for four months while the layer that consumes them never received a thing.\n\nA bare\n\n`except: pass`\n\ndoes not handle an error. It deletes the evidence\n\nthat one occurred.\n\nGo grep your project for `except:`\n\nfollowed by `pass`\n\n. I will wait.\n\nRule now:\n\n```\nexcept Exception as e:\n    log_event(\"learning_context_failed\", repr(e))   # never silent\n    ctx = None\n```\n\nOne line. Alternative cost four months.\n\nA TURBO toggle on the dashboard. ** Clickable**.\n\nI tested the feature by calling the function directly.\n\nI tested the button by looking at it.\n\nThe two halves of a feature can both be correct and still not be a feature.\n\n** Packaged Windows apps live in a folder users cannot right-click into**, so app creates a desktop shortcut for them. It has to point at an identifier built from the package family name plus the Application Id in the manifest.\n\n```\nmanifest:       Application Id=\"App\"\nshortcut code:  ...PCWorkman_4hekbcs2ddfbc!PCWorkmanHCK\n```\n\nEvery Store user who used that feature got a shortcut that launched nothing.\n\nNo error. Zero bug reports, because **nobody files a ticket about a shortcut that does nothing.**\n\nThey double-click twice, shrug, and never use it again.\n\nIdentifiers that must agree across two files will eventually disagree. Test\n\nthe agreement, not either side of it.\n\nA process-inspection engine with signature, typosquat and masquerade checks. It correctly catches `svch0st.exe`\n\nimpersonating `svchost.exe`\n\n.\n\nIt also flagged `spoolsv.exe`\n\n, with a valid Microsoft signature from the\n\ncorrect System32 path.\n\nCause: the process library carries a note meaning \"heavy, watch resource\n\nuse\". That note was raising the **security** verdict.\n\nTwo different kinds of truth in one field.\n\nWhen one field carries two kinds of truth, one of them will eventually\n\nanswer a question it was never asked.\n\n| What happened | What the system reported |\n|---|---|\n| Settings never written | Applied successfully |\n| Temperature never read | All clear |\n| Learning never called | A confident answer |\n| Feature never connected | A working button |\n| Shortcut never valid | A shortcut on the desktop |\n| Advisory note misread | A security verdict |\n\n**A silent failure is a false claim of success.**\n\nNot an absence of output. A wrong output that happens to be reassuring.\n\nI work with an AI assistant and say so on every post. Normally that means a conversation:\n\nask, read properly, argue with a third of it, keep what survives.\n\nThen came a release week. Store submission, a version bump across 42 files, a build, a package, and day shifts behind a steering wheel.\n\nI started accepting more and reading less. In one week:\n\n**Plausible data written into a database.**\n\nThirty-five entries added to the known-process library.\n\nVendor fields looked entirely reasonable.\n\n`bash.exe`\n\nwas attributed to \"*Git Development Community*\".\n\n**Real Authenticode signer** is a person's name, and the engine compares expected vendor to actual signature, so a mismatch raises a warning.\n\nResult: **three processes that were merely unrecognised became flagged as\nsuspicious.** Confident, plausible, and worse than writing nothing.\n\n**A regex that passed tests and failed in production.**\n\n```\n# Passed every unit test. Never matched in the running app.\npos = text_widget.search(r'\\[-> [^\\]]+\\]', idx, regexp=True)\n```\n\nThe unit tests used Python's `re`\n\n. The widget's search is evaluated by Tcl, whose bracket expressions do not treat `[^\\]]`\n\nsame way.\n\n**Two engines, one string, no error message.**\n\nEvery link rendered as plain text.\n\nFix: search for a literal prefix, parse with the engine the pattern was\n\nwritten for.\n\n``` php\npos = text_widget.search('[-> ', idx)          # literal, engine-agnostic\nm = re.match(r'\\[-> ([^\\]]+)\\]', line_text)    # Python parses Python\n```\n\n**A git reset --hard that wiped a day of uncommitted work.**\n\n**A cleanup script that ate 38 commas.**\n\nA punctuation pass across 15 HTML files removed the comma after 38 closing tags. `\"Driver conflicts, leftover GPU packages\"`\n\nbecame `\"Driver conflictsleftover GPU packages\"`\n\n.\n\nNothing crashed. Every page rendered perfectly.\n\nNone of these threw an exception.\n\n**All of them produced output that looked right.**\n\nThis is not an argument against working this way. Project moves faster because of it. It is an argument about **where review has to happen**.\n\nGenerated code is fluent by construction: it compiles, reads well, uses the right function names.\n\n**Fluency is not correctness, and fluency is exactly what makes the difference invisible.**\n\n**Verify facts, never accept plausible ones.** If a value can be read\n\nfrom the system, read it. A plausible fact is more dangerous than a missing one, because a missing one gets checked.\n\n**Ask which engine actually runs this.** Before trusting a green test, ask whether it exercises the same runtime the user hits.\n\n**Never let a destructive command through unread.**\n\nAnything with `--hard`\n\n, `--force`\n\n, `rm`\n\n, `DROP`\n\nor `reset`\n\ngets read character by character. Back up first.\n\n**Check the output, not the exit code.** The comma script \"*succeeded*\".\n\nThe vendor entries \"*succeeded*\". All six bugs above \"*succeeded*\".\n\n**Write a ratchet the same day.** The fix is half the work.\n\nThe other half is a test that fails the build if it comes back.\n\nThey only turn one way.\n\nEvery bug here has one now.\n\n**Click the thing.** After a refactor that split one module into seven, 96\n\ntests stayed green while every sidebar page silently fell back to the\n\ndashboard.\n\nNot one test built the real window. Five minutes of human clicking caught what the whole suite could not.\n\n**Instrument the first divergence, not the damage at the end.**\n\nThree days on a replay bug, measuring how far apart two runs ended up. That number tells you the size of the damage and nothing else.\n\nLogging first tick where they stopped agreeing turned evenings into minutes.\n\n**Trust probes, not names.** Detecting a read-only install folder by checking whether the path contains `WindowsApps`\n\nis a guess about the world. A write probe is a fact about it.\n\nI build alone. No reviewer, nobody to ask\n\n\"did you check that it actually saved?\"\n\nThese bugs did not survive because they were subtle.\n\nSeveral were obvious.\n\nThey survived because exactly one person could have caught them, and that person had already decided the feature worked.\n\n**You do not write a test for a feature you already believe works.**\n\nThat is not a technical problem. It is the problem of being the only witness.\n\nI am 22, in Poland, self-taught after a technical school, and twelve projects died before this one.\n\nThe laptop most of it was built on is from 2014 and hits 94 degrees.\n\nThe day job has been a warehouse, then welding plastic, now\n\na taxi.\n\nYou do not do careful review at midnight after a twelve-hour shift.\n\n**You do the thing that feels finished.**\n\nEverything in the first half of this article is what \"feels finished\" looks like six months later.\n\nThree things help, and none is discipline: **write in public**\n\n(the difference between how you describe a feature and what it does is where these live), **ship to people who owe you nothing**\n\n(a tester refused to accept \"it works on my machine\" about a console that would not close, and he was right), and **keep a log** for the version of you in six months.\n\nWe are getting better at producing code that reads correctly and faster at\n\nproducing it. Neither makes code more likely to do what you meant.\n\n**Do not accept an outcome as proof of an action.** Not from your code, not from your tools, not from anything that generates text for you, and not from yourself at midnight.\n\nLook for the receipt.\n\n*I build PC Workman, a free Windows system monitor\nwith a fully offline assistant. 331 automated tests and a public list of\neverything above.*", "url": "https://wpnews.pro/news/the-bug-that-passes-every-test-and-does-nothing-i-let-an-ai-write-more-than-i-of", "canonical_source": "https://dev.to/huckler/the-bug-that-passes-every-test-and-does-nothing-i-let-an-ai-write-more-than-i-read-and-the-tests-228o", "published_at": "2026-08-10 10:27:11+00:00", "updated_at": "2026-08-10 10:47:25.792782+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools"], "entities": ["GitHub", "psutil", "Windows", "PCWorkman"], "alternates": {"html": "https://wpnews.pro/news/the-bug-that-passes-every-test-and-does-nothing-i-let-an-ai-write-more-than-i-of", "markdown": "https://wpnews.pro/news/the-bug-that-passes-every-test-and-does-nothing-i-let-an-ai-write-more-than-i-of.md", "text": "https://wpnews.pro/news/the-bug-that-passes-every-test-and-does-nothing-i-let-an-ai-write-more-than-i-of.txt", "jsonld": "https://wpnews.pro/news/the-bug-that-passes-every-test-and-does-nothing-i-let-an-ai-write-more-than-i-of.jsonld"}}