That's how it started.
I'd been building ATLOCK, a Windows security suite with an encrypted password vault, file guard, 2FA, the works. v5 was nearly done. So I did what every developer does at some point: I pasted 8,500 lines of Python into an AI and asked,
"Any bugs?"
It came back with six. Emojis, severity colors, function names, even a smug little "I didn't invent these, you can verify them yourself."
Reader, I took that offer literally. 😏
For every claim I did one thing: open the actual file and check.
No vibes. No trust. Just grep and line numbers.
Final score:
| # | Claim | Verdict |
|---|---|---|
| 1 | Vault recovery key crashes | 🔴 Real |
| 2 | Watchdog silently skips services | 🤥 Fiction |
| 3 | Enabling 2FA locks you out | 🟠 Real |
| 4 | HIBP request on every keystroke | 🟡 Real |
| 5 | Wrong key derivation for v2 vaults | 😇 Real, but harmless |
| 6 | Factory reset race condition | 🟡 Real |
Four real bugs, one fake, one harmless. Let's go through them. 🍿
Creating a Vault Recovery Key called this:
enc_b64 = base64.b64encode(bytes(self._sess.view())).decode("ascii")
And SecureBuffer looks like this:
class SecureBuffer:
def get_bytes(self) -> bytes: ...
def wipe(self): ...
Click "Create Vault Recovery Key" → AttributeError. 💀
It's the one feature meant to save you when you forget your password, and it crashed on creation.
Fix: one line.
enc_b64 = base64.b64encode(self._sess.get_bytes()).decode("ascii")
Lesson: a recovery feature nobody tests is a recovery feature that doesn't exist.
This one is my favorite, because it's sneaky.
When you enable TOTP in Settings, the app creates a fresh PasswordVault(), writes the 2FA config to disk, and shows you a lovely QR code. Everything looks perfect.
But the main app is holding a different vault object, loaded once at startup, with a memory that has no idea 2FA now exists.
So you lock the vault, try to unlock it, and your own app says:
"TOTP is required by your security policy but not set up. Access denied." 🙃
Yes. The policy says "2FA required", the stale in-memory copy says "2FA doesn't exist", and the vault, being a good fail-closed security tool, slams the door on you.
Funny part: it's the safe kind of bug. Annoying, but it never leaked anything. A password manager that locks you out beats one that lets everyone in. 😅
Fix: re-read the 2FA config from disk right before checking the second factor.
def _sync_mfa_from_disk(self):
with contextlib.suppress(Exception):
disk = atlock_read_json(self._path, None)
if isinstance(disk, dict):
if isinstance(disk.get("mfa"), dict): self._db["mfa"] = disk["mfa"]
else: self._db.pop("mfa", None)
self._sync_mfa_from_disk()
ok2, msg = atl_second_factor(self._db, master, "Vault unlock", ask_code=self.ask_code)
Bonus: this also covers someone enabling 2FA from the CLI while the app is open.
When you add a vault entry, ATLOCK checks your password against Have I Been Pwned. Great feature. Terrible implementation:
self._pw_e.bind("<KeyRelease>", self._on_password_change) # every. single. key.
Type a 12-character password and the app spawns about 7 threads and 7 API calls. Rate limits and UI lag, free of charge.
Fix: a classic debounce. Wait until the user stops typing for 700 ms, then check once.
self._hibp_after = self.after(700, lambda: self._run_hibp(pw))
If the password changed while the request was in flight, the stale result is thrown away. No more flickering "✓ Safe" labels for passwords you've already deleted.
Factory Reset worked like this:
The problem: step 1 happens while the old app still holds its file handles. On Windows, deleting a file another process still has open gives you PermissionError, and you get a half-erased reset. Spooky. 👻
Fix: the reset process now receives the parent's PID and politely waits for it to exit.
subprocess.Popen([sys.executable, sys.argv[0], "--reset",
f"--reset-wait-pid={os.getpid()}"])
On Windows it waits on the process handle (up to 10 seconds) before wiping anything. Manners matter, even in a destructive operation.
The claim: Quick Lock always derives the master key with the legacy v1 function, even for v2 vaults.
True in the code. But I traced who can ever reach it: TOTP and FIDO2 setup require a v3 vault. A legacy v2 vault can never have a second factor, so there's nothing to unwrap with the wrong key.
So: technically inconsistent, practically unreachable. I left it alone. Knowing what not to fix is also engineering. 🧘
The most dramatic claim:
"Watchdog calls DefenderGuard.run_ps, but the method is _run_ps! It silently skips services and scheduled tasks! A big hole in persistence detection!"
It even offered proof: "the WMI function spells it correctly, so the other two are obviously typos."
I searched the entire file:
grep -n "DefenderGuard\.run_ps" ATLOCK_V5.py
Zero results. Every call was already _run_ps. The bug was invented, complete with a convincing explanation of how it happened and why it matters.
And the best part? I fed the fixed file back to the AI, and it confidently re-confirmed all the old bugs, including the fake one, even the ones I'd already fixed. It even reproduced the typo with a stray space (DefenderGuard. run_ps) that exists nowhere in my code. 🤡
1. AI code review is a great scout, and a terrible judge.
It found four real bugs I'd missed. That's genuinely valuable. It also invented one with the same confident voice. You can't tell them apart without checking.
2. Demand line numbers.
A real bug can be pointed at. If a claim has no line number, it's a rumor.
3. grep is a lie detector.
Every claim here took about ten seconds to verify. Ten seconds beats a week of "fixing" a bug that doesn't exist.
4. Trace before you fix.
Bug #5 was real and still not worth a patch. Reachability matters more than correctness-in-the-abstract.
5. Fail-closed bugs are the good kind of bad.
When your security tool breaks, it should break shut.
ATLOCK is a Windows security suite by Akhouri Systems: an Argon2id-based encrypted password vault, file guard, TOTP and FIDO2 2FA, panic/decoy vault, breach checks, startup watchdog, and more.
That's 33 downloads away. If you've ever wanted to be the reason a security tool ships its next version, you know what to do. 😄
⭐ Repo: github.com/Akhouri-Anmol-Kumar/ATLOCK
Has an AI ever confidently "found" a bug that didn't exist in your code? Drop your best hallucination story below. 👇 I'll start a collection.
Built by Akhouri Anmol Kumar, one very suspicious grep at a time. 🔍