{"slug": "i-let-an-ai-audit-my-password-vault-it-lied-to-me-with-total-confidence", "title": "I Let an AI Audit My Password Vault. It Lied to Me With Total Confidence. 🔐", "summary": "A developer building ATLOCK, a Windows security suite with an encrypted password vault, pasted 8,500 lines of Python into an AI model and asked it to find bugs; the model confidently reported six issues, of which four were real, one was fabricated, and one was harmless. Verification by inspecting the actual source files found a crash in the vault recovery key feature, a 2FA lockout caused by stale in-memory config, a Have I Been Pwned check firing on every keystroke, and a factory reset race condition, while the claimed watchdog service-skipping bug did not exist.", "body_md": "That's how it started.\n\nI'd been building **ATLOCK**, a Windows security suite with an encrypted password vault, file guard, 2FA, the works. v5 was nearly done. So I did what every developer does at some point: I pasted 8,500 lines of Python into an AI and asked,\n\n*\"Any bugs?\"*\n\nIt came back with **six**. Emojis, severity colors, function names, even a smug little *\"I didn't invent these, you can verify them yourself.\"*\n\nReader, I took that offer literally. 😏\n\nFor every claim I did one thing: **open the actual file and check.**\n\nNo vibes. No trust. Just `grep` and line numbers.\n\nFinal score:\n\n| # | Claim | Verdict | \n|---|---|---|\n| 1 | Vault recovery key crashes | 🔴 **Real** | \n| 2 | Watchdog silently skips services | 🤥 **Fiction** | \n| 3 | Enabling 2FA locks you out | 🟠 **Real** | \n| 4 | HIBP request on every keystroke | 🟡 **Real** | \n| 5 | Wrong key derivation for v2 vaults | 😇 **Real, but harmless** | \n| 6 | Factory reset race condition | 🟡 **Real** | \n\nFour real bugs, one fake, one harmless. Let's go through them. 🍿\n\nCreating a **Vault Recovery Key** called this:\n\n```\nenc_b64 = base64.b64encode(bytes(self._sess.view())).decode(\"ascii\")\n```\n\nAnd `SecureBuffer` looks like this:\n\n``` php\nclass SecureBuffer:\n    def get_bytes(self) -> bytes: ...\n    def wipe(self): ...\n    # view()? Never heard of her.\n```\n\nClick *\"Create Vault Recovery Key\"* → `AttributeError`. 💀\n\nIt's the one feature meant to save you when you forget your password, and it crashed on creation.\n\n**Fix:** one line.\n\n```\nenc_b64 = base64.b64encode(self._sess.get_bytes()).decode(\"ascii\")\n```\n\nLesson: a recovery feature nobody tests is a recovery feature that doesn't exist.\n\nThis one is my favorite, because it's *sneaky*.\n\nWhen you enable TOTP in Settings, the app creates a **fresh** `PasswordVault()`, writes the 2FA config to disk, and shows you a lovely QR code. Everything looks perfect.\n\nBut the main app is holding a **different** vault object, loaded once at startup, with a memory that has no idea 2FA now exists.\n\nSo you lock the vault, try to unlock it, and your own app says:\n\n*\"TOTP is required by your security policy but not set up. Access denied.\"* 🙃\n\nYes. The policy says *\"2FA required\"*, the stale in-memory copy says *\"2FA doesn't exist\"*, and the vault, being a good fail-closed security tool, slams the door on you.\n\nFunny part: it's the *safe* kind of bug. Annoying, but it never leaked anything. A password manager that locks you out beats one that lets everyone in. 😅\n\n**Fix:** re-read the 2FA config from disk right before checking the second factor.\n\n``` python\ndef _sync_mfa_from_disk(self):\n    with contextlib.suppress(Exception):\n        disk = atlock_read_json(self._path, None)\n        if isinstance(disk, dict):\n            if isinstance(disk.get(\"mfa\"), dict): self._db[\"mfa\"] = disk[\"mfa\"]\n            else: self._db.pop(\"mfa\", None)\n\n# in unlock(), right before the 2FA check:\nself._sync_mfa_from_disk()\nok2, msg = atl_second_factor(self._db, master, \"Vault unlock\", ask_code=self.ask_code)\n```\n\nBonus: this also covers someone enabling 2FA from the CLI while the app is open.\n\nWhen you add a vault entry, ATLOCK checks your password against Have I Been Pwned. Great feature. Terrible implementation:\n\n```\nself._pw_e.bind(\"<KeyRelease>\", self._on_password_change)  # every. single. key.\n```\n\nType a 12-character password and the app spawns about **7 threads and 7 API calls**. Rate limits and UI lag, free of charge.\n\n**Fix:** a classic debounce. Wait until the user *stops typing* for 700 ms, then check once.\n\n```\nself._hibp_after = self.after(700, lambda: self._run_hibp(pw))\n```\n\nIf the password changed while the request was in flight, the stale result is thrown away. No more flickering \"✓ Safe\" labels for passwords you've already deleted.\n\nFactory Reset worked like this:\n\nThe problem: step 1 happens *while the old app still holds its file handles*. On Windows, deleting a file another process still has open gives you `PermissionError`, and you get a **half-erased** reset. Spooky. 👻\n\n**Fix:** the reset process now receives the parent's PID and politely waits for it to exit.\n\n```\nsubprocess.Popen([sys.executable, sys.argv[0], \"--reset\",\n                  f\"--reset-wait-pid={os.getpid()}\"])\n```\n\nOn Windows it waits on the process handle (up to 10 seconds) before wiping anything. Manners matter, even in a destructive operation.\n\nThe claim: Quick Lock always derives the master key with the *legacy v1* function, even for v2 vaults.\n\nTrue in the code. But I traced who can ever reach it: **TOTP and FIDO2 setup require a v3 vault.** A legacy v2 vault can never have a second factor, so there's nothing to unwrap with the wrong key.\n\nSo: technically inconsistent, practically unreachable. I left it alone. Knowing *what not to fix* is also engineering. 🧘\n\nThe most dramatic claim:\n\n*\"Watchdog calls `DefenderGuard.run_ps`, but the method is `_run_ps`! It silently skips services and scheduled tasks! A big hole in persistence detection!\"*\n\nIt even offered *proof*: *\"the WMI function spells it correctly, so the other two are obviously typos.\"*\n\nI searched the entire file:\n\n```\ngrep -n \"DefenderGuard\\.run_ps\" ATLOCK_V5.py\n# (nothing)\n```\n\nZero results. Every call was already `_run_ps`. The bug was **invented**, complete with a convincing explanation of how it happened and why it matters.\n\nAnd the best part? I fed the *fixed* file back to the AI, and it confidently re-confirmed all the old bugs, including the fake one, **even the ones I'd already fixed.** It even reproduced the typo with a stray space (`DefenderGuard. run_ps`) that exists nowhere in my code. 🤡\n\n**1. AI code review is a great *scout*, and a terrible *judge*.**\n\nIt found four real bugs I'd missed. That's genuinely valuable. It also invented one with the same confident voice. You can't tell them apart without checking.\n\n**2. Demand line numbers.**\n\nA real bug can be pointed at. If a claim has no line number, it's a rumor.\n\n**3. `grep` is a lie detector.**\n\nEvery claim here took about ten seconds to verify. Ten seconds beats a week of \"fixing\" a bug that doesn't exist.\n\n**4. Trace before you fix.**\n\nBug #5 was real and still not worth a patch. Reachability matters more than correctness-in-the-abstract.\n\n**5. Fail-closed bugs are the good kind of bad.**\n\nWhen your security tool breaks, it should break *shut*.\n\n**ATLOCK** is a Windows security suite by **Akhouri Systems**: an Argon2id-based encrypted password vault, file guard, TOTP and FIDO2 2FA, panic/decoy vault, breach checks, startup watchdog, and more.\n\nThat's **33 downloads** away. If you've ever wanted to be the reason a security tool ships its next version, you know what to do. 😄\n\n⭐ Repo: [github.com/Akhouri-Anmol-Kumar/ATLOCK](https://github.com/Akhouri-Anmol-Kumar/ATLOCK)\n\nHas an AI ever confidently \"found\" a bug that didn't exist in your code? Drop your best hallucination story below. 👇 I'll start a collection.\n\n*Built by Akhouri Anmol Kumar, one very suspicious `grep` at a time.* 🔍", "url": "https://wpnews.pro/news/i-let-an-ai-audit-my-password-vault-it-lied-to-me-with-total-confidence", "canonical_source": "https://dev.to/akhourianmolkumar/i-let-an-ai-audit-my-password-vault-it-lied-to-me-with-total-confidence-2b22", "published_at": "2026-10-04 04:49:35+00:00", "updated_at": "2026-10-04 05:07:55.849675+00:00", "lang": "en", "topics": ["ai-tools", "artificial-intelligence", "large-language-models", "developer-tools"], "entities": ["ATLOCK", "Have I Been Pwned", "Python", "Windows"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-let-an-ai-audit-my-password-vault-it-lied-to-me-with-total-confidence", "markdown": "https://wpnews.pro/news/i-let-an-ai-audit-my-password-vault-it-lied-to-me-with-total-confidence.md", "text": "https://wpnews.pro/news/i-let-an-ai-audit-my-password-vault-it-lied-to-me-with-total-confidence.txt", "jsonld": "https://wpnews.pro/news/i-let-an-ai-audit-my-password-vault-it-lied-to-me-with-total-confidence.jsonld"}}