# I Let an AI Audit My Password Vault. It Lied to Me With Total Confidence. 🔐

> Source: <https://dev.to/akhourianmolkumar/i-let-an-ai-audit-my-password-vault-it-lied-to-me-with-total-confidence-2b22>
> Published: 2026-10-04 04:49:35+00:00

That's how it started.

I'd been building **ATLOCK**, a Windows security suite with an encrypted password vault, file guard, 2FA, the works. v5 was nearly done. So I did what every developer does at some point: I pasted 8,500 lines of Python into an AI and asked,

*"Any bugs?"*

It came back with **six**. Emojis, severity colors, function names, even a smug little *"I didn't invent these, you can verify them yourself."*

Reader, I took that offer literally. 😏

For every claim I did one thing: **open the actual file and check.**

No vibes. No trust. Just `grep` and line numbers.

Final score:

| # | Claim | Verdict | 
|---|---|---|
| 1 | Vault recovery key crashes | 🔴 **Real** | 
| 2 | Watchdog silently skips services | 🤥 **Fiction** | 
| 3 | Enabling 2FA locks you out | 🟠 **Real** | 
| 4 | HIBP request on every keystroke | 🟡 **Real** | 
| 5 | Wrong key derivation for v2 vaults | 😇 **Real, but harmless** | 
| 6 | Factory reset race condition | 🟡 **Real** | 

Four real bugs, one fake, one harmless. Let's go through them. 🍿

Creating a **Vault Recovery Key** called this:

```
enc_b64 = base64.b64encode(bytes(self._sess.view())).decode("ascii")
```

And `SecureBuffer` looks like this:

``` php
class SecureBuffer:
    def get_bytes(self) -> bytes: ...
    def wipe(self): ...
    # view()? Never heard of her.
```

Click *"Create Vault Recovery Key"* → `AttributeError`. 💀

It's the one feature meant to save you when you forget your password, and it crashed on creation.

**Fix:** one line.

```
enc_b64 = base64.b64encode(self._sess.get_bytes()).decode("ascii")
```

Lesson: a recovery feature nobody tests is a recovery feature that doesn't exist.

This one is my favorite, because it's *sneaky*.

When you enable TOTP in Settings, the app creates a **fresh** `PasswordVault()`, writes the 2FA config to disk, and shows you a lovely QR code. Everything looks perfect.

But the main app is holding a **different** vault object, loaded once at startup, with a memory that has no idea 2FA now exists.

So you lock the vault, try to unlock it, and your own app says:

*"TOTP is required by your security policy but not set up. Access denied."* 🙃

Yes. The policy says *"2FA required"*, the stale in-memory copy says *"2FA doesn't exist"*, and the vault, being a good fail-closed security tool, slams the door on you.

Funny part: it's the *safe* kind of bug. Annoying, but it never leaked anything. A password manager that locks you out beats one that lets everyone in. 😅

**Fix:** re-read the 2FA config from disk right before checking the second factor.

``` python
def _sync_mfa_from_disk(self):
    with contextlib.suppress(Exception):
        disk = atlock_read_json(self._path, None)
        if isinstance(disk, dict):
            if isinstance(disk.get("mfa"), dict): self._db["mfa"] = disk["mfa"]
            else: self._db.pop("mfa", None)

# in unlock(), right before the 2FA check:
self._sync_mfa_from_disk()
ok2, msg = atl_second_factor(self._db, master, "Vault unlock", ask_code=self.ask_code)
```

Bonus: this also covers someone enabling 2FA from the CLI while the app is open.

When you add a vault entry, ATLOCK checks your password against Have I Been Pwned. Great feature. Terrible implementation:

```
self._pw_e.bind("<KeyRelease>", self._on_password_change)  # every. single. key.
```

Type a 12-character password and the app spawns about **7 threads and 7 API calls**. Rate limits and UI lag, free of charge.

**Fix:** a classic debounce. Wait until the user *stops typing* for 700 ms, then check once.

```
self._hibp_after = self.after(700, lambda: self._run_hibp(pw))
```

If the password changed while the request was in flight, the stale result is thrown away. No more flickering "✓ Safe" labels for passwords you've already deleted.

Factory Reset worked like this:

The problem: step 1 happens *while the old app still holds its file handles*. On Windows, deleting a file another process still has open gives you `PermissionError`, and you get a **half-erased** reset. Spooky. 👻

**Fix:** the reset process now receives the parent's PID and politely waits for it to exit.

```
subprocess.Popen([sys.executable, sys.argv[0], "--reset",
                  f"--reset-wait-pid={os.getpid()}"])
```

On Windows it waits on the process handle (up to 10 seconds) before wiping anything. Manners matter, even in a destructive operation.

The claim: Quick Lock always derives the master key with the *legacy v1* function, even for v2 vaults.

True in the code. But I traced who can ever reach it: **TOTP and FIDO2 setup require a v3 vault.** A legacy v2 vault can never have a second factor, so there's nothing to unwrap with the wrong key.

So: technically inconsistent, practically unreachable. I left it alone. Knowing *what not to fix* is also engineering. 🧘

The most dramatic claim:

*"Watchdog calls `DefenderGuard.run_ps`, but the method is `_run_ps`! It silently skips services and scheduled tasks! A big hole in persistence detection!"*

It even offered *proof*: *"the WMI function spells it correctly, so the other two are obviously typos."*

I searched the entire file:

```
grep -n "DefenderGuard\.run_ps" ATLOCK_V5.py
# (nothing)
```

Zero results. Every call was already `_run_ps`. The bug was **invented**, complete with a convincing explanation of how it happened and why it matters.

And the best part? I fed the *fixed* file back to the AI, and it confidently re-confirmed all the old bugs, including the fake one, **even the ones I'd already fixed.** It even reproduced the typo with a stray space (`DefenderGuard. run_ps`) that exists nowhere in my code. 🤡

**1. AI code review is a great *scout*, and a terrible *judge*.**

It found four real bugs I'd missed. That's genuinely valuable. It also invented one with the same confident voice. You can't tell them apart without checking.

**2. Demand line numbers.**

A real bug can be pointed at. If a claim has no line number, it's a rumor.

**3. `grep` is a lie detector.**

Every claim here took about ten seconds to verify. Ten seconds beats a week of "fixing" a bug that doesn't exist.

**4. Trace before you fix.**

Bug #5 was real and still not worth a patch. Reachability matters more than correctness-in-the-abstract.

**5. Fail-closed bugs are the good kind of bad.**

When your security tool breaks, it should break *shut*.

**ATLOCK** is a Windows security suite by **Akhouri Systems**: an Argon2id-based encrypted password vault, file guard, TOTP and FIDO2 2FA, panic/decoy vault, breach checks, startup watchdog, and more.

That's **33 downloads** away. If you've ever wanted to be the reason a security tool ships its next version, you know what to do. 😄

⭐ Repo: [github.com/Akhouri-Anmol-Kumar/ATLOCK](https://github.com/Akhouri-Anmol-Kumar/ATLOCK)

Has an AI ever confidently "found" a bug that didn't exist in your code? Drop your best hallucination story below. 👇 I'll start a collection.

*Built by Akhouri Anmol Kumar, one very suspicious `grep` at a time.* 🔍