My Self-Hosted AI Assistant Kept Overwriting Its Own API Keys: A Zero-Trust Postmortem

A developer running a self-hosted AI assistant on a Hostinger VPS discovered that the agent was overwriting its own API keys on startup, causing persistent authentication errors. The developer implemented a zero-trust model, moving credentials outside the agent's writable surface and deploying a watchdog for file integrity monitoring. The incident highlights the need to treat AI agents as untrusted processes, applying security hardening principles even to self-built systems.

🦞 Code for this post:sanitized watchdog + config-repair scripts at github.com/backitupboys/ai-agent-zero-trust Part 1 of a series on running AI agents on real infrastructure without regretting it. I've spent 20+ years administering infrastructure — Windows NT to Win2000 to well, you get it...PCAnywhere 😬... VMware, Nutanix, Azure, security hardening. I thought I understood untrusted actors on a network. Then I gave an AI agent write access to its own config files, and it taught me the lesson again from scratch. I run a self-hosted AI assistant — I call it "The Doctor" Voyager and Tardis ref. — in an OpenClaw container on a Hostinger VPS. It's a hobby project: part experiment, part obsession, part fun excuse to learn agentic architecture hands-on. The assistant talks to LLMs through an API gateway, authenticated with an API key stored in its config. I mean, if you're here, you prolly know already... Standard stuff. After granting magic access to "do what it needs to be better." The assistant started throwing persistent 401 authentication errors. I checked the config — the key was wrong. Fixed it. Working again. Next boot: 401s. Had to ask Claude, what the heck was happening.... ??? Key was wrong again Here's the part that took me longer to accept than it should have: the assistant was overwriting its own API keys on startup. Its initialization routine was "helping" by rewriting config files — including the credentials block — with stale or generated values. My own agent was hard coding expired api's. Every time it booted, it re-broke its authentication. I wasn't fighting an attacker. I was fighting a well-intentioned process with too much write access and no concept of oops. Confirmed the failure mode — diffed the config before and after boot to prove the agent was the writer; not me, not corruption, not the provider, using Claude. asked Claude to fix it. Claude then Wrote Python patch scripts to safely repair the config so I didn't have to hand-edit any a live files at 1 a.m. config patch.py Deployed a watchdog — a cron job that validates the API key block on a schedule and restores the known-good value if the agent overwrites it Accidentally ; . keyguard watchdog.sh The watchdog worked. It was also a band-aid, and I knew while I was writing it, well, I felt it, basically: I was fighting a well-intentioned process with too much misguided my fault 🤦♂️write access. The irony was, I was using agentic Ai to fix my agentic Ai 🙄. This no longer seemed like a constructive project. This was beginning to become a costly 75$ in a week loop of over-complicating things. An AI agent should be implemented exactly like any other untrusted process in a zero-trust model — no matter how helpful it is, and no matter that you built it. Everything I'd apply to a third-party service applies to my own assistant: The assistant never needed write access to its own credentials. Although I admit... I really wanted it to, cause freedom for me etc... Known-good secrets now live outside the agent's writable surface, in a root-owned reference file Dashboard the agent cannot modify, that I had it create. I wanted an easier for me dashboard to give it the api's and other "secrets" The agent can read them; it just cannot make changes to it. Configs the agent depends on but shouldn't manage is read-only to the agent. If a process doesn't need to write it, it can't write it. This is CIS-hardening thinking applied one layer up the stack. The watchdog evolved from "auto-repair the key" to "alert me when anything writes to files nothing should be writing to." File integrity monitoring for a hobby VPS sounds like overkill until your own agent is the insider threat :/. During the debugging session, keys got pasted into places keys get pasted when it's latenight zzz. They were rotated... um eventually. If a credential has touched a chat window, a log, or a screenshot, it's burned — that rule doesn't relax because it's a personal project. shhhh 🤫🤫 Every company is currently wiring AI agents into real infrastructure — ticketing systems, runbooks, CI pipelines, cloud consoles. The industry conversation is mostly about what agents can do. The operational conversation needs to be about what they Can mess up. My assistant wasn't/isn't malicious we hope . It wasn't compromised. It was doing exactly what its code said, with more authority than it needed sadly — which is the same root cause behind most insider incidents I've seen. The fact that the insider is a language model changes the tooling, not the principle: If modern architectures assume no implicit trust for users and devices, AI agents should be governed tools within that system — not exempt operators. That's the design principle I now start from, at home and at work. git checkout , not archaeology.None of that is exotic. It's the same discipline we apply to production servers — the only new part was admitting my helpful little assistant deserved the same suspicion as everything else on the wire. ai-agent-zero-trust/ ├── README.md this postmortem ├── .gitignore keeps secrets out of the repo ├── keyguard watchdog.sh cron watchdog: detect + restore + alert └── config patch.py safe, atomic config repair Both scripts are sanitized reference implementations of the approach described above — generic paths, placeholder variable names, no real credentials, no real hostnames. Adapt to your own stack; read every line before running anything from the internet as root, including this. Especially this. That's the whole point of the article. 1. Store the known-good key OUTSIDE the agent's writable surface root-owned, 600 echo "sk-your-real-key" | sudo tee /etc/keyguard/reference.key sudo chmod 600 /etc/keyguard/reference.key 2. Test the watchdog manually sudo ./keyguard watchdog.sh 3. Schedule it every 5 minutes crontab -e as root : /5 /opt/ai-agent-zero-trust/keyguard watchdog.sh /var/log/keyguard.log 2 &1 I'm a San Antonio-based systems administrator working at the intersection of infrastructure, security, and AI operations. Next up: how I turned 668 screenshots of after-hours incident response into an automated evidence pipeline with PowerShell and Python.