{"slug": "my-paranoid-coding-machine-setup", "title": "My Paranoid Coding Machine Setup", "summary": "Brendan Long moved all of his Claude Code AI agents into an unprivileged VM running under an unprivileged user account, using rootless Podman instead of Docker because any user who can run Docker can easily gain root access inside the VM. For GPU access, the agents SSH into an unprivileged gVisor plus nvproxy container through Long's GPU queueing tool, which allowlists Nvidia driver calls to reduce the driver-exploit attack surface. Long also had Claude rewrite the C code of his locally hosted Kokoro-82M text-to-speech service in Rust and sandboxed it with GPU access, keeping the only internet-facing port, exposed via Cloudflare Tunnel, free of privileges and GPU access.", "body_md": "The combination of running AI agents which can install and run anything with supply chain attacks / [the Hackening](https://www.lesswrong.com/posts/iqgZaJMnmyDCYfnuC/the-hackening-1)<sup>[6](#fn:6)</sup> has made me much more paranoid about my personal machine than I used to be. I’ve been slowly improving this, and finally moved all of my AI agents into a VM today. I’m sharing this to show how far you can take this, even if you have requirements like GPU access or the ability to run (something close enough to) Docker.\n\n*This is a follow-up / longer version of* *a recent shortform*[7](#fn:7)*.*\n\n## Goals[#](#goals)\n\nI run Claude Code in [a custom harness](https://github.com/brendanlong/clawed-abode)<sup>[8](#fn:8)</sup> which currently disables all permission prompts. I’m not really that worried about Claude doing something dangerous on purpose, but I worry about:\n\n- Claude breaking something by accident (mistyping a command that turns into `rm -rf /` ).\n- Claude installing malware (most likely via a supply chain attack).\n\nMy biggest worry is kernel exploits giving attackers access to my user account, allowing them to install a keylogger and steal my password manager vault<sup>[1](#fn:1)</sup>.\n\nClaude needs the ability to run Docker for testing, and shared access to my GPU for ML work.\n\n## Non-Goals[#](#non-goals)\n\nTo get its work done, Claude needs access to many API keys (GitHub, OpenRouter, etc.). These keys are already as limited as they can be, and I limit the amount of API credits each key can access anyway because Claude doesn’t understand the value of money.\n\n## Agent VM[#](#agent-vm)\n\nI moved Claude agents to an unprivileged user account inside of an unprivileged VM. Any user that can run Docker can easily get root access (and root access within the VM makes sandbox escape much easier) so agents use [rootless Podman](https://developers.redhat.com/blog/2020/09/25/rootless-containers-with-podman-the-basics)<sup>[9](#fn:9)</sup> instead.\n\nGPU drivers allow direct access to hardware, and assigning the GPU to a VM would prevent my computer from being used by humans, so the VM can’t access the GPU directly.\n\n## GPU Container[#](#gpu-container)\n\nFor GPU usage, agents use [my GPU queueing tool](https://www.brendanlong.com/gpuc-a-single-user-gpu-queue-that-doesnt-require-sudo.html)<sup>[12](#fn:12)</sup> over SSH into an unprivileged container using [gVisor](https://gvisor.dev/)<sup>[13](#fn:13)</sup> + nvproxy. This has an allowlist for Nvidia driver calls, which [reduces the attack surface](https://gvisor.dev/docs/user_guide/gpu/#nvproxy-security)<sup>[14](#fn:14)</sup> for driver exploits (although imperfectly).\n\n## Production TTS Containers[#](#production-tts-containers)\n\nI run [a Text-to-Speech service for my RSS reader](https://lionreader.com/demo/all?entry=text-to-speech)<sup>[15](#fn:15)</sup> on this machine, since everything commercially available is either weirdly slow<sup>[2](#fn:2)</sup> or ridiculously expensive. I decided to run [Kokoro-82M](https://huggingface.co/hexgrad/Kokoro-82M)<sup>[16](#fn:16)</sup> locally, but didn’t want to expose a service containing C code<sup>[3](#fn:3)</sup> to the internet. I had Claude [rewrite the C code in Rust](https://github.com/brendanlong/optimized-kokoro)<sup>[17](#fn:17)</sup>, and (just in case) heavily sandbox the code with GPU access:\n\nThe only port opened to the internet (via [Cloudflare Tunnel](https://developers.cloudflare.com/tunnel/)[18](#fn:18)<sup>[4](#fn:4)</sup>) is written entirely in Rust and has no privileges or GPU access. It talks to a separate container with GPU access (through gVisor + nvproxy, like my other GPU container) over a very simple binary protocol where the only inputs are tokenized phonemes (a list of numbers).\n\nI’m still unsure if this piece is sufficiently paranoid, but I really like having fast TTS and don’t want to spend $200+/mo for a cloud GPU (or to send my users’ data to [other people’s gaming machines](https://vast.ai/)<sup>[19](#fn:19)</sup>).\n\n## Gotchas[#](#gotchas)\n\n### Segfaults in the agent VM[#](#segfaults-in-the-agent-vm)\n\nThis may have something to do with faulty hardware on my machine<sup>[5](#fn:5)</sup>, but switching from `host-passthrough` to `host-model` in libvirt fixed it for me.\n\n### [File descriptor leak in gVisor](https://github.com/google/gvisor/pull/17011)<sup>[20](#fn:20)</sup> with `directfs=false`[#](#file-descriptor-leak-in-gvisor-with-directfsfalse)\n\n[20](#fn:20)\n\nI was using this to avoid running the sandbox as root. Moving the GPU container to rootless Podman with directfs worked instead.\n\n## Other Security Hardening[#](#other-security-hardening)\n\n- Kernel updates are applied without restarting via [Ubuntu Pro](https://ubuntu.com/pro)<sup>[21](#fn:21)</sup> /[Livepatch](https://ubuntu.com/security/livepatch)<sup>[22](#fn:22)</sup> (free for up to 5 machines, although the agent VM counts for one).\n- Secure Boot is enabled, using Canonical’s signed driver modules.\n- Several unused kernel modules are disabled, especially network protocols that I don’t use (`sctp` ,`rds` ,`tipc` and`n_hdlc` ).\n- I removed my own account from the `docker` group to limit malware’s impact on my own account, although a keylogger could still get past this.\n- A firewall blocks all inbound requests except via Tailscale, and my Tailscale policy is fairly locked-down now: My personal devices can access each other via advertised services and (key-only) SSH; the agent VM can only access the TTS service.\n- Package managers proxy installs through [Socket Firewall](https://socket.dev/features/firewall)<sup>[23](#fn:23)</sup> (free), which warns or blocks known malware in packages.\n\n## Next Steps[#](#next-steps)\n\nThe biggest remaining security improvement for me is probably [running every application in a separate VM](https://www.qubes-os.org/)<sup>[24](#fn:24)</sup>, or at least running my browser in a VM (although I already use [the Snap version of Firefox](https://snapcraft.io/firefox)<sup>[25](#fn:25)</sup>, which does [some sandboxing](https://snapcraft.io/docs/explanation/security/security-policies/)<sup>[26](#fn:26)</sup>).\n\nI’m also slightly worried about Canonical or another vendor I depend on getting hacked and putting malware in the official repos. I haven’t figured out how to mitigate this yet.\n\n## Summary[#](#summary)\n\nIf you’re sufficiently paranoid, it’s possible to have AI agents running on your machine while sandboxing them to a pretty acceptable level, even if they need GPU access and tools like Docker.\n\n1. I intentionally don’t keep 2FA keys in my password manager vault, so this wouldn’t be entirely catastrophic, but most of my accounts don’t use 2FA and it would still be very annoying to change hundreds of passwords. [↩](#fnref:1)\n2. [OpenRouter claims that Kokoro-82M’s latency is <300 ms](https://openrouter.ai/hexgrad/kokoro-82m#performance)<sup>[27](#fn:27)</sup> , but in practice I was seeing several seconds, especially for longer inputs. My local machine using a low-end previous generation GPU responds in ~50 ms and processes at 200× realtime.[↩](#fnref:2)\n3. [espeak-ng](https://github.com/espeak-ng/espeak-ng)<sup>[28](#fn:28)</sup> CVEs that haven’t been fixed in a released version:[CVE-2023-49990](https://nvd.nist.gov/vuln/detail/CVE-2023-49990)<sup>[29](#fn:29)</sup> ,[CVE-2023-49991](https://nvd.nist.gov/vuln/detail/CVE-2023-49991)<sup>[30](#fn:30)</sup> ,[CVE-2023-49992](https://nvd.nist.gov/vuln/detail/CVE-2023-49992)<sup>[31](#fn:31)</sup> ,[CVE-2023-49993](https://nvd.nist.gov/vuln/detail/CVE-2023-49993)<sup>[32](#fn:32)</sup> .[↩](#fnref:3)\n4. I chose Cloudflare Tunnel over Tailscale Funnel because it adds additional protections in front of your service, including an API key handled by Cloudflare before hitting a second API key handled by my service. [↩](#fnref:4)\n5. My CPU is known to be flaky in some situations. [↩](#fnref:5)\n6. https://www.lesswrong.com/posts/iqgZaJMnmyDCYfnuC/the-hackening-1 - “The Hackening — LessWrong”\n7. https://www.lesswrong.com/posts/kS5efcTP86EDTDbGh/brendan-long-s-shortform?commentId=wb8i468tNgdmmKqhY - “Brendan Long’s Shortform — LessWrong”\n8. https://github.com/brendanlong/clawed-abode - “GitHub: brendanlong/clawed-abode”\n9. https://developers.redhat.com/blog/2020/09/25/rootless-containers-with-podman-the-basics - “Rootless containers with Podman: The basics | Red Hat Developer”\n10. https://tailscale.com/ - “Tailscale | Secure Connectivity for AI, IoT & Multi-Cloud”\n11. https://code.visualstudio.com/docs/remote/vscode-web - “Visual Studio Code for the Web”\n12. https://www.brendanlong.com/gpuc-a-single-user-gpu-queue-that-doesnt-require-sudo.html - “gpuc: A single-user GPU queue that doesn’t require sudo”\n13. https://gvisor.dev/ - “The Container Security Platform - gVisor”\n14. https://gvisor.dev/docs/user_guide/gpu/#nvproxy-security - “GPU Support - gVisor”\n15. https://lionreader.com/demo/all?entry=text-to-speech - “Text-to-Speech Narration - Lion Reader”\n16. https://huggingface.co/hexgrad/Kokoro-82M - “hexgrad/Kokoro-82M · Hugging Face”\n17. https://github.com/brendanlong/optimized-kokoro - “GitHub: brendanlong/optimized-kokoro”\n18. https://developers.cloudflare.com/tunnel/ - “Cloudflare Tunnel · Cloudflare Docs”\n19. https://vast.ai/ - “Rent GPUs | Vast.ai”\n20. https://github.com/google/gvisor/pull/17011 - “GitHub: google/gvisor #17011”\n21. https://ubuntu.com/pro - “Ubuntu Pro | Ubuntu”\n22. https://ubuntu.com/security/livepatch - “Livepatch | Security | Ubuntu”\n23. https://socket.dev/features/firewall - “Socket Firewall”\n24. https://www.qubes-os.org/ - “Qubes OS: A reasonably secure operating system | Qubes OS”\n25. https://snapcraft.io/firefox - “Install firefox on Linux | Snap Store”\n26. https://snapcraft.io/docs/explanation/security/security-policies/ - “Security policies - Snap documentation”\n27. https://openrouter.ai/hexgrad/kokoro-82m#performance - “Kokoro 82M - Text-to-Speech API Pricing | OpenRouter”\n28. https://github.com/espeak-ng/espeak-ng - “GitHub: espeak-ng/espeak-ng”\n29. https://nvd.nist.gov/vuln/detail/CVE-2023-49990 - “NVD - Home”\n30. https://nvd.nist.gov/vuln/detail/CVE-2023-49991 - “NVD - Home”\n31. https://nvd.nist.gov/vuln/detail/CVE-2023-49992 - “NVD - Home”\n32. https://nvd.nist.gov/vuln/detail/CVE-2023-49993 - “NVD - Home”", "url": "https://wpnews.pro/news/my-paranoid-coding-machine-setup", "canonical_source": "https://www.brendanlong.com/my-paranoid-coding-machine-setup.html", "published_at": "2026-11-10 00:56:00+00:00", "updated_at": "2026-11-10 01:22:14.902792+00:00", "lang": "en", "topics": ["ai-agents", "ai-safety", "ai-tools"], "entities": ["Brendan Long", "Claude Code", "Podman", "gVisor", "Nvidia", "Kokoro-82M", "Cloudflare Tunnel", "Rust"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/my-paranoid-coding-machine-setup", "markdown": "https://wpnews.pro/news/my-paranoid-coding-machine-setup.md", "text": "https://wpnews.pro/news/my-paranoid-coding-machine-setup.txt", "jsonld": "https://wpnews.pro/news/my-paranoid-coding-machine-setup.jsonld"}}