{"slug": "llm-under-proxmox-with-gpu-passthrough", "title": "LLM under Proxmox with GPU passthrough?", "summary": "Forum users on Level1Techs report that running local LLMs on a Proxmox host with an Nvidia RTX 3090 passed through to a VM works without known blockers, with one user saying their local agents run in a Proxmox VM with \"slightly higher average TPS than expected.\" A second user describes two approaches: installing drivers on the host and passing through device paths to multiple machines, or passing through the entire PCIe device so one VM gets exclusive access, noting they have done both for Jellyfin transcodes and a virtual gaming PC. A third user runs a 3090 in a vLLM container on Unraid, with agents reaching it over a URL and port via Tailscale.", "body_md": "Hello fellow nerds,\n\nI have a Proxmox machine with a 5950X and 128GB of DDR4. I am interested in putting a 3090 in, passing it through to a VM, and running Strata with Qwen 3.8 Flash Next IQ3_S.\n\nAre there any known blockers for this? Or potential Proxmox related performance bottlenecks?\n\nThanks!\n\n \n\n \nDon’t believe so, should work just fine\n\nMy local agents are all setup in a VM on my proxmox server and it was a pretty seamless experience. I am getting a slightly higher average TPS than expected as well\n\n \n1 Like\n\n \n[nand](https://forum.level1techs.com/u/nand)\n3\n \nYeah worked fine for me with Nvidia.\n\nYou could even do drivers on the host and pass through paths to machines that need the card. (I’ll call it method 1)\n\nI did method 1 for Jellyfin transcodes in the past. You can also pass through the entire pcie device (method 2) if you’d like a vm to get it exclusively, I’ve done that too for a virtual gaming pc.\n\nI haven’t tried cuda specifically but I think it’s at a higher level of the stack and should work.\n\nSo tl;dr: +1 to [@ItTakes2ToMango](https://forum.level1techs.com/u/ittakes2tomango)\n\n \n\n \nI use unraid, and have my 3090 in a vllm container. Then my agents hit that url and port to facilitate. Tailscale is based.", "url": "https://wpnews.pro/news/llm-under-proxmox-with-gpu-passthrough", "canonical_source": "https://forum.level1techs.com/t/llm-under-proxmox-with-gpu-passthrough/260136#post_4", "published_at": "2026-11-05 20:12:00+00:00", "updated_at": "2026-11-05 20:20:15.019289+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-agents"], "entities": ["Proxmox", "Nvidia", "RTX 3090", "AMD 5950X", "Qwen", "vLLM", "Unraid", "Tailscale"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/llm-under-proxmox-with-gpu-passthrough", "markdown": "https://wpnews.pro/news/llm-under-proxmox-with-gpu-passthrough.md", "text": "https://wpnews.pro/news/llm-under-proxmox-with-gpu-passthrough.txt", "jsonld": "https://wpnews.pro/news/llm-under-proxmox-with-gpu-passthrough.jsonld"}}