My adventures with local AI, Ollama, Open WebUI and Proxmox MCP
I’ve been experimenting with using a local LLM to monitor and eventually manage my Proxmox homelab through MCP.
My setup:
AMD RX 6700 XT 12GB running Ollama on Windows
Open WebUI running in a separate Linux VM
Proxmox MCP connected to a dedicated test host
Qwen3.5:4B / 9B and IBM Granite 4.1:8B Getting everything connected was fairly straightforward, but getting the models to actually use the tools reliably was another story!
Qwen struggled with tool selection, especially when Open WebUI’s built-in tools were enabled. Tool Search was painfully slow, and I even had Qwen hallucinate a running Debian VM that didn’t exist. Responses were also surprisingly slow despite 100% GPU off.
Then I tried IBM Granite 4.1:8B, and things improved dramatically.
Granite was faster, correctly queried Proxmox, reported actual VM/container inventories, checked CPU/RAM/storage resources, and even investigated whether the host could accommodate a new VM.
It’s not perfect — some calculations and technical details needed correcting — but it’s easily been my best experience so far.
Next experiment: Giving Granite restricted write access so it can actually create a VM through MCP, with dry-run validation and human approval.
It’s been a surprisingly fun rabbit hole, and a good reminder that choosing the right model matters just as much as the hardware!