Contemplating expanding my homelab AI solution, and would like to get some input from others on my planning thoughts. Currently running a dedicated box with below.
Ubuntu 26.04 LTS
5070 16GB VRAM
64GB RAM
10G LAN
Use cases are Ollama, Open-WebUI, Hermes, code assistance, and a bunch of automation tools like n8n that use ollama externally. Can’t do video generation or some of the better and larger models which is a big blocker right now. The issue is pretty clear, not enough VRAM. At this point, I have 3 solutions with different costs:
Would love to understand other peoples experience and appreciate any lessons learned. Thanks.
source & further reading
forum.level1techs.com — original article
R-Core for AGI — Break Gen1
Kimi k3 locally with 8gb ram 0 vram?!?!
Designing an open-source budget 4x V100 SXM2 NVLink backplane — passive PCIe vs PEX8749?