Cheap GPU vs Expensive GPU: Running Big LLMs Locally on a Budget
A $2,000 AMD Radeon R9700 paired with 256GB of system RAM sustained 16 concurrent coding agents on a 124 billion parameter mixture-of-experts model, while a dual RTX Pro 6000 machine with 192GB of VRAM but only 128GB of …