{"slug": "running-local-llms-gpu-recommendations", "title": "Running Local LLMs GPU recommendations", "summary": "A Level1Techs forum thread on local LLM GPU recommendations surfaced community benchmarks, including a user reporting Qwen3.8-flash-next running on a CSQ-RCO-IQ3_S quant at roughly 2,500 tokens/s prefill and 90-100 tokens/s decode, with a fast PCIe 4 or newer NVMe drive recommended. Another user described an always-on LLM server built around a Dell Pro Max T2 with an Intel 285, 64GB DDR5, an RTX 2000 ADA and an Intel B70 running Qwen 3.8-27b Q4, assembled for about $2,500 total, while others discussed repurposing data-center MI25 GPUs for desktop LLM use.", "body_md": "[edt](https://forum.level1techs.com/u/edt)\n21\n \nTry Strata.  A person I know manages to run Qwen3.8-flash-next using a CSQ-RCO-IQ3_S quant at prefix of 2500t/s and decode at 90-100t/s.  You will want a fast nvme (pci4 or more).\n\n \n1 Like\n\n \nYou may like this thread about using the MI25 as a cheap card for LLMs: [Converting a data center GPU for desktop use - #73 by Preston_Bannister](https://forum.level1techs.com/t/converting-a-data-center-gpu-for-desktop-use/250919/73)\n\nMyself and few others have done a chunk of testing on these cards specifically.\n\n \n2 Likes\n\n \nWhat I have found works for me:\n\nI still just use my 5090 for gaming only, so take my opinions with a hearty does of seasoning.\n\nBut my dedicated, always-on llm server hosted via WebUI over Tailscale connecting to my Dell Pro Max T2 with Intel 285, 64GB ddr5, RTX 2000 ADA and an Intel B70 I added later. The B70 handles the main model- Qwen 3.8-27b Q4, while the RTX 2000 manages whisper, vision, and helper models. I use a  custom harness-LangGraph. LiteLLM router, MCP Servers, Langfuse, RAG to my NAS and more.\n\nThis system has meaningfully improved my work and my life, though heavy admin and projects still go through cloud model review- Opus 5.5\n\nI got lucky and bought that PC for like $1200, the video card was another$1200, but total costs are around $2500, and it has definitely saved me at least that much time. My industry and my workplace are demonstrably uninterested in LLM tools, so my embrace of them gives me a chance to best serve my colleagues and clients when they are about to do the equivalent of long division without an abacus.\n\nI’m waiting for someone to convince me that a second B70 would  meaningfully improve my life so if anyone has experience, please let me know, and if you wanna see 5k, cinematic settings of me doing emotes in fortnite, all ya gotta do is ask\n\n \n\n \n[f41n](https://forum.level1techs.com/u/f41n)\n24\n \nThat’s actually the thread I stumbled on and made me not only start buying MI25’s, but join this forum \n\n \n\n \nSweet! You should definitely post there or a new thread and let us know what kinda things you’re doing and how the performance is.\n\nI’ve been meaning to get back into testing my setup since we no longer have unlimited tokens at work.\n\n \n\n \n[f41n](https://forum.level1techs.com/u/f41n)\n26\n \nThat’s the plan, but it’ll take a while for me to get the system up and running and full built out, then I’ll do a write up of sorts.", "url": "https://wpnews.pro/news/running-local-llms-gpu-recommendations", "canonical_source": "https://forum.level1techs.com/t/running-local-llms-gpu-recommendations/258522?page=2#post_26", "published_at": "2026-10-18 15:25:30+00:00", "updated_at": "2026-10-18 15:49:31.403057+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools"], "entities": ["Level1Techs", "Qwen3.8-flash-next", "Qwen 3.8-27b", "Intel B70", "RTX 2000 ADA", "Dell Pro Max T2", "MI25", "LangGraph"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/running-local-llms-gpu-recommendations", "markdown": "https://wpnews.pro/news/running-local-llms-gpu-recommendations.md", "text": "https://wpnews.pro/news/running-local-llms-gpu-recommendations.txt", "jsonld": "https://wpnews.pro/news/running-local-llms-gpu-recommendations.jsonld"}}