{"slug": "thoughts-from-existing-b70-users", "title": "Thoughts from existing B70 users?", "summary": "Existing B70 users report that the best Intel LLM performance is achieved using a specific GitHub gist, with Qwen3.8-27B reaching 2313.29 tokens/s prefill and 30.76 tokens/s generation on a B70. One user, who has ordered a second B70 for tensor parallelism or independent model runs, notes that setup requires fiddling, including patching vLLM with open PRs for prefix caching and considering KV cache offloading versus MTP, with MTP currently preferred.", "body_md": "The best Intel llm performance is unlocked with the urakozz github gist - I can’t post links but you can bing that.\n\n```\n| model       |   test |            t/s |     peak t/s |      ttfr (ms) |   est_ppt (ms) |   e2e_ttft (ms) |\n|:------------|-------:|---------------:|-------------:|---------------:|---------------:|----------------:|\n| Qwen3.8-27B | pp4096 | 2313.29 ± 8.87 |              | 1860.31 ± 6.81 | 1771.10 ± 6.81 |  1860.31 ± 6.81 |\n| Qwen3.8-27B |  tg256 |   30.76 ± 0.01 | 31.67 ± 0.47 |                |                |                 |\n```\n\nAlso 12k pp/s 100tok/s on qwen 3.6 35b a3b - although that model isn’t as capable as I’d like. I’m more than happy with daily driving 3.8 27b with hermes\n\nIt does take a bit of fiddling. I patched my vllm with some open PR’s to fix prefix caching and something else I’ve forgotten. Currently fumbling my way through enabling kv cache offloading - right now the choice is mtp or cache offload, and mtp wins for me.\n\nIn fact I’m so happy with the b70 that I have a second one arriving this week, to be run on a bifurcated pcie 4.0 slot - either tensor parallelism works or I run 2 independent models\n\nIf there’s any interest I’m happy to go into more detail", "url": "https://wpnews.pro/news/thoughts-from-existing-b70-users", "canonical_source": "https://forum.level1techs.com/t/thoughts-from-existing-b70-users/248928?page=2#post_31", "published_at": "2026-08-17 21:26:23+00:00", "updated_at": "2026-08-17 21:41:25.149043+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "developer-tools"], "entities": ["Intel", "Qwen3.8-27B", "vLLM", "B70", "urakozz"], "alternates": {"html": "https://wpnews.pro/news/thoughts-from-existing-b70-users", "markdown": "https://wpnews.pro/news/thoughts-from-existing-b70-users.md", "text": "https://wpnews.pro/news/thoughts-from-existing-b70-users.txt", "jsonld": "https://wpnews.pro/news/thoughts-from-existing-b70-users.jsonld"}}