{"slug": "cerebras-wse-3-smokes-h100-on-llama-3-70b-inference-at-a", "title": "Cerebras WSE-3 smokes H100 on Llama 3 70B inference at a", "summary": "Cerebras Systems claims its Wafer-Scale Engine 3 (WSE-3) delivers 1,800 tokens per second at 23 kW for Llama 3 70B inference, versus about 850 tokens per second at 56 kW for an 8×H100 DGX system, yielding a five-to-one performance-per-watt advantage. However, the WSE-3 only excels when the model and KV cache fit in its 44 GB on-die SRAM, requiring INT4/INT8 quantization for Llama 3 70B, and Nvidia's CUDA ecosystem remains more mature for general-purpose training and inference.", "body_md": "# Cerebras WSE-3 smokes H100 on Llama 3 70B inference at a\n\nThe architecture difference is the whole story. Nvidia's approach is scale-out: stitch GPUs together with NVLink/NVSwitch, fight tail latency across the fabric, burn cycles on kernel launches and synchronization. Cerebras went scale-up — 900,000 cores on one piece of silicon, 44 GB on-die SRAM (no HBM, no DRAM hops), and a 2D torus mesh that moves data at 100+ TB/s internally. For dense LLM inference where the model fits in SRAM, you eliminate the memory wall entirely.\n\n```\n# Rough power/perf comparison from their published figures\nCS-3 (1 system):     1,800 tok/s  @ 23 kW  →  78 tok/s/W\n8×H100 DGX:          ~850 tok/s  @ 56 kW  →  15 tok/s/W\n```\n\nFive-to-one perf-per-watt is the kind of gap that makes CFOs pay attention when you're planning a 10 MW cluster.\n\nCaveats matter though. The WSE-3 only shines when your model + KV cache fits in that 44 GB SRAM. Llama 3 70B at FP16 is ~140 GB — so they're running INT4 or INT8 quantization, and even then it's tight. For models above ~30B params at decent precision, you're back to multi-system clustering with their MemoryX boxes, and the programming model gets more complex. Nvidia's advantage stays the software stack: CUDA, TensorRT-LLM, vLLM, Triton — the entire ecosystem assumes Nvidia. Cerebras SDK is Python-based but you're writing to a completely different execution model (dataflow vs. SIMT). Porting effort is real.\n\nAlso worth noting: these are inference numbers. Training is a different beast — Nvidia still owns that mindshare and the tooling maturity. Cerebras has done impressive work on sparse training and they've shown GPT-3 scale runs, but the day-to-day ML engineer experience is years behind.\n\nIf you're building a dedicated inference fleet for a known model family (Llama, Mistral, proprietary fine-tunes) and power/rack space are constraints, the CS-3 deserves a serious eval. For general-purpose training + inference clusters where flexibility wins? H100/B200 still the default call.\n\nCurious if anyone here has touched the Cerebras SDK for a real workload — the dataflow programming model is a mental shift.\n\n[Nvidia's buyback binge looks smart until you check the 2h ago](/en/news/6956/)\n\n[Cerebras CS-4 rack density pushes wafer-scale cooling to new 5h ago](/en/news/6937/)\n\n[PantheonGPU proves that telemetry alone is a lie for GPU health 22h ago](/en/news/6849/)\n\n[Groq is spending billions to poach Nvidia engineers 1d ago](/en/news/6810/)\n\n[Microsoft is hitting a massive hardware wall that could stall 1d ago](/en/news/6771/)\n\n[Big Tech is spending way more on AI than the balance sheets 2d ago](/en/news/6657/)\n\n[Next Nvidia's buyback binge looks smart until you check the →](/en/news/6956/)", "url": "https://wpnews.pro/news/cerebras-wse-3-smokes-h100-on-llama-3-70b-inference-at-a", "canonical_source": "https://promptcube3.com/en/news/6962/", "published_at": "2026-08-19 19:02:08+00:00", "updated_at": "2026-08-19 19:47:00.269860+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-infrastructure", "ai-chips", "large-language-models"], "entities": ["Cerebras Systems", "WSE-3", "CS-3", "Nvidia", "H100", "Llama 3 70B", "CUDA", "MemoryX"], "alternates": {"html": "https://wpnews.pro/news/cerebras-wse-3-smokes-h100-on-llama-3-70b-inference-at-a", "markdown": "https://wpnews.pro/news/cerebras-wse-3-smokes-h100-on-llama-3-70b-inference-at-a.md", "text": "https://wpnews.pro/news/cerebras-wse-3-smokes-h100-on-llama-3-70b-inference-at-a.txt", "jsonld": "https://wpnews.pro/news/cerebras-wse-3-smokes-h100-on-llama-3-70b-inference-at-a.jsonld"}}