Cerebras WSE-3 smokes H100 on Llama 3 70B inference at a

Cerebras Systems claims its Wafer-Scale Engine 3 (WSE-3) delivers 1,800 tokens per second at 23 kW for Llama 3 70B inference, versus about 850 tokens per second at 56 kW for an 8×H100 DGX system, yielding a five-to-one performance-per-watt advantage. However, the WSE-3 only excels when the model and KV cache fit in its 44 GB on-die SRAM, requiring INT4/INT8 quantization for Llama 3 70B, and Nvidia's CUDA ecosystem remains more mature for general-purpose training and inference.

Cerebras WSE-3 smokes H100 on Llama 3 70B inference at a The architecture difference is the whole story. Nvidia's approach is scale-out: stitch GPUs together with NVLink/NVSwitch, fight tail latency across the fabric, burn cycles on kernel launches and synchronization. Cerebras went scale-up — 900,000 cores on one piece of silicon, 44 GB on-die SRAM no HBM, no DRAM hops , and a 2D torus mesh that moves data at 100+ TB/s internally. For dense LLM inference where the model fits in SRAM, you eliminate the memory wall entirely. Rough power/perf comparison from their published figures CS-3 1 system : 1,800 tok/s @ 23 kW → 78 tok/s/W 8×H100 DGX: ~850 tok/s @ 56 kW → 15 tok/s/W Five-to-one perf-per-watt is the kind of gap that makes CFOs pay attention when you're planning a 10 MW cluster. Caveats matter though. The WSE-3 only shines when your model + KV cache fits in that 44 GB SRAM. Llama 3 70B at FP16 is ~140 GB — so they're running INT4 or INT8 quantization, and even then it's tight. For models above ~30B params at decent precision, you're back to multi-system clustering with their MemoryX boxes, and the programming model gets more complex. Nvidia's advantage stays the software stack: CUDA, TensorRT-LLM, vLLM, Triton — the entire ecosystem assumes Nvidia. Cerebras SDK is Python-based but you're writing to a completely different execution model dataflow vs. SIMT . Porting effort is real. Also worth noting: these are inference numbers. Training is a different beast — Nvidia still owns that mindshare and the tooling maturity. Cerebras has done impressive work on sparse training and they've shown GPT-3 scale runs, but the day-to-day ML engineer experience is years behind. If you're building a dedicated inference fleet for a known model family Llama, Mistral, proprietary fine-tunes and power/rack space are constraints, the CS-3 deserves a serious eval. For general-purpose training + inference clusters where flexibility wins? H100/B200 still the default call. Curious if anyone here has touched the Cerebras SDK for a real workload — the dataflow programming model is a mental shift. Nvidia's buyback binge looks smart until you check the 2h ago /en/news/6956/ Cerebras CS-4 rack density pushes wafer-scale cooling to new 5h ago /en/news/6937/ PantheonGPU proves that telemetry alone is a lie for GPU health 22h ago /en/news/6849/ Groq is spending billions to poach Nvidia engineers 1d ago /en/news/6810/ Microsoft is hitting a massive hardware wall that could stall 1d ago /en/news/6771/ Big Tech is spending way more on AI than the balance sheets 2d ago /en/news/6657/ Next Nvidia's buyback binge looks smart until you check the → /en/news/6956/