cd /news/ai-infrastructure/measured-attention-kernels-vs-amd-s-… · home › topics › ai-infrastructure › article
[ARTICLE · art-172855] src=github.com ↗ pub= topic=ai-infrastructure verified=true sentiment=· neutral

Measured attention kernels vs. AMD's own on MI350X, with every loss published

Riptide published measured benchmarks of its attention kernels against AMD's own on the AMD Instinct MI350X, reporting a 3.32× speedup [3.28, 3.38] over AMD's best PyTorch backend for d128 causal forward-plus-backward at sequence length 512, while losing 0.82–0.84× at 16k tokens for d128 bf16. All runs used bf16 with 32 heads, 16k tokens per batch, ROCm 10.2 and PyTorch 2.15, with 95% confidence intervals and float64 correctness checks, and Riptide published every loss alongside the wins. The kernels are not open source and are offered to partners for evaluation and licensing, and against AMD's AITER library Riptide's deterministic backward ran 1.51–2.88× faster while AITER's head-dim-128 forward stayed faster.

recommendation

Benchmark attention kernels per head dimension and sequence length before adopting them: Riptide's kernels win 3.32× at d128 causal S=512 but lose 0.82–0.84× at 16k tokens.

more in playbook: Inference and self-hosting
read4 min views6 publishedNov 19, 2026
Measured attention kernels vs. AMD's own on MI350X, with every loss published
Image: Michielbdejong (auto-discovered)

Measured results for Riptide's GPU kernels on AMD Instinct MI350X, and the benchmark harness that produced them. Every number is measured in the same session as AMD's own kernels, with 95% confidence intervals and a float64 correctness check.

This repository holds the results, the method and the harness. The kernels themselves are not open source. They

are available to partners for evaluation and licensing: sam@s7vc.com. All reports: RESULTS.md · each run has a verdict, a chart, "where we lose", how it was measured,

the raw data and a machine-readable results.json (format). F.scaled_dot_product_attention, bf16, 32 heads, 16k tokens per batch. TFLOP/s at the median of 30 samples [95% CI]. "AMD best" is the faster of PyTorch's FLASH_ATTENTION (AOTriton) and EFFICIENT_ATTENTION backends in the same session. ROCm 10.2, PyTorch 2.15. fwd+bwd is one training step of the attention layer.

| shape | Riptide | AMD best | Riptide / AMD best |

|---|---|---|---|
| d128 causal fwd+bwd, S = 512 | 249 [247, 252] | 75 | **3.32×** [3.28, 3.38] | 
| d64 causal fwd+bwd, S = 512 | 213 [211, 215] | 69 | **3.10×** [3.07, 3.14] | 
| d128 fwd+bwd, S = 1024 | 477 [472, 481] | 417 | **1.14×** [1.12, 1.17] | 
| d128 causal fwd+bwd, S = 2048 | 444 [440, 447] | 389 | **1.14×** [1.12, 1.16] | 
| d64 forward, S = 16384 | 717 [714, 718] | 594 | **1.21×** [1.19, 1.22] | 
| d64 causal fwd+bwd, S = 16384 | 514 [513, 515] | 464 | **1.11×** [1.10, 1.11] | 
| d128 forward, S = 16384 | 821 [819, 823] | 982 | 0.84× [0.83, 0.84] | 
| d128 fwd+bwd, S = 16384 | 635 [635, 635] | 756 | 0.84× [0.84, 0.84] | 
| d128 causal fwd+bwd, S = 16384 | 597 [596, 597] | 732 | 0.82× [0.81, 0.82] | 
  • Ahead : head dim 64 at every length (forward 1.14–1.39×, training 1.03–1.14× from 1k tokens, 1.9–3.1× at 512); head dim 128 training up to 1k (full) and 2k (causal) tokens, 3.32× at 512 causal; head dim 128 causal forward up to 2k tokens and fp16 forward at every length.
  • Behind : head dim 128 bf16 from 2k (full) / 4k (causal) tokens: 0.82–0.84× at 16k, up from 0.61× / 0.48× that morning. fp16 head dim 128 forward is even or ahead at every length (1.0–1.21×).
Full report: [results/attention/2026-11-19f/REPORT.md](https://github.com/s7ventures/riptide-amd/blob/main/results/attention/2026-11-19f/REPORT.md)
(the day's earlier runs: 2026-11-19, -19b, -19c, -19d).

Run d kernels (the AITER run came before run f's forward changes). Same pod, AITER amd-aiter 0.1.7 flash_attn_func, bf16, 32 heads, 16k tokens per batch, S = 1k–16k, median of 30 calls; outputs agree to bf16 rounding. Raw rows: results/attention/2026-11-19-aiter/vs_aiter.jsonl.

forward fwd+bwd, vs AITER's deterministic backward
d64 full / causal 1.00–1.12× / 1.03–1.12× 1.51–1.77× / 1.85–2.88×
d128 full / causal 0.78–0.90× / 0.78–0.82× 1.05–1.20× / 1.61–1.72×

AITER ran its deterministic backward (Riptide's backward is deterministic too); its non-deterministic backward has not been timed yet, so these rows are not a comparison with AITER's fastest backward. AITER's head-dim-128 forward is faster than Riptide's.

case Riptide baseline ratio
Qwen2.5-0.5B decode, batch 1, prompt 128 399 tok/s 106 tok/s (Hugging Face eager) 3.75× [3.73, 3.75]
same, batch 8 2.36×
same vs torch.compile, batch 1 1.37× [1.36, 1.38]
MNIST MLP training step 0.127 ms 0.225 ms (torch.cuda graphs) 1.76× [1.59, 1.90]
empty kernel launch 1.53 µs 6.83 µs (Triton) 4.47×

Level or behind: graph replay per launch equals torch.cuda graphs; batch 8 decode vs torch.compile 0.96× / 0.79×.

All rows: results/launch/2026-11-19/headline.md. bench/ measures AMD's own stack on any MI300/MI350 machine: GEMM, attention, elementwise ops, convolution, training, LLM inference, the library CLIs and MIGraphX. Statistics, correctness checks and the FLOP accounting are in docs/methodology.md; how to run it is in docs/reproducing.md. The Riptide rows need the riptide-amd package (partners); everything else runs as is.

Evaluation access and licensing: sam@s7vc.com · LinkedIn: Samario Torres · riptidecompute.com The harness and documentation in this repository are licensed under Apache-2.0 (see LICENSE). The results are published for reference; please cite them as in CITATION.cff.

── more in #ai-infrastructure 4 stories · sorted by recency
── more on @riptide 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/measured-attention-k…] indexed:0 read:4min 2026-11-19 · —