{"slug": "opengemm-open-source-b200-gemm-kernels", "title": "OpenGEMM: Open-source B200 GEMM kernels", "summary": "OpenGEMM, an open-source project from developer aramesh10, released CUDA GEMM kernels for NVIDIA's B200 GPU (sm_100a), installable via pip and requiring PyTorch 2.8+ and CUDA 12.9+. The library emits standalone CUDA kernels for dtypes including bf16, f16, tf32, s8, u8, e4m3, e5m2, e3m2, e2m3, and e2m1, plus block-scaled nvfp4, mxfp8, and mxfp4 formats, and compiles its two libraries on the first gemm() call in about 25 seconds. Optimized configs are stored in configs.json, with unoptimized shapes autotuned and saved locally to ./opengemm-configs/tuned_configs.json or the OPENGEMM_CONFIGS environment variable.", "body_md": "GEMM kernels for NVIDIA B200 (sm_100a) in CUDA.\n\n``` python\nimport opengemm as og\n\nc = og.gemm(a, b)                    # C[M, N] = A[M, K] @ B[N, K].T\nc = og.gemm(a, b, sfa, sfb)          # block-scaled: nvfp4, mxfp8, mxfp4\n\nog.emit_kernel(a, b, file=\"k.cu\")    # emits .cu/.cuh for this shape\nc = og.run_kernel(\"k.cu\", a, b)      # compiles emitted kernel and runs it\n```\n\nCheck [API.md](https://github.com/aramesh10/OpenGEMM/blob/main/API.md) for documentation.\n\nFrom PyPI\n\n```\npip install opengemm\n```\n\nFrom a clone:\n\n```\ngit clone https://github.com/aramesh10/OpenGEMM.git\ncd OpenGEMM\npip install -e .\n```\n\nRequirements:\n\n- sm_100a\n- PyTorch 2.8+\n- CUDA 12.9+\n\nThe kernels are compiled into two libraries on the first `gemm()` call and takes ~25s. Run `python -m opengemm` or from python `og.prebuild()` to pay the cost at install time instead.\n\nGive your agent this prompt to use OpenGEMM as a tool:\n\n```\nOpenGEMM emits standalone CUDA GEMM kernels for B200 (sm_100a), no GPU\nneeded to emit:\n\npython -c \"\nimport opengemm as og\nS = dict(m=1024, n=1024, k=1024)\n\nog.emit_kernel(**S, atype='bf16', file='k')         # writes k.cu and k.cuh\nog.emit_kernel(**S, atype='e4m3', btype='e5m2')     # mixed, names itself\nog.emit_kernel(**S, atype='e2m1', sftype='ue4m3')   # block-scaled (nvfp4)\nsrc, hdr = og.emit_kernel(**S, atype='bf16')        # the text, always returned\nprint(src, hdr)\n\"\natype / btype: bf16 f16 tf32 s8 u8 e4m3 e5m2 e3m2 e2m3 e2m1\nsftype (block-scaled): ue4m3 (nvfp4) or ue8m0 (mxfp8, mxfp4)\n```\n\n`C[M, N] = A[M, K] @ B[N, K].T`. Both operands are row-major with K innermost.\n\n| GEMM | `atype` /`btype` | `sftype` | output | `torch.dtype` (in → out) | \n|---|---|---|---|---|\n| bfloat16 | bf16 | — | f32 | `bfloat16` →`float32` | \n| float16 | f16 | — | f32 | `float16` →`float32` | \n| tf32 | tf32 | — | f32 | `float32` →`float32` | \n| int8 | s8 | — | s32 | `int8` →`int32` | \n| uint8 | u8 | — | s32 | `uint8` →`int32` | \n| fp8 | e4m3 | — | f32 | `float8_e4m3fn` →`float32` | \n| fp8 | e5m2 | — | f32 | `float8_e5m2` →`float32` | \n| mixed fp8 | e4m3, e5m2 | — | f32 | `float8_e4m3fn` ,`float8_e5m2` →`float32` | \n| fp6 | e3m2 | — | f32 | `uint8` →`float32` | \n| fp6 | e2m3 | — | f32 | `uint8` →`float32` | \n| fp4 | e2m1 | — | f32 | `uint8` →`float32` | \n| nvfp4 | e2m1 | ue4m3 (per 16) | bf16 | `float4_e2m1fn_x2` ,`float8_e4m3fn` →`bfloat16` | \n| mxfp8 | e4m3 | ue8m0 (per 32) | bf16 | `float8_e4m3fn` ,`float8_e8m0fnu` →`bfloat16` | \n| mxfp4 | e2m1 | ue8m0 (per 32) | bf16 | `float4_e2m1fn_x2` ,`float8_e8m0fnu` →`bfloat16` | \n\nNote: fp6 and fp4 have no torch dtype. They arrive densely packed in `uint8` and are named - `gemm(a, b, atype=\"e2m1\")`\nUse `btype=` when the two operands differ.\n\nOutput is `[M, N]`, row-major, like `torch.mm`.\n\nThere is no heursitic to choose the config. Optimized configs are stored in `configs.json`.\nIf a particular shape has not been optimized, the library autotunes and returns and saves the best config locally to `./opengemm-configs/tuned_configs.json` or to `OPENGEMM_CONFIGS` env variable.\n\n```\nCUDA_VISIBLE_DEVICES=0 python scripts/tune.py --dtype f16 --shape 4096 4096 4096\nCUDA_VISIBLE_DEVICES=0 python scripts/benchmark.py --dtype bf16 e4m3    # vs cuBLAS\nCUDA_VISIBLE_DEVICES=0 python scripts/test.py                           # correctness\n```\n\n`tune.py` ablates every compiled configuration for a shape and records the best performing config to `configs.json`\n\n```\npython scripts/emit_kernel.py --dtype e4m3 --shape 4096 4096 4096 --file emitted/e4m3_4k.cu\npython scripts/run_kernel.py emitted/e4m3_4k.cu       # correctness, then timing vs cuBLAS\n```\n\nOpenGEMM can also emit the optimized CUDA files for a kernel given a shape and dtype. It can be ran with `scripts/run_kernel.py` or built with `nvcc`:\n\n```\nnvcc -O3 -std=c++20 -gencode=arch=compute_100a,code=sm_100a --expt-relaxed-constexpr -shared -Xcompiler -fPIC -lcuda <KERNEL_FILE>.cu -o <KERNEL_FILE>.so\n```\n\nThe entry point is `extern \"C\" void mm_<dtype>_<M>_<N>_<K>(a, b, c, stream)`,\nor `smm_<dtype>_<M>_<N>_<K>(a, b, sfa, sfb, c, stream)`\n\n`emit_kernel` reads only shapes and dtypes, so meta tensors work:\n`emit_kernel(torch.empty(4096, 4096, dtype=torch.bfloat16, device=\"meta\"), ...)`.", "url": "https://wpnews.pro/news/opengemm-open-source-b200-gemm-kernels", "canonical_source": "https://github.com/aramesh10/OpenGEMM/tree/main", "published_at": "2026-09-13 17:19:02+00:00", "updated_at": "2026-09-13 17:46:50.975649+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-chips", "machine-learning", "developer-tools"], "entities": ["OpenGEMM", "NVIDIA B200", "CUDA", "PyTorch", "aramesh10", "cuBLAS", "nvcc", "sm_100a"], "alternates": {"html": "https://wpnews.pro/news/opengemm-open-source-b200-gemm-kernels", "markdown": "https://wpnews.pro/news/opengemm-open-source-b200-gemm-kernels.md", "text": "https://wpnews.pro/news/opengemm-open-source-b200-gemm-kernels.txt", "jsonld": "https://wpnews.pro/news/opengemm-open-source-b200-gemm-kernels.jsonld"}}