{"slug": "nanogemm-bare-metal-avx2-gemm-in-100kb-faster-than-numpy", "title": "NanoGEMM – Bare-Metal AVX2 GEMM in ~100KB, Faster Than NumPy", "summary": "NanoGEMM, a bare-metal AVX2/FMA matrix multiplication engine, claims to outperform NumPy 2.2.3 on small-to-medium matrices, achieving a 2.83x speedup on 16x16 float32 matrices (1.23 µs vs 3.21 µs) but falling behind on larger sizes, with 128x128 and 256x256 matrices running at 0.60x and 0.28x the speed, respectively. The open-source project, available via pip and GitHub, uses register tiling and cache blocking to eliminate BLAS overhead for sub-microsecond CPU inference.", "body_md": "**NanoGEMM** is a minimalist, bare-metal General Matrix Multiplication (GEMM) engine designed for sub-microsecond CPU inference and high-performance computing in Python.\n\nBuilt with direct **AVX2 / FMA (256-bit SIMD)** assembly-level register tiling and cache blocking, NanoGEMM eliminates the heavy function-call dispatch, thread-pool barriers, and memory-packing overhead of heavyweight BLAS libraries (OpenBLAS, MKL) for small-to-medium tensors.\n\nMeasured on **Intel/AMD x86-64 CPU (AVX2 + FMA)** against **NumPy 2.2.3** (single-precision `float32`):\n\n| Matrix Dimension | NumPy 2.2.3 Latency | NanoGEMM Latency | Speedup Factor | NanoGEMM Throughput | \n|---|---|---|---|---|\n| **`16 x 16`** | `3.21 µs` | **`1.23 µs`** (C:` 0.65 µs` ) | 🚀 **2.83x FASTER** | `2.83 GFLOPS` | \n| **`32 x 32`** | `5.75 µs` | **`2.74 µs`** (C:` 2.18 µs` ) | 🚀 **2.26x FASTER** | `15.13 GFLOPS` | \n| **`64 x 64`** | `18.70 µs` | **`17.76 µs`** (C:` 16.39 µs` ) | 🚀 **1.10x FASTER** | `27.08 GFLOPS` | \n| **`128 x 128`** | `114.07 µs` | `182.46 µs` | `0.60x` | `23.42 GFLOPS` | \n| **`256 x 256`** | `426.24 µs` | `1501.24 µs` | `0.28x` | `22.35 GFLOPS` | \n\n💡 **Why is NanoGEMM faster on small/medium matrices?**\n\nTraditional BLAS engines incur 3–10 µs of fixed overhead per invocation due to dynamic runtime dispatch, argument sanitization, thread synchronization, and packing buffers. NanoGEMM utilizes a zero-allocation, direct register-tiled microkernel that executes in **sub-microsecond time** immediately upon invocation.\n\n- \n**Register allocation:** Utilizes 12`ymm` registers (`ymm0` –`ymm11` ) as 256-bit floating-point accumulators storing a$6 \\times 16$ tile of matrix$C$ .\n- \n**Vector broadcast & FMA:** Two`ymm` registers load vectors from$B$ , while individual scalar elements of$A$ are broadcast across`ymm` using`_mm256_set1_ps` and accumulated via fused multiply-add (`_mm256_fmadd_ps` ).\n- \n**Zero Spilling:** Fits completely inside the 16 available x86-64 YMM registers without stack eviction.\n\n- \n$L_1$ /$L_2$ Cache Tiling:$M_c = 64, N_c = 128, K_c = 128$ ) to maintain maximum L1/L2 data cache hit ratios and eliminate memory bus thrashing.\n- \n**Vectorized Edge Handling:** Arbitrary matrix dimensions (non-multiples of 6 or 16) are processed using boundary SIMD edge loops without padding or buffer allocations.\n\n```\n       Matrix A (M x K)              Matrix B (K x N)\n     [ . . . . . . . . ]           [ . . . ymm0 . . . ]\n     [ . . . . . . . . ]           [ . . . ymm1 . . . ]\n     [ a0 a1 a2 a3 . . ]     x     [ . . . . .  . . . ]\n     [ . . . . . . . . ]           [ . . . . .  . . . ]\n     [ . . . . . . . . ]\n             │                             │\n             └──────────────┬──────────────┘\n                            ▼\n                Matrix C (6 x 16 Tile)\n             [ ymm0  ymm1  ] -> Row 0\n             [ ymm2  ymm3  ] -> Row 1\n             [ ymm4  ymm5  ] -> Row 2\n             [ ymm6  ymm7  ] -> Row 3\n             [ ymm8  ymm9  ] -> Row 4\n             [ ymm10 ymm11 ] -> Row 5\npip install nanogemm\ngit clone https://github.com/eminsk/nanogemm.git\ncd nanogemm\npip install -e .\npython\nimport nanogemm as ng\nimport numpy as np\n\n# Verify SIMD hardware acceleration\nprint(\"Active ISA:\", ng.get_simd_isa())\n# Output: Active ISA: AVX2+FMA (256-bit SIMD, 6x16 register tiling)\n\n# Allocate input matrices\nA = np.random.randn(32, 64).astype(np.float32)\nB = np.random.randn(64, 128).astype(np.float32)\n\n# Direct hardware-accelerated MatMul: C = A @ B\nC = ng.matmul(A, B)\n\n# Or with pre-allocated zero-copy output buffer for maximum performance:\nout = np.empty((32, 128), dtype=np.float32)\nng.matmul(A, B, out=out)\n\n# Standard BLAS SGEMM interface: C = alpha * (A @ B) + beta * C\nres = ng.sgemm(A, B, alpha=2.0, beta=0.5, c=out)\n```\n\nRun the comprehensive correctness test suite comparing NanoGEMM with NumPy reference outputs across random uniforms, normals, non-square dimensions, and prime shapes:\n\n```\npython tests/test_correctness.py\n```\n\nRun the official benchmark against your installed NumPy BLAS:\n\n```\npython benchmarks/bench_vs_numpy.py\n```\n\nNanoGEMM is developed by [**@eminsk**](https://github.com/eminsk) as part of an engineering ecosystem focused on low-level hardware performance, assembly programming, and native desktop computing:\n\n- 🎥 [**screenvideo**](https://github.com/eminsk/screenvideo) — Lightweight desktop screen recorder featuring WASAPI loopback audio and a standalone pure x64 Flat Assembler (FASM) native edition.\n- 📊 [**xlsx_vievers**](https://github.com/eminsk/xlsx_vievers) — Desktop spreadsheet processor with 80+ formula functions, Chart Wizard, and hardware-accelerated SIMD SSE2 math engine.\n- 📈 [**yfinance-ta-patterns**](https://github.com/eminsk/yfinance-ta-patterns) — Candlestick pattern scanner and AI ranking suite powered by TA-Lib and quantitative backtesting.\n- 🔍 [**StackOverflowAPI**](https://github.com/eminsk/StackOverflowAPI) — Desktop client for Stack Overflow built with CustomTkinter and native FASM x64 search client.\n\nMIT License — Copyright (c) 2026 [eminsk](https://github.com/eminsk).", "url": "https://wpnews.pro/news/nanogemm-bare-metal-avx2-gemm-in-100kb-faster-than-numpy", "canonical_source": "https://github.com/eminsk/nanogemm", "published_at": "2026-09-07 15:00:48+00:00", "updated_at": "2026-09-07 15:27:43.166923+00:00", "lang": "en", "topics": ["artificial-intelligence", "machine-learning", "ai-infrastructure", "ai-tools", "developer-tools"], "entities": ["NanoGEMM", "NumPy 2.2.3", "OpenBLAS", "MKL", "AVX2", "FMA", "Intel", "AMD"], "alternates": {"html": "https://wpnews.pro/news/nanogemm-bare-metal-avx2-gemm-in-100kb-faster-than-numpy", "markdown": "https://wpnews.pro/news/nanogemm-bare-metal-avx2-gemm-in-100kb-faster-than-numpy.md", "text": "https://wpnews.pro/news/nanogemm-bare-metal-avx2-gemm-in-100kb-faster-than-numpy.txt", "jsonld": "https://wpnews.pro/news/nanogemm-bare-metal-avx2-gemm-in-100kb-faster-than-numpy.jsonld"}}