{"slug": "show-hn-agentic-cuda-kernel-optimizer", "title": "Show HN: Agentic CUDA Kernel Optimizer", "summary": "A developer released an agentic CUDA kernel optimizer on Hacker News' Show HN that uses LangGraph to autonomously generate, validate, benchmark, and refine GPU kernels, defaulting to OpenAI's gpt-5-mini model with medium reasoning effort. The tool compiles kernels with NVRTC through the CUDA Driver API, validates every case against NumPy references, and ranks candidates by the geometric mean of kernel latency across performance cases, measured with 10 warmup and 100 timed launches via CUDA events. It was developed on Windows with an RTX 3060 Laptop GPU and requires Python 3.12+, CMake 3.24+, a C++17 compiler, an NVIDIA GPU with compatible CUDA Toolkit/driver, and an OpenAI API key, with optional Nsight Compute profiling and NVIDIA documentation research flags.", "body_md": "An agentic CUDA kernel optimizer that turns workload descriptions into GPU implementations through an automated cycle of code generation, correctness checks, benchmarking, and refinement. Powered by LangGraph, the agent explores kernel implementations and launch configurations, queries GPU properties, and can research NVIDIA documentation for optimization guidance and inspect Nsight Compute counters to inform its next experiment. Each experiment is recorded, and the fastest validated implementation is retained.\n\nThe model can change both kernel code and per-case launch configurations. A standalone C++ harness compiles kernels with NVRTC, launches them through the CUDA Driver API, and saves outputs. Python handles comparison and candidate selection.\n\n1. Load or generate a signature, input cases, reference kernel, and initial kernel.\n2. Run the reference and evaluate the initial implementation.\n3. Propose a change, compile it, compare outputs with NumPy, and measure kernel latency.\n4. Feed results back into the next attempt; repair invalid candidates within the iteration budget.\n5. Save the fastest validated candidate, execution history, and a timing heatmap.\n\nEvery case must pass validation. Ranking uses the geometric mean of latency across performance cases; small correctness cases do not affect the score. Timing defaults to 10 warmup launches and 100 measured launches using CUDA events. Compilation time and profiler replay timings are excluded from ranking.\n\nDeveloped on Windows with an RTX 3060 Laptop GPU. Requires Python 3.12+, an NVIDIA GPU and compatible CUDA Toolkit/driver, CMake 3.24+, a C++17 compiler, and an OpenAI API key. The build commands below use Visual Studio 2026 with the C++ tools installed.\n\nFrom the repository root:\n\n```\npython -m venv .venv\n.venv\\Scripts\\python -m pip install -r optimizer_agent/requirements.txt\ncmake -S cuda_test_harness -B cuda_test_harness/build -DCMAKE_BUILD_TYPE=Release\ncmake --build cuda_test_harness/build --parallel\n```\n\nCreate a `.env` file in the repository root:\n\n```\nOPENAI_API_KEY=your-key-here\n.venv\\Scripts\\python optimizer_agent/optimizer_agent.py --description \"Single-precision GEMM with rectangular matrices.\" --max-iterations 7\n```\n\nThe default model is `gpt-5-mini` with medium reasoning effort. API usage is billed to your account. Use `-h` for all options, or `--config optimizer_agent/example.json` for the included configuration example.\n\nSupply your own workload components with `--signature`, `--reference`, `--initial-kernel`, and `--input-cases`. Omitted components are inferred or generated. To continue from an earlier run using its saved inputs:\n\n```\n.venv\\Scripts\\python optimizer_agent/optimizer_agent.py --description \"Single-precision GEMM with rectangular matrices.\" --input-cases results/run-001/input_cases.json --reference results/run-001/reference.cu --initial-kernel results/run-001/best.cu --max-iterations 7\n```\n\nThis example assumes the earlier run generated `reference.cu`; supplied references are saved as `supplied-reference.cu`. Input manifests retain paths to their binary data, so keep those files available.\n\nOptional flags:\n\n- `--use-nsight` : profile a performance case after each valid candidate and expose profiler tools to the model. Requires Nsight Compute and permission to access GPU performance counters.\n- `--nvidia-research` : retrieve NVIDIA guidance before generating kernels.\n\nFloat32 GEMM on an RTX 3060 Laptop GPU, with NVIDIA research and Nsight Compute enabled.\n\n| Run 022 | Run 023 | \n|---|---|\n| Optimizing the generated starting kernel. | Continuing from run 022's best kernel with the same inputs and reference. | \n\nEach session gets a directory under `results/run-NNN/` containing kernel sources, requests, input/output data, model/tool responses, `history.json`, and `summary.json`. Successful runs export `best.cu`, per-case replay requests, and `heatmap.png`/` heatmap.svg`. Nsight reports are saved when profiling is enabled.\n\nThis is an experimental optimizer for individual kernels. Passing supplied cases does not prove general correctness, and a generated reference is not an independent correctness oracle. Improvements are workload-dependent; no comparison against cuBLAS or other vendor libraries is currently included. Keep the GPU otherwise idle when comparing timings.\n\nGenerated input scripts execute locally as Python subprocesses without a sandbox. Generated CUDA kernels also run on the local GPU.\n\nThe graph below is rendered from the compiled LangGraph workflow with Nsight profiling enabled. Dashed edges are conditional routes. Without `--use-nsight`, evaluation routes directly to the next attempt or finalization; NVIDIA research is skipped unless `--nvidia-research` is set.", "url": "https://wpnews.pro/news/show-hn-agentic-cuda-kernel-optimizer", "canonical_source": "https://github.com/bertaye/agentic-cuda-optimizer", "published_at": "2026-09-25 10:32:58+00:00", "updated_at": "2026-09-25 11:00:40.037095+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "ai-infrastructure", "large-language-models"], "entities": ["LangGraph", "NVIDIA", "Nsight Compute", "CUDA", "NVRTC", "OpenAI", "gpt-5-mini", "RTX 3060 Laptop GPU"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/show-hn-agentic-cuda-kernel-optimizer", "markdown": "https://wpnews.pro/news/show-hn-agentic-cuda-kernel-optimizer.md", "text": "https://wpnews.pro/news/show-hn-agentic-cuda-kernel-optimizer.txt", "jsonld": "https://wpnews.pro/news/show-hn-agentic-cuda-kernel-optimizer.jsonld"}}