Inside NVIDIA’s cuDNN Graph API: Fusion, Autotuning, and Plan Reuse with cuDNN Frontend

NVIDIA's cuDNN Frontend graph API lets developers express a computation as a graph of operations, have cuDNN select an execution engine, and then override that engine choice, according to a Marktechpost tutorial built on the open-source cudnn-frontend repository. The tutorial runs the full five-step pipeline — validate, build operation graph, create execution plans, check support, and build plans — on a single Colab GPU, executing against a variant pack of pointers and validating each result against a PyTorch reference. Topics progress from a single fused convolution to autotuning across engine configs, FP8-style epilogues, attention, plan serialization, dynamic shapes, and CUDA graph capture.

In this tutorial https://github.com/MARKTECHPOST-AI-MEDIA-INC/AI-Agents-Projects-Tutorials/blob/main/Deep%20Learning/cudnn frontend nvidia tutorial Marktechpost.ipynb , we work through the cuDNN Frontend https://github.com/NVIDIA/cudnn-frontend ‘s graph API from below the framework: we describe a computation as a graph of operations, let cuDNN pick an engine to run it, and then take control of that choice ourselves. Every kernel we build here is expressed the same way: we declare tensors by their dimensions and strides, chain operations onto them, run the five-step build pipeline of validate, build operation graph, create execution plans, check support, and build plans, and then execute against a variant pack of pointers. We run it all on a single Colab GPU, checking each result against a PyTorch reference so we can see both that the fusion is correct and what it costs. The topics build on each other, moving from a single fused convolution to autotuning across engine configs, FP8-style epilogues, attention, plan serialization, dynamic shapes, and CUDA graph capture. python import os import sys import glob import math import time import ctypes import traceback import subprocess RESULTS = {} def banner title : print "\n" + "=" 78 print title print "=" 78 def section name : def wrap fn : def run a, kw : banner name try: out = fn a, kw RESULTS name = out if isinstance out, str else "ok" return out except Exception as e: RESULTS name = f"SKIPPED / FAILED - {type e . name }: {e}" print f"\n {name} did not complete: {type e . name }: {e}" traceback.print exc limit=3 return None return run return wrap banner "0. Install nvidia-cudnn-frontend and locate libcudnn" subprocess.run sys.executable, "-m", "pip", "install", "-q", "nvidia-cudnn-frontend" , check=True, import torch assert torch.cuda.is available , "No GPU. Runtime - Change runtime type - GPU." torch.backends.cudnn.enabled = True = torch.nn.functional.conv2d torch.randn 1, 1, 8, 8, device="cuda" , torch.randn 1, 1, 3, 3, device="cuda" torch.cuda.synchronize try: import nvidia.cudnn libdir = os.path.join os.path.dirname nvidia.cudnn. file , "lib" os.environ "CUDNN PATH" = os.path.dirname nvidia.cudnn. file os.environ "LD LIBRARY PATH" = libdir + ":" + os.environ.get "LD LIBRARY PATH", "" for so in sorted glob.glob os.path.join libdir, "libcudnn .so " : try: ctypes.CDLL so, mode=ctypes.RTLD GLOBAL except OSError: pass except Exception as e: print f" no pip cuDNN package found, relying on system cuDNN: { e} " import cudnn print " cuDNN frontend imported successfully." banner "1. Environment" DEV = torch.device "cuda" MAJOR, MINOR = torch.cuda.get device capability SM = MAJOR 10 + MINOR CUDNN VER = cudnn.backend version print f" GPU : {torch.cuda.get device name 0 }" print f" Compute capability : sm {SM}" print f" Torch / CUDA : {torch. version } / {torch.version.cuda}" print f" cuDNN backend : {CUDNN VER}" try: print f" cuDNN version str : {cudnn.backend version string }" except Exception: pass DTYPE = torch.bfloat16 if SM = 80 else torch.float16 HAS SDPA = SM = 80 print f" Working dtype : {DTYPE}" print f" Fused SDPA usable : {HAS SDPA}" HANDLE = cudnn.create handle TORCH2CUDNN = { torch.float16: cudnn.data type.HALF, torch.bfloat16: cudnn.data type.BFLOAT16, torch.float32: cudnn.data type.FLOAT, torch.int32: cudnn.data type.INT32, torch.int64: cudnn.data type.INT64, torch.int8: cudnn.data type.INT8, torch.uint8: cudnn.data type.UINT8, } def tensor of graph, t, name : return graph.tensor name=name, dim=list t.size , stride=list t.stride , data type=TORCH2CUDNN t.dtype , def scalar of graph, name : return graph.tensor name=name, dim= 1, 1, 1 , stride= 1, 1, 1 , data type=cudnn.data type.FLOAT, is pass by value=True, def build graph, heur=None, policy=None : heur = heur or cudnn.heur mode.A, cudnn.heur mode.FALLBACK graph.validate graph.build operation graph graph.create execution plans heur graph.check support if policy is None: graph.build plans else: graph.build plans policy return graph def workspace for graph : n = graph.get workspace size return torch.empty max n, 1 , device=DEV, dtype=torch.uint8 def bench fn, warmup=10, iters=50 : for in range warmup : fn torch.cuda.synchronize s, e = torch.cuda.Event True , torch.cuda.Event True s.record for in range iters : fn e.record torch.cuda.synchronize return s.elapsed time e / iters def tflops flops, ms : return flops / ms 1e-3 / 1e12 def report tag, ms, flops=None : extra = f" {tflops flops, ms :7.2f} TFLOP/s " if flops else "" print f" {tag:<34s} {ms:8.3f} ms{extra}" We start by installing nvidia-cudnn-frontend and solving the problem that trips up most first runs: making libcudnn.so visible to the frontend’s dynamic loader. We force PyTorch to load its bundled cuDNN first and then preload the shared objects explicitly, so the frontend’s own dlopen resolves against a library already resident in the process. We then report the compute capability, pick bfloat16 or float16 accordingly, create the cuDNN handle, and define the helpers for tensor description, graph building, workspace allocation, and event-based benchmarking that the rest of the notebook reuses. N, C, H, W = 32, 128, 56, 56 K, R, S = 256, 3, 3 PAD, STR, DIL = 1, 1, 1 P = H + 2 PAD - DIL R - 1 - 1 // STR + 1 Q = W + 2 PAD - DIL S - 1 - 1 // STR + 1 CONV FLOPS = 2 N K P Q C R S CONV STATE = {} @section "2. Fused Conv - Bias - ReLU" def conv fusion : x = torch.randn N, C, H, W, device=DEV, dtype=DTYPE .to memory format=torch.channels last w = torch.randn K, C, R, S, device=DEV, dtype=DTYPE .to memory format=torch.channels last b = torch.randn 1, K, 1, 1, device=DEV, dtype=DTYPE y = torch.empty N, K, P, Q, device=DEV, dtype=DTYPE .to memory format=torch.channels last g = cudnn.pygraph handle=HANDLE, name="conv bias relu", io data type=TORCH2CUDNN DTYPE , intermediate data type=cudnn.data type.FLOAT, compute data type=cudnn.data type.FLOAT, X = tensor of g, x, "X" Wt = tensor of g, w, "W" Bt = tensor of g, b, "bias" conv = g.conv fprop image=X, weight=Wt, padding= PAD, PAD , stride= STR, STR , dilation= DIL, DIL , compute data type=cudnn.data type.FLOAT, biased = g.bias input=conv, bias=Bt Y = g.relu input=biased Y.set output True .set data type TORCH2CUDNN DTYPE Y.set dim list y.size .set stride list y.stride t0 = time.perf counter build g build ms = time.perf counter - t0 1e3 ws = workspace for g pack = {X: x, Wt: w, Bt: b, Y: y} g.execute pack, ws torch.cuda.synchronize ref = torch.relu torch.nn.functional.conv2d x, w, bias=b.flatten , padding=PAD err = y.float - ref.float .abs .max .item scale = ref.float .abs .max .item print f" problem : N{N} C{C} {H}x{W} - K{K} {R}x{S} {DTYPE} " print f" build : {build ms:.1f} ms workspace: {ws.numel /1024:.1f} KiB" print f" max |err|: {err:.4f} ref max {scale:.2f}, rel {err/max scale,1e-9 :.2e} " assert err / max scale, 1e-9 < 5e-2, "numerical mismatch vs PyTorch" ms cudnn = bench lambda: g.execute pack, ws ms torch = bench lambda: torch.relu torch.nn.functional.conv2d x, w, bias=b.flatten , padding=PAD print report "cuDNN FE single fused kernel ", ms cudnn, CONV FLOPS report "PyTorch conv+bias, then relu ", ms torch, CONV FLOPS print f" speedup: {ms torch/ms cudnn:.2f}x" CONV STATE.update graph=g, pack=pack, ws=ws, x=x, w=w, b=b, y=y return f"{ms cudnn:.3f} ms, {tflops CONV FLOPS, ms cudnn :.1f} TFLOP/s" conv fusion We build our first graph, a convolution followed by a bias add and a ReLU, all fused into a single kernel. We keep every tensor in channels last because that is what gives cuDNN the NHWC strides its tensor-core engines want, and we pin the output dimensions and strides explicitly so the result is written back in the same layout. We validate the output against torch.nn.functional.conv2d, then benchmark the fused graph against PyTorch running the convolution and activation as separate kernels. @section "3. Autotuning: build ALL plans, time each engine config" def autotune : x, w, b, y = CONV STATE "x" , CONV STATE "w" , CONV STATE "b" , CONV STATE "y" g = cudnn.pygraph handle=HANDLE, name="conv autotune", io data type=TORCH2CUDNN DTYPE , intermediate data type=cudnn.data type.FLOAT, compute data type=cudnn.data type.FLOAT, X = tensor of g, x, "X" Wt = tensor of g, w, "W" Bt = tensor of g, b, "bias" Y = g.relu input=g.bias input=g.conv fprop image=X, weight=Wt, padding= PAD, PAD , stride= STR, STR , dilation= DIL, DIL , compute data type=cudnn.data type.FLOAT , bias=Bt Y.set output True .set data type TORCH2CUDNN DTYPE Y.set dim list y.size .set stride list y.stride g.validate g.build operation graph g.create execution plans cudnn.heur mode.A, cudnn.heur mode.B, cudnn.heur mode.FALLBACK g.check support g.build plans cudnn.build plan policy.ALL n plans = g.get execution plan count print f" {n plans} candidate engine configs survived support checks\n" pack = {X: x, Wt: w, Bt: b, Y: y} timings = for i in range n plans : try: g.build plan at index i ws sz = max g.get workspace size plan at index i , 1 ws = torch.empty ws sz, device=DEV, dtype=torch.uint8 ms = bench lambda: g.execute plan at index pack, ws, i , warmup=3, iters=15 timings.append ms, i, ws sz print f" plan {i: 3d}: {ms:8.3f} ms " f"{tflops CONV FLOPS, ms :7.2f} TFLOP/s ws={ws sz/1024:8.1f} KiB" except Exception as e: print f" plan {i: 3d}: unusable {type e . name } " assert timings, "no plan executed" timings.sort best ms, best i, best ws = timings 0 worst ms = timings -1 0 print f"\n fastest = plan {best i} @ {best ms:.3f} ms" print f" slowest = {worst ms:.3f} ms - {worst ms/best ms:.1f}x spread across engines" print " Takeaway: heuristics are good, but for a hot shape you ship the" print " autotuned index or the serialized plan from section 6 ." return f"best plan {best i} @ {best ms:.3f} ms {worst ms/best ms:.1f}x spread " autotune We rebuild the same convolution but stop trusting the heuristic, asking for plans from heuristic modes A, B, and FALLBACK and compiling all of them with build plan policy.ALL. We then walk the plan list, build each config, allocate its specific workspace, and time it with execute plan at index, printing throughput and workspace size for every candidate. The spread between the fastest and slowest engine is the point of the exercise, because it tells us how much we gain by shipping an autotuned index instead of accepting the default pick. php @section "4. Matmul - scale - bias - activation - AMAX" def matmul epilogue : Bsz, M, Kd, Nd = 16, 512, 1024, 512 MM FLOPS = 2 Bsz M Nd Kd a = torch.randn Bsz, M, Kd, device=DEV, dtype=DTYPE bm = torch.randn Bsz, Kd, Nd, device=DEV, dtype=DTYPE bias = torch.randn 1, 1, Nd, device=DEV, dtype=DTYPE out = torch.empty Bsz, M, Nd, device=DEV, dtype=DTYPE amax = torch.empty 1, 1, 1, device=DEV, dtype=torch.float32 alpha val = 0.125 alpha = torch.full 1, 1, 1 , alpha val, dtype=torch.float32 g = cudnn.pygraph handle=HANDLE, name="matmul epilogue", io data type=TORCH2CUDNN DTYPE , intermediate data type=cudnn.data type.FLOAT, compute data type=cudnn.data type.FLOAT, A = tensor of g, a, "A" Bt = tensor of g, bm, "B" BIAS = tensor of g, bias, "bias" ALPHA = scalar of g, "alpha" acc = g.matmul A=A, B=Bt, compute data type=cudnn.data type.FLOAT scaled = g.mul a=acc, b=ALPHA biased = g.bias input=scaled, bias=BIAS act name = "relu" if hasattr g, "gelu" : try: act = g.gelu input=biased act name = "gelu" except Exception: act = g.relu input=biased else: act = g.relu input=biased print f" activation used: {act name}" OUT = act OUT.set output True .set data type TORCH2CUDNN DTYPE have amax = True try: AMAX = g.reduction input=act, mode=cudnn.reduction mode.AMAX, compute data type=cudnn.data type.FLOAT AMAX.set output True .set data type cudnn.data type.FLOAT AMAX.set dim 1, 1, 1 .set stride 1, 1, 1 except Exception as e: have amax = False print f" AMAX reduction unavailable here: {e} " build g ws = workspace for g pack = {A: a, Bt: bm, BIAS: bias, ALPHA: alpha, OUT: out} if have amax: pack AMAX = amax g.execute pack, ws torch.cuda.synchronize ref = torch.matmul a.float , bm.float alpha val + bias.float ref = torch.nn.functional.gelu ref if act name == "gelu" else torch.relu ref rel = out.float - ref .abs .max / ref.abs .max .item print f" shape : {Bsz},{M},{Kd} x {Bsz},{Kd},{Nd} " print f" rel err : {rel:.2e}" if have amax: print f" fused AMAX {amax.item :.4f} vs torch {ref.abs .max .item :.4f}" ms = bench lambda: g.execute pack, ws def torch ref : r = torch.baddbmm bias.expand Bsz, M, Nd , a, bm, beta=1.0, alpha=alpha val r = torch.nn.functional.gelu r if act name == "gelu" else torch.relu r return r.abs .amax ms t = bench torch ref print report "cuDNN FE one fused kernel ", ms, MM FLOPS report "PyTorch bmm + act + amax ", ms t, MM FLOPS print f" speedup: {ms t/ms:.2f}x -- the win is the epilogue traffic, not the GEMM" return f"{ms:.3f} ms, {tflops MM FLOPS, ms :.1f} TFLOP/s, {ms t/ms:.2f}x vs torch" matmul epilogue We move to a batched matmul and hang a full epilogue off it: an alpha scale supplied as a pass-by-value host scalar, a bias add, an activation, and an AMAX reduction over the result. The AMAX in the same kernel is the pattern that FP8 training relies on, since it collects the scale factor for the next quantization step without a second pass over the output. We compare against a PyTorch chain of baddbmm, activation, and amax, which makes clear that the speedup comes from eliminating epilogue memory traffic rather than from a faster GEMM. python @section "5. SDPA Flash Attention with causal masking" def sdpa demo : if not HAS SDPA: raise RuntimeError f"fused SDPA needs SM80+ Ampere , this GPU is sm {SM}" b, h, s, d = 4, 16, 1024, 64 scale = 1.0 / math.sqrt d SDPA FLOPS = 4 b h s s d 0.5 q = torch.randn b, h, s, d, device=DEV, dtype=DTYPE k = torch.randn b, h, s, d, device=DEV, dtype=DTYPE v = torch.randn b, h, s, d, device=DEV, dtype=DTYPE o = torch.empty b, h, s, d, device=DEV, dtype=DTYPE g = cudnn.pygraph handle=HANDLE, name="sdpa", io data type=TORCH2CUDNN DTYPE , intermediate data type=cudnn.data type.FLOAT, compute data type=cudnn.data type.FLOAT, Q, Kt, V = tensor of g, q, "Q" , tensor of g, k, "K" , tensor of g, v, "V" causal = True try: O, stats = g.sdpa name="sdpa", q=Q, k=Kt, v=V, is inference=True, attn scale=scale, use causal mask=True except TypeError: try: O, stats = g.sdpa name="sdpa", q=Q, k=Kt, v=V, is inference=True, attn scale=scale, diagonal alignment=cudnn.diagonal alignment.TOP LEFT, right bound=0 except Exception: causal = False O, stats = g.sdpa name="sdpa", q=Q, k=Kt, v=V, is inference=True, attn scale=scale print f" causal masking: {causal}" O.set output True .set data type TORCH2CUDNN DTYPE O.set dim list o.size .set stride list o.stride build g ws = workspace for g pack = {Q: q, Kt: k, V: v, O: o} g.execute pack, ws torch.cuda.synchronize ref = torch.nn.functional.scaled dot product attention q, k, v, is causal=causal, scale=scale rel = o.float - ref.float .abs .max / ref.float .abs .max .item print f" shape : b{b} h{h} s{s} d{d} workspace {ws.numel /1024:.1f} KiB" print f" rel err : {rel:.2e}" ms = bench lambda: g.execute pack, ws ms t = bench lambda: torch.nn.functional.scaled dot product attention q, k, v, is causal=causal, scale=scale print report "cuDNN FE SDPA", ms, SDPA FLOPS report "torch SDPA backend's choice ", ms t, SDPA FLOPS print " Note: torch may already be dispatching to cuDNN or FlashAttention," print " so parity here is the expected, healthy outcome." return f"{ms:.3f} ms, {tflops SDPA FLOPS, ms :.1f} TFLOP/s" sdpa demo @section "6. Serialize a built graph, reload it, execute by UID" def serialization : Bsz, M, Kd, Nd = 8, 256, 512, 256 a = torch.randn Bsz, M, Kd, device=DEV, dtype=DTYPE bm = torch.randn Bsz, Kd, Nd, device=DEV, dtype=DTYPE out = torch.empty Bsz, M, Nd, device=DEV, dtype=DTYPE UID A, UID B, UID C = 1, 2, 3 g = cudnn.pygraph handle=HANDLE, name="serializable mm", io data type=TORCH2CUDNN DTYPE , intermediate data type=cudnn.data type.FLOAT, compute data type=cudnn.data type.FLOAT, A = tensor of g, a, "A" .set uid UID A Bt = tensor of g, bm, "B" .set uid UID B C = g.matmul A=A, B=Bt, compute data type=cudnn.data type.FLOAT C.set output True .set data type TORCH2CUDNN DTYPE .set uid UID C t0 = time.perf counter build g cold ms = time.perf counter - t0 1e3 blob = g.serialize print f" cold build : {cold ms:.1f} ms" print f" serialized plan : {len blob } bytes cache this to disk / ship it " t0 = time.perf counter g2 = cudnn.pygraph try: g2.deserialize HANDLE, blob except TypeError: g2.deserialize blob warm ms = time.perf counter - t0 1e3 print f" deserialize : {warm ms:.1f} ms - {cold ms/max warm ms,1e-6 :.1f}x faster startup" ws = torch.empty max g2.get workspace size , 1 , device=DEV, dtype=torch.uint8 g2.execute {UID A: a, UID B: bm, UID C: out}, ws, handle=HANDLE torch.cuda.synchronize ref = torch.bmm a.float , bm.float rel = out.float - ref .abs .max / ref.abs .max .item print f" rel err after reload: {rel:.2e}" return f"{len blob } B blob, reload {cold ms/max warm ms,1e-6 :.1f}x faster than rebuild" serialization We build a fused scaled dot-product attention graph with causal masking and check it against torch.nn.functional.scaled dot product attention, guarding the whole section behind an SM80 check because the fused kernels need Ampere or newer. We write the causal argument with fallbacks, since the frontend has moved from use causal mask toward diagonal alignment and bound arguments across its 1.x releases. We then serialize a built matmul graph to bytes, reload it into a fresh graph object, and execute it via integer UIDs, which lets us skip the compilation cost entirely at process startup. python @section "7. Dynamic shapes with a shared kernel cache" def dynamic shapes : kc = cudnn.create kernel cache def make n : x = torch.randn n, 64, 32, 32, device=DEV, dtype=DTYPE .to memory format=torch.channels last w = torch.randn 64, 64, 3, 3, device=DEV, dtype=DTYPE .to memory format=torch.channels last y = torch.empty n, 64, 32, 32, device=DEV, dtype=DTYPE .to memory format=torch.channels last g = cudnn.pygraph handle=HANDLE, name=f"dyn {n}", io data type=TORCH2CUDNN DTYPE , intermediate data type=cudnn.data type.FLOAT, compute data type=cudnn.data type.FLOAT, kernel cache=kc, is dynamic shape enabled=True, X, Wt = tensor of g, x, "X" , tensor of g, w, "W" Y = g.conv fprop image=X, weight=Wt, padding= 1, 1 , stride= 1, 1 , dilation= 1, 1 , compute data type=cudnn.data type.FLOAT Y.set output True .set data type TORCH2CUDNN DTYPE Y.set dim list y.size .set stride list y.stride t0 = time.perf counter build g ms = time.perf counter - t0 1e3 ws = workspace for g g.execute {X: x, Wt: w, Y: y}, ws torch.cuda.synchronize return ms times = n, make n for n in 8, 16, 24, 32 for n, ms in times: print f" batch {n: 3d}: build {ms:7.1f} ms" first, rest = times 0 1 , m for , m in times 1: print f"\n first shape {first:.1f} ms, later shapes avg {sum rest /len rest :.1f} ms" print " The cache lets shape-variant graphs reuse an already-JIT'd kernel," print " which is what keeps variable batch/seqlen serving out of rebuild hell." return f"first {first:.0f} ms vs subsequent {sum rest /len rest :.0f} ms" dynamic shapes @section "8. CUDA Graph capture around a cuDNN execution plan" def cuda graph capture : if not CONV STATE: raise RuntimeError "section 2 did not run, nothing to capture" g, pack, ws = CONV STATE "graph" , CONV STATE "pack" , CONV STATE "ws" eager ms = bench lambda: g.execute pack, ws side = torch.cuda.Stream side.wait stream torch.cuda.current stream with torch.cuda.stream side : cudnn.set stream handle=HANDLE, stream=side.cuda stream for in range 3 : g.execute pack, ws, handle=HANDLE torch.cuda.current stream .wait stream side torch.cuda.synchronize cg = torch.cuda.CUDAGraph with torch.cuda.graph cg : cudnn.set stream handle=HANDLE, stream=torch.cuda.current stream .cuda stream g.execute pack, ws, handle=HANDLE cudnn.set stream handle=HANDLE, stream=torch.cuda.current stream .cuda stream replay ms = bench lambda: cg.replay report "plain execute ", eager ms report "cuda graph replay ", replay ms print f" launch overhead removed: { eager ms-replay ms 1e3:.1f} us/iter" print " Pointers are frozen at capture time -- reuse the same buffers and" print " copy new data into them, or re-capture." return f"{eager ms:.3f} - {replay ms:.3f} ms via replay" cuda graph capture banner "SUMMARY" for name, res in RESULTS.items : print f" {name:<58s} {res}" print """ Where to go next - samples/python in the repo: FP8/MXFP8 attention, paged KV cache, MoE grouped GEMM - python/cudnn/: the open-sourced CuTe DSL kernels SDPA, grouped GEMM + SwiGLU, block-sparse and native sparse attention you can read and modify - debugging: CUDNN FRONTEND LOG INFO=1 and CUDNN FRONTEND LOG FILE=stdout use level 10 during CUDA graph capture -- level 1 dumps tensors and is not capture-safe """ We finish with two production concerns. First, we share a kernel cache across four graphs that differ only in batch size and time each build, so we can see later shapes reuse an already compiled kernel instead of paying the JIT cost again. Then we capture the convolution plan inside a CUDA graph, setting the cuDNN handle’s stream to the capture stream. So the work lands in the graph, and we measure how much per-iteration launch overhead the replay removes. In conclusion, what we built here was small in code but broad in scope: a convolution, a matmul, and an attention kernel, each expressed as a graph rather than a library call. Working at that level changed what we could decide. We chose which operations collapsed into a single kernel, so the bias adds, activations, and AMAX reductions we folded into the epilogues never wrote an intermediate to memory. We chose the engine ourselves instead of accepting a heuristic, and timing every candidate config told us what that choice was worth. We also chose when to pay for compilation, pushing it out of the hot path with serialized plans, a kernel cache shared across shapes, and CUDA graph capture. The checks against PyTorch mattered as much as the timings, since the places where we merely matched it were usually places where PyTorch was already calling cuDNN underneath. That marked out where this API earns its keep: fusions with no framework-level equivalent, shapes hot enough to justify autotuning, and small kernels where startup and launch costs dominate. Check out the FULL CODES here https://github.com/MARKTECHPOST-AI-MEDIA-INC/AI-Agents-Projects-Tutorials/blob/main/Deep%20Learning/cudnn frontend nvidia tutorial Marktechpost.ipynb . All credit goes to the researcher of this project. Also, feel free to follow us on Twitter https://x.com/intent/follow?screen name=marktechpost and don’t forget to join our 150k+ML SubReddit https://www.reddit.com/r/machinelearningnews/ and Subscribe to our Newsletter https://magic.beehiiv.com/v1/f5e63dd4-5653-4f09-83e2-321a8b1ba526?email={{email}} . Wait are you on telegram? now you can join us on telegram as well. https://t.me/machinelearningresearchnews Need to partner with us for promoting your GitHub Repo OR Hugging Face Page OR Product Release OR Webinar etc.? Connect with us https://forms.gle/wbash1wF6efRj8G58 Sana Hassan, a consulting intern at Marktechpost and dual-degree student at IIT Madras, is passionate about applying technology and AI to address real-world challenges. With a keen interest in solving practical problems, he brings a fresh perspective to the intersection of AI and real-life solutions.