{"slug": "anatomy-of-a-cuda-binary", "title": "Anatomy of a CUDA Binary", "summary": "NVIDIA's CUDA binary format (cubin) is an ELF64 file with undocumented NVIDIA-specific sections that encode kernel machine code, parameter layout, and metadata. A reverse-engineering analysis reveals the section layout, EIATTR metadata encoding, and constant bank conventions for sm_100 (B200) hardware, based on validation against ptxas output. The findings are critical for developers building custom cubin emitters or understanding CUDA driver requirements.", "body_md": "# Anatomy of a CUDA Binary\n\nWhen you compile a CUDA kernel, the final artifact is a **cubin** — a CUDA binary. It is a standard ELF64 file with NVIDIA-specific sections that encode everything the CUDA driver needs to load and launch a kernel: the machine code, the parameter layout, register allocation metadata, and a collection of attributes that have no public documentation.\n\nThis post walks through the section layout of a cubin, explains the `.nv.info`\n\nmetadata format that makes a kernel self-describing, and shows how these pieces connect at the byte level. The material is drawn from building a from-scratch cubin emitter and validating it against `ptxas`\n\noutput on B200 silicon.\n\nA note on methodology:Everything in this article is based on my analysis of cubins produced by`ptxas`\n\nand validated on real hardware. Nvidia does not publish a specification for the cubin section layout, the`.nv.info`\n\nEIATTR encoding, or the constant bank parameter conventions described here. The structures and semantics are reverse-engineered from compiled binaries. Readers should verify against their own cubins and toolkit versions.\n\n## The ELF Container\n\nA cubin is an ELF64 executable. The header identifies it:\n\n```\n1\n2\n3\n4\n5\ne_ident[EI_OSABI]    = 0x41       (CUDA ABI, not the older 0x33)\ne_ident[EI_ABIVERSION] = 8\ne_type               = ET_EXEC    (loadable, not relocatable)\ne_machine            = EM_CUDA    (0xBE)\ne_flags              = 0x06006402 (for sm_100, 64-bit addressing)\n```\n\nThe `e_flags`\n\nfield encodes the SM architecture in bits 8–15. For `sm_100`\n\n(B200), that is `0x64`\n\n= 100 decimal. Bit 1 marks 64-bit addressing. Bits 24–26 carry a format version that the driver checks.\n\nOlder cubins used `ELFOSABI_CUDA = 0x33`\n\nwith ABI version 0. The CUDA 13 driver rejects these — `cuModuleGetFunction`\n\nreturns `CUDA_ERROR_NOT_SUPPORTED`\n\n. If you are emitting cubins from scratch, the ABI version matters.\n\n## Section Layout\n\nA single-kernel cubin contains roughly twelve sections. We will use a minimal kernel as the running example — an integer add, written in LLVM IR:\n\n```\n1\n2\n3\n4\ndefine i32 @add(i32 %a, i32 %b) {\n  %c = add i32 %a, %b\n  ret i32 %c\n}\n```\n\nCompiling this with `llc -march=sass -filetype=obj`\n\nproduces a relocatable CUDA ELF. After finalization, the cubin contains these sections:\n\n| # | Section | Type | Purpose |\n|---|---|---|---|\n| 0 | (null) | `SHT_NULL` | ELF convention |\n| 1 | `.text.add` | `SHT_PROGBITS` | SASS machine code (128-bit instructions) |\n| 2 | `.nv.constant0.add` | `SHT_PROGBITS` | Constant bank 0 (parameter region) |\n| 3 | `.nv.info` | `SHT_LOPROC` | Module-level metadata (EIATTR stream) |\n| 4 | `.nv.info.add` | `SHT_LOPROC` | Per-kernel metadata (EIATTR stream) |\n| 5 | `.note.nv.cuver` | `SHT_NOTE` | CUDA version note |\n| 6 | `.note.nv.tkinfo` | `SHT_NOTE` | Toolkit release note |\n| 7 | `.nv.compat` | `0x70000086` | SM compatibility descriptor |\n| 8 | `.nv.callgraph` | `0x70000001` | Intra-module call graph |\n| 9 | `.symtab` | `SHT_SYMTAB` | Symbol table |\n| 10 | `.strtab` | `SHT_STRTAB` | String table |\n| 11 | `.shstrtab` | `SHT_STRTAB` | Section-name string table |\n\nThe sections fall into four categories: the executable code, the kernel ABI metadata, driver compatibility notes, and the standard ELF bookkeeping.\n\nThis is the minimal set. A real cubin produced by `ptxas`\n\nfor a production kernel is larger — typically ~22 sections, 8 symbols, and 5 program headers. One notable addition is `.nv.shared.reserved.0`\n\n, a `SHT_NOBITS`\n\nsection (64 bytes) that reserves shared memory space. It comes with two weak symbols (`.nv.reservedSmem.offset0`\n\nand `__nv_reservedSMEM_offset_0_alias`\n\n) and its own `PT_LOAD`\n\nprogram header. Other additional sections include `.nv.shared.<kernel>`\n\n(shared memory allocation), `.nv.global`\n\n(global variable metadata), `.nv.constant2.<kernel>`\n\n(compiler-generated constant data), `.debug_*`\n\nsections (DWARF debug info when compiled with `-G`\n\n), and `.rel.*`\n\nrelocation sections. The table above covers the sections that are structurally required for any kernel to load and execute.\n\n`.text.<kernel>`\n\n— The Machine Code\n\nThe `.text.add`\n\nsection contains the SASS instructions. Each instruction is a 128-bit (16-byte) word. The section flags are `SHF_ALLOC | SHF_EXECINSTR`\n\n— `SHF_ALLOC`\n\nmeans the section occupies memory at runtime (the driver must load it onto the GPU), and `SHF_EXECINSTR`\n\nmarks it as containing executable machine instructions. The section is aligned to 128 bytes.\n\n```\n1\n2\n3\nBits   0..104:  instruction body (opcode, operands, modifiers)\nBits 105..121:  scheduling control (stall, yield, barriers)\nBits 122..125:  operand reuse flags\n```\n\nThe kernel’s entry symbol is a `STT_FUNC`\n\nin the symtab with `st_other = 0x10`\n\n(`STO_CUDA_ENTRY`\n\n), pointing at the start of this section. The driver uses `STO_CUDA_ENTRY`\n\nto distinguish kernel entry points from device functions.\n\n`.nv.constant0.<kernel>`\n\n— The Constant Bank\n\nKernel parameters are passed through constant bank 0. The section `.nv.constant0.add`\n\nis a `SHT_PROGBITS`\n\nsection sized to cover the parameter region:\n\n```\n1\nsize = param_base + param_size\n```\n\nThe `param_base`\n\nis set by the CUDA toolkit — not a fixed ISA constant. The values observed on current toolkits are:\n\n| SM range (current toolkit) | `param_base` |\n|---|---|\n| sm_70 – sm_89 (Volta through Ampere) | `0x160` (352 bytes) |\n| sm_90 (Hopper) | `0x210` (528 bytes) |\n| sm_100 – sm_12x (Blackwell) | `0x380` (896 bytes) |\n\nThese values can shift across toolkit versions for the same architecture. We have observed a sibling slot in the constant bank (the memory-descriptor offset) move from `0x208`\n\nto `0x220`\n\nfor sm_90 between CUDA 11 and CUDA 12, confirming that the layout is a toolkit convention, not an architectural invariant.\n\nThe first `param_base`\n\nbytes are reserved for driver-managed metadata (grid dimensions, block dimensions, shared memory size, etc.). User-specified kernel parameters are laid out contiguously starting at `param_base`\n\n, each aligned to its ABI alignment.\n\nFor a kernel `add(int a, int b)`\n\n, the two 4-byte parameters occupy offsets `0x0`\n\nand `0x4`\n\nrelative to `param_base`\n\n. The total section size is `0x380 + 8 = 0x388`\n\non sm_100.\n\nThe code reads parameters with `LDC`\n\n(load constant) instructions:\n\n```\n1\n2\nLDC R0, c[0x0][param_base + 0x0]   // load 'a'\nLDC R1, c[0x0][param_base + 0x4]   // load 'b'\n```\n\nThe `.nv.info`\n\nmetadata, the constant bank section, and the `LDC`\n\ninstructions must all agree on the base offset. If they disagree, the driver copies launch arguments to one offset and the kernel reads from another — wrong results, no crash, no diagnostic.\n\n`.nv.info`\n\n— The EIATTR Metadata Format\n\nThe `.nv.info`\n\nsections are the most opaque part of a cubin. They use `SHT_LOPROC`\n\n(`0x70000000`\n\n), a processor-specific section type. The content is a flat stream of **EIATTR** (ELF Info ATTRibute) entries. There is no public documentation for this format.\n\nEach EIATTR entry has a fixed 4-byte header:\n\n```\n1\n2\n3\nbyte 0:  format (EIFMT)\nbyte 1:  attribute code (EIATTR)\nbyte 2-3: value or size (depends on format)\n```\n\nThe format byte determines how the value is encoded:\n\n| EIFMT | Value | Encoding |\n|---|---|---|\n`EIFMT_NVAL` | 1 | No value. Bytes 2–3 are zero padding. |\n`EIFMT_BVAL` | 2 | One inline byte in byte 2. Byte 3 is padding. |\n`EIFMT_HVAL` | 3 | 16-bit inline value in bytes 2–3 (little-endian). |\n`EIFMT_SVAL` | 4 | Bytes 2–3 are a 16-bit size N. N bytes of payload follow immediately. |\n\nThis is a TLV (type-length-value) 1 scheme, except the “length” is implicit for formats 1–3 (always 4 bytes total) and explicit only for format 4.\n\n### Module-Level `.nv.info`\n\nThe module-level `.nv.info`\n\nsection (no kernel suffix) contains attributes keyed by symbol index. For each kernel, it emits three entries:\n\n**REGCOUNT (0x2f):** The number of general-purpose registers the kernel uses. Format is `EIFMT_SVAL`\n\nwith an 8-byte payload: `(symbol_index:u32, regcount:u32)`\n\n.\n\nFor a kernel `add`\n\nthat touches registers `R0`\n\nand `R1`\n\n, the regcount is 2:\n\n```\n1\n2\n3\n4\n5\n6\n04 2f 08 00   03 00 00 00   02 00 00 00\n│  │  │       │             └─ regcount = 2\n│  │  │       └─ symbol index = 3 (the kernel entry)\n│  │  └─ payload size = 8 bytes\n│  └─ EIATTR_REGCOUNT = 0x2f\n└─ EIFMT_SVAL = 4\n```\n\n**FRAME_SIZE (0x11):** Stack frame size in bytes. Same 8-byte keyed format. Zero for a leaf kernel.\n\n**MIN_STACK_SIZE (0x12):** Minimum stack size. Also zero for a leaf kernel.\n\nThe driver uses regcount to compute occupancy — how many thread blocks can run concurrently on one SM. Over-reporting wastes occupancy. Under-reporting causes the hardware to clobber live registers.\n\n### Per-Kernel `.nv.info.<kernel>`\n\nThe per-kernel section carries the kernel’s ABI contract with the driver. The entries, in order:\n\n**SW_INFO (0x37):** Software info. An `EIFMT_SVAL`\n\nwith a 4-byte payload containing the section’s own byte size. Self-referential — you compute the size, write it, and the size includes the bytes you just wrote. In practice, this means building the section twice.\n\n**Attribute 0x35:** An `EIFMT_NVAL`\n\nflag that `ptxas`\n\nemits before `PARAM_CBANK`\n\n. Its exact semantics are unclear; omitting it does not affect loading, but real cubins always include it.\n\n**PARAM_CBANK (0x0a):** Constant bank descriptor. `EIFMT_SVAL`\n\n, 8-byte payload:\n\n```\n1\n(cbank_symbol_index:u32, (param_size << 16) | param_base : u32)\n```\n\nThis tells the driver which symbol table entry identifies the constant bank section, how large the parameter region is, and where it starts within the bank. For `add(int, int)`\n\non sm_100:\n\n```\n1\n2\n3\n4\ncbank_sym = 2          (section symbol for .nv.constant0.add)\nparam_base = 0x380\nparam_size = 8          (two i32s)\npacked = (8 << 16) | 0x380 = 0x00080380\n```\n\n**CBANK_PARAM_SIZE (0x19):** An `EIFMT_HVAL`\n\nwith the parameter region size as a 16-bit value. Redundant with the high half of `PARAM_CBANK`\n\n, but `ptxas`\n\nalways emits both.\n\n**KPARAM_INFO (0x17):** One entry per kernel parameter, emitted in reverse ordinal order (highest ordinal first — this matches `ptxas`\n\noutput). Each is `EIFMT_SVAL`\n\nwith a 12-byte payload:\n\n```\n1\n(index:u32, ordinal:u16, offset:u16, packed:u32)\n```\n\nThe `packed`\n\nfield encodes `(size << 18) | 0x1f000`\n\n. The `0x1f000`\n\nconstant carries the parameter space and log-alignment for by-value parameters. For a 4-byte `int`\n\nparameter: `packed = (4 << 18) | 0x1f000 = 0x0011f000`\n\n.\n\n**MAXREG_COUNT (0x1b):** An `EIFMT_HVAL`\n\nwith the maximum register count hint (usually `0xff`\n\n= no limit).\n\n**EXIT_INSTR_OFFSETS (0x1c):** An `EIFMT_SVAL`\n\nlisting the byte offsets of every `EXIT`\n\ninstruction in the `.text`\n\nsection. The driver likely uses these for pre-emption — knowing where a kernel can cleanly yield control — though the exact use is not documented. Each offset is a `u32`\n\n. For a kernel whose `EXIT`\n\nis the 5th instruction (offset `4 * 16 = 0x40`\n\n):\n\n```\n1\n2\n3\n4\n5\n04 1c 04 00   40 00 00 00\n│  │  │       └─ EXIT at byte offset 0x40\n│  │  └─ payload size = 4\n│  └─ EIATTR_EXIT_INSTR_OFFSETS = 0x1c\n└─ EIFMT_SVAL = 4\n```\n\n## The Note Sections\n\nTwo ELF note sections carry driver compatibility information:\n\n** .note.nv.cuver** (note type\n\n`0x3e8`\n\n): A 12-byte descriptor encoding the SM architecture and a format version. The CUDA driver checks this to determine if the cubin is compatible with the current GPU.** .note.nv.tkinfo** (note type\n\n`0x7d0`\n\n): The toolkit release string. Contains the `ptxas`\n\nversion, the CUDA release string (e.g., “Cuda compilation tools, release 12.8, V12.8.93”), and the compilation flags (`-arch sm_100 -m 64`\n\n). The driver validates this to reject cubins compiled with unsupported toolkit versions.Both use the standard ELF note format: `(namesz, descsz, type)`\n\nheader, followed by the vendor name “NVIDIA Corp” (NUL-terminated, 4-byte padded), followed by the descriptor.\n\n`.nv.callgraph`\n\nand `.nv.compat`\n\n** .nv.callgraph** (type\n\n`0x70000001`\n\n): Encodes the intra-module call graph. For a leaf kernel (no device function calls), this contains four entries: the kernel node (id 0) connected to four sentinel edges (`0xffffffff`\n\nthrough `0xfffffffc`\n\n) that mark it as a call-graph leaf.** .nv.compat** (type\n\n`0x70000086`\n\n): An SM compatibility descriptor. For a single-architecture cubin, this is a fixed 16-byte block encoding feature-level requirements.## The Symbol Table\n\nThe cubin’s `.symtab`\n\ncontains four entries for a single-kernel module:\n\n| Index | Type | Other | Section | Name | Purpose |\n|---|---|---|---|---|---|\n| 0 | `STT_NOTYPE` | 0 | `SHN_UNDEF` | (null) | ELF convention |\n| 1 | `STT_SECTION` | 0 | `.text.add` | — | Section symbol for .text |\n| 2 | `STT_SECTION` | 0 | `.nv.constant0.add` | — | Section symbol for constant bank |\n| 3 | `STT_FUNC` | `0x10` | `.text.add` | `add` | Kernel entry point |\n\nThe `st_other = 0x10`\n\n(`STO_CUDA_ENTRY`\n\n) on the kernel symbol is what distinguishes a kernel entry from a device function. The `.nv.info`\n\nentries reference symbols by index — REGCOUNT references symbol 3, PARAM_CBANK references symbol 2 — so the indices must be stable.\n\n## Putting It Together\n\nHere is the complete flow from source to loaded kernel:\n\n``` php\nflowchart LR\n    A[\".cu source\"] --> B[\"ptxas / llc\"]\n    B --> C[\"cubin (ELF64)\"]\n    C --> D[\"cuModuleLoad\"]\n    D --> E[\"Driver reads sections\"]\n    E --> F[\".nv.info → regcount,<br/>params, exits\"]\n    E --> G[\".nv.constant0 → allocate<br/>param bank\"]\n    E --> H[\".text → load SASS<br/>to SM\"]\n    F --> I[\"Compute occupancy,<br/>register EXIT offsets\"]\n    G --> J[\"Copy launch args<br/>to param_base\"]\n    H --> K[\"Execute\"]\n```\n\nThe driver’s loading sequence:\n\n- Parse the ELF header. Check\n`e_flags`\n\nfor SM compatibility. - Validate the\n`.note.nv.cuver`\n\nand`.note.nv.tkinfo`\n\nnotes. - Read the\n`.nv.info`\n\nstream: extract regcount, frame size, stack size. - Read\n`.nv.info.<kernel>`\n\n: extract parameter layout, constant bank descriptor, EXIT offsets. - Allocate constant bank memory. Copy host-side launch arguments to\n`param_base`\n\nwithin it. - Load\n`.text`\n\ninto instruction memory on the target SM. - Configure the warp scheduler with the register count (for occupancy) and EXIT offsets.\n- Launch.\n\nIf any of these metadata sections are malformed or inconsistent, the behavior ranges from `CUDA_ERROR_INVALID_IMAGE`\n\nto silent wrong results — there is no “metadata validation” pass that checks cross-section consistency before launch.\n\n## What This Means in Practice\n\nFor most CUDA developers, the cubin is an opaque blob that `nvcc`\n\nproduces and the driver consumes. But the structure matters when:\n\n**You want to understand the internals of a SASS binary.** For most CUDA developers, the cubin is a black box. Knowing the section layout — what`.nv.info`\n\nencodes, how the constant bank is sized, where the driver finds EXIT offsets — turns it into something you can read and reason about.**Debugging and performance analysis.** This is a crucial skill for both. When a kernel silently produces wrong results, the`.nv.info`\n\nstream is the authoritative record of what`ptxas`\n\ndecided: how many registers, where the parameters live, which instructions are exits. When occupancy is lower than expected, the regcount in`.nv.info`\n\ntells you why. Reading these sections directly (rather than trusting`cuobjdump`\n\n’s summary) gives you the ground truth.**Portability of compiled cubins.** The constant bank parameter base has changed silently across toolkit versions:`0x160`\n\non Volta through Ampere,`0x210`\n\non Hopper,`0x380`\n\non Blackwell (on current toolkits). None of this is documented, and the values are toolkit conventions that can shift even within a single architecture across CUDA releases. A cubin compiled with one toolkit version cannot be assumed portable to another, because the code (LDC offsets), the`.nv.info`\n\n(PARAM_CBANK), and the`.nv.constant0`\n\nsection size all encode the base independently. If they disagree, the driver copies launch arguments to one offset and the kernel reads from another.\n\n## References\n\n*Disclaimer: This article was generated using the Gemini 3.1 Pro and Claude Opus 4.8 models.*\n\n[CC BY 4.0](https://creativecommons.org/licenses/by/4.0/)by the author.", "url": "https://wpnews.pro/news/anatomy-of-a-cuda-binary", "canonical_source": "https://hiraditya.github.io/posts/anatomy-of-a-cuda-binary/", "published_at": "2026-07-14 19:00:00+00:00", "updated_at": "2026-07-15 14:20:33.385399+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "machine-learning", "ai-infrastructure"], "entities": ["NVIDIA", "CUDA", "ptxas", "B200", "LLVM"], "alternates": {"html": "https://wpnews.pro/news/anatomy-of-a-cuda-binary", "markdown": "https://wpnews.pro/news/anatomy-of-a-cuda-binary.md", "text": "https://wpnews.pro/news/anatomy-of-a-cuda-binary.txt", "jsonld": "https://wpnews.pro/news/anatomy-of-a-cuda-binary.jsonld"}}