CUDA++ for the NVIDIA DGX Spark (GB10, sm_121). It is the GPU half of the Omega runtime. First-party CUDA++ material in this revision is licensed under Apache-2.0 WITH LLVM-exception. You write a GPU program, CUDA++ turns it into the GB10's own machine code and runs it on the Spark's GPU with no CUDA user-space software at build or run time: no CUDA toolkit, no nvcc, no PTX, no libcuda, no cuBLAS. NVIDIA's compiler and disassembler were used offline as references while the encoding was worked out (how I did it). NVIDIA's open kernel module (580.173.02) and the GPU's signed GSP firmware are still in the path. It's a research-stage toolchain and runtime, not a drop-in replacement for all of CUDA.
Who this is for, and what hardware
Supported The NVIDIA DGX Spark. Its GB10 Grace Blackwell chip (GPU compute capability sm_121, PCI id 10de:2e12) is the only GPU CUDA++ has machine code for.Other GB10 systems Not run. CUDA++ recognises the GPU by its PCI id, not by the system maker, so another GB10 system with the same kernel module is the same target to the code. Nobody has run CUDA++ on one. You need Linux on the Spark's ARM CPU (aarch64). NVIDIA's open kernel module, version 580.173.02: make doctor and make hello-gpu refuse another version, and /dev/nvidiactl, /dev/nvidia0 and /dev/nvidia-uvm must exist. NVIDIA's signed GSP firmware, which that module loads into the GPU. git, make and a C compiler. No CUDA toolkit.Not supported Other NVIDIA GPUs: PLANNED, not supported. A target row for the RTX 5070 (sm_120) exists on Omega's branch rig/gb20x (aien-dev/omega-mv, src/omega_bw_target.h), built from public sources and never run on that chip (portability). Every other GPU, NVIDIA or not (AMD, Intel, Apple, Qualcomm, Arm, Imagination and the rest): PLANNED targets, not supported. CUDA++ has machine code, a driver client and receipts for exactly one GPU today, the GB10; the multi-backend split is docs/ARCHITECTURE.md (PROPOSED), and no capability claim about any other GPU is public until a kernel runs on it with a receipt. Unmodified CUDA applications: not supported; a CUDA++ program calls CUDA++'s C kernel builder, and CUDA++ does not compile CUDA C++ or PTX.
Licensing status: This revision uses Apache-2.0 WITH LLVM-exception for first-party material. For the historical MIT version, use 232332422bfdc31c8918085b50f13c1e18ab702e. The old MIT grant and third-party notices are preserved. See the transition and provenance audit.
It runs on the NVIDIA GB10 (the DGX Spark, compute capability sm_121) today. On 2026-10-21 the model Aien serves on the Spark, Qwen3.6-35B-A3B (a mixture-of-experts model with about 3B parameters active per token), was served by the C GPU stack this repository was extracted from (the live build used the Omega repository's copy of these sources). The live server had only the C library, the maths library, libgcc_s and the mapped, and zero CUDA libraries (receipt in Omega: evidence/WHAT-OMEGA-IS-20261021/live-serve-no-cuda.txt). That stack is mostly C. Two pieces of it now run as code written in the Omega language: the sm_121 instruction encoder (om/encoder/bw_enc_sm121.om) and the cubin packer ( om/cubin/cubin_elf.om), compiled by the pinned Omega compiler and linked through an omega-bridge binding. Their C bodies are out of the default build and kept only as differential oracles; the .om encoder is byte-identical to the C one on every kernel the test suite builds (evidence, migration table in CUDApp#79). The rest (kernel builders, register allocation, runtime) is still C, and Omega programs compiled to the GPU by the Omega compiler itself is Omega bar C1, still BUILDING.
CUDA++ keeps growing to cover the jobs of NVIDIA's user-space software on the Spark, and then do things CUDA does not:
- NVIDIA's open kernel modules
nvidia.koandnvidia-uvm.kostay. GB10 support belongs in the upstream Linux drivers Nova and nouveau, not in a third driver; Aien's own driver branch (L332, aienux#441) is a research harness, not a product; - my own disassembler replaces
nvdisasm; - the GSP firmware stays NVIDIA's too, for a physical reason: the GPU checks its signature and boots nothing else;
- beyond CUDA: owning the whole path from program to chip, so one submission per generated word, whole-model kernels that stay resident on the GPU, and zero-copy use of the memory the CPU and GPU share.
CUDA++ is not affiliated with or endorsed by NVIDIA; CUDA is NVIDIA's trademark.
New to CUDA++? Start with the first-run and independent verification guide. It separates host-only tests from real GB10 execution, documents the current support boundary, and explains how to interpret receipts. See contributing, security reporting, and the readiness roadmap before relying on it in a deployment.
On this page: the stack · how I did it · the machine code · the driver, part by part · one launch, start to finish · CUDA, piece by piece · speed · run your first kernel
Between a CUDA program and the GPU sit four closed NVIDIA layers: the compiler, the maths libraries, the runtime and the driver library. CUDA++ does their jobs with its own code, at research stage. Under them are NVIDIA's kernel module, whose source NVIDIA publishes, and the firmware inside the GPU. CUDA++ talks to the kernel module directly, and it stays; GB10 support goes upstream to Nova and nouveau.
I built it alone on one DGX Spark in under four weeks, directing AI coding agents. NVIDIA publishes the source of its Linux kernel module, but not its compiler, its driver library or the GPU's machine code. So I skipped every closed layer:
- No libcuda. I wrote my own client for the open kernel module's ioctls, from its MIT headers: client, device, address space, memory, command ring, compute object, doorbell, completion.The driver, part by part shows every piece.
- No nvcc, no PTX. The GPU's machine code is undocumented, so I reverse engineered it: compiled small probe kernels of my own once with NVIDIA's compiler, read the machine code back with NVIDIA's disassembler, compared kernels that differ in one detail to find which bits mean what, cross-checked against Mesa's open NVIDIA compiler, and believed nothing until the chip ran it and gave the right answer. Arithmetic the documents do not pin down (the tensor cores) was measured on the chip until a model matched every output bit.
- My own encoder, runtime and libraries on top: 59 instruction kinds, a register allocator, launch descriptors, command lists, matrix multiply, attention.
I'm not the only one mapping this instruction set in the open, and I credit the others: Mesa's NAK compiler has an SM120 encoder tested against nvdisasm; cubit is an MIT assembler and disassembler for SM120 and SM121a; blackwell-isa publishes an sm121a encoding table; nvidia-sass-document documents SASS encodings; tinygrad's NV backend talks to the same open kernel module interface. Cross-checking CUDA++ against them is PROPOSED.
No NVIDIA binary is in this repository. One listing printed by NVIDIA's nvdisasm is stored as a test pin: the decode of CUDA++'s own trace-emitter words (PROVENANCE.md section 6). The whole method, step by step with the files and receipts: docs/HOW_I_DID_IT.md.
Every GB10 instruction is 16 bytes. The low bits say what to do and to which registers; the top 23 bits tell the warp scheduler how long to wait before the next instruction and which earlier results to wait for. Here is the add from the first kernel, every bit accounted for:
Those scheduling bits are what a compiler normally hides. Slow instructions (memory loads, constant loads, special registers) set one of six scoreboards; a later instruction that needs the result waits on it. Here is the whole first kernel with its waits drawn in:
Getting one of these bits wrong does not fail loudly: the GPU reads a register before the load has landed and returns a wrong number. That is why every encoding has golden words in the tests and a chip run behind it. The figures on this page are drawn from the repository's own data by tools/figures/make_figures.py; the two above decode the words in src/omega_blackwell_encoder.c.
libcuda is closed, but everything it does to the hardware goes through ioctls on three device files, and NVIDIA's open kernel module defines every one of those calls in its public headers. I copied 67 of those headers unchanged (third_party/nvidia-open-580.173.02/, MIT) and wrote my own client against them. This is the whole of it:
| Part | Where | What it does |
|---|---|---|
| Open | nvrm_open() innvrm/nvrm.c |
Opens /dev/nvidiactl and/dev/nvidia-uvm , finds the GPU (NV_ESC_CARD_INFO ), allocates the root client, device and subdevice, reads the class list to pick the Blackwell classes, reads the SM version and GPU UUID, maps the 64 KiB usermode page that holds the doorbell, and creates an address space that the unified-memory driver manages. |
| Memory | nvrm_alloc() ,nvrm_free() |
One system-memory object per buffer. The GB10 has no separate video memory, so each buffer is mapped at the same address for the CPU and the GPU: a pointer means the same thing on both sides and nothing is copied. Anything the CPU polls is allocated GPU-uncached (nvrm_alloc_gpu_uncached() ). |
| Channel | nvrm_channel() |
Channel group, context share, a command ring of 1,024 entries with its USERD control page, and the Blackwell compute object. Asks for the channel's work-submit token, registers the channel with the unified-memory driver and switches scheduling on. |
| Submit | nvrm_enqueue() ,nvrm_ring() |
Writes one 8-byte ring entry (pushbuffer address and length), advances GPPut with a full memory barrier, then rings the doorbell: a single 32-bit store of the token. No system call is made to launch work. |
| Wait | m16/m16_gpu_wait.c |
The one completion wait: polls a word the GPU writes, with a hard deadline that is never extended and a stall deadline that restarts only on progress. |
| Qualification | m16/m16_requalify.c ,m16/m16_concurrent.c |
Proofs on the chip. Requalify queues work without ringing the doorbell and checks nothing happens for 100 ms, rings it, checks the marker turns to0x16c0ffee , and reads/proc/self/maps to provelibcuda was never loaded. Concurrent runs two channels and checks neither can fire the other. |
| Platform contract | src/omega_gpu_platform.h |
What the runtime may ask of the machine: typed handles instead of file descriptors, explicit rights on every buffer, no silent fallback. The Linux client above is one implementation; AIEN's own driver plugs in at the same seam. |
| Session, command lists, engine | src/omega_gpu_session.c ,src/omega_gpu_cmdlist.c ,src/omega_gpu_engine.c |
One device per process, kept open; many launches recorded into one submission; a job runner that returns success or exactly one named failure, and refuses all further GPU work if it cannot tell whether the GPU finished. |
What the headers do not say, I learned on the chip and wrote down as rules, each with a receipt or an issue behind it:
- A word the CPU polls must be GPU-uncached. Otherwise the GPU keeps the new value in its L2 cache and the CPU never sees it change. This stalled the work once, on 2026-10-02; the platform now refuses such a buffer.
- New code needs a shader-cache invalidate before it runs. The GPU's instruction cache does not see CPU writes; after a code address was reused, the chip ran the old instructions. Every launch that uploads code now starts with
INVALIDATE_SHADER_CACHES(receipt FB1-CUT1B-4345406 in Omega). - Dependent launches need a wait-for-idle between them. Without one, two grids on one channel may overlap. The order of the launch commands follows Mesa's NVK driver.
- Blackwell's control page reports no read pointer. The client counts completed ring entries itself and never overruns entries that have not been seen to finish.
- Creating the compute object needs one free, physically contiguous 2 MiB block for its context buffer. On a busy host it is briefly missing, so that one allocation is retried, bounded (omega#523).
Under the client: NVIDIA's open kernel module (nvidia.ko, nvidia-uvm.ko) and the GSP firmware inside the GPU. Both stay: GB10 support goes upstream to Nova and nouveau, Aien's own driver branch (L332) is a research harness only, and the firmware is NVIDIA's because the chip checks its signature. Every object and call in order, with line references: docs/DRIVER.md.
The CPU does five things, all of them plain memory writes except the last, which is a single store to a GPU register. The GPU then does the rest on its own: it fetches the ring entry, reads the commands, reads the launch descriptor (the QMD: grid size, block size, registers, shared memory, code address), runs the grid, and writes two completion markers with a cache flush between them. Results are already in memory the CPU can read. Recording many launches into one submission (src/omega_gpu_cmdlist.c) is how a whole generated word goes to the GPU at once.
| CUDA piece | Its job | CUDA++ replacement | Status |
|---|---|---|---|
| CUDA C++ language | How you write a kernel | The kernel builder (C calls). Omega's languages reaching the GPU is Omega bar C1, built on this toolchain | PARTIAL: C calls only |
| nvcc, NVVM, PTX, ptxas | Compile the kernel to machine code | My own sm_121 encoder and register allocator | PASS: every GPU kernel this stack runs is encoded by it |
| libcuda (driver API) | Talk to the kernel driver | nvrm client, ioctls straight to the open kernel module |
PARTIAL: implemented; M16 historical summary is unauditable here (see below) |
| libcudart (runtime API) | Device, memory, launch, streams | Session, command lists, job engine | PARTIAL: one device, one process; no streams or events yet; a timeout ends GPU use for the process |
| cuBLAS | Matrix multiply | Matmul, tensor-core GEMM, FP8 and BF16 libraries | PARTIAL: shaped for the models AIEN runs; no any-shape GEMM yet |
| cuDNN, FlashAttention | Attention and network layers | Decode and prefill attention | PARTIAL: the fused fast paths are model-shaped; since runtime API 0.6.0 a general fallback ( omega_gpu_attention_general , EXPERIMENTAL) takes head_dim 1..256, any MHA/GQA/MQA ratio, causal or not, sliding windows and any length, chip-exercised on GB10 (evidence/ATTN-GENERAL-20261027-4d51f91f/ ). It is a scalar f32 kernel without tensor cores, and has no ALiBi, softcap or dropout |
| CUB, Thrust | Reductions, scans, sorts | Frozen-order reductions ( evidence/E1-CLOSURE/ ) |
PARTIAL: no general scan or sort |
| cuFFT, cuRAND | FFT, random numbers | none yet | MISSING |
| cuobjdump, nvdisasm | Read machine code back | My own sm_121 disassembler | BUILDING: on Omega branch c6/disasm (19eebe0e) it agrees withnvdisasm on 331 kernels; not in CUDA++ yet. Some host tests here still callnvdisasm as a cross-check when it is installed |
| Nsight | Measure | Ceiling probes: 115.14 TFLOPS tensor-core peak, 195.1 GB/s reads ( evidence/OWN30B-T3-TC-PEAK-BW-5faa0f70-20261013T2236Z/ ) |
PARTIAL: probes, no single tool |
| CUDA samples, docs | Your first kernel | examples/hello_gpu ,docs/runtime/ |
PARTIAL: one example |
nvidia.ko ,nvidia-uvm.ko |
Kernel driver | stays NVIDIA's open kernel module; GB10 support goes upstream to Nova and nouveau (L332 is a research harness) | stays |
| GSP firmware | Runs inside the GPU | stays NVIDIA's (signature-checked by the chip) | not replaceable |
PASS means a receipt exists on a named commit. Status words come from Omega's docs/OMEGA.md, section 2. ROADMAP.md lists every NVIDIA piece on the Spark with its receipt, and the bars CUDA++ must pass before it says "replaces CUDA".
The imported evidence/m16-blackwell-native-path-requalification-receipt.json
is an unauditable historical summary, not qualification of this checkout.
It names implementation commit a2c0d7fbc03f7ce91fcb6389312a0ee8630322cd,
but its evidence/m16-requalification raw logs and superseded receipt are absent
from this repository. Its historical PASS and durability fields are preserved
unchanged, not endorsed as current evidence. No immutable durable source for those
missing logs has been established. Current native qualification is NOT_RUN;
it needs retained raw logs from a new authorized GPU run, not a host-only check.
Every speed number is held against what this machine can physically do, measured by my own probe kernels. Generating a word of text reads each active weight once and does about one operation per byte, far left of the ridge, so its speed is set by memory, not by the tensor cores. The head-to-head measurement against llama.cpp and vLLM on this Spark is Omega bar P0 (AIEN L394) and has not been published. Until it is, this repository claims no speed comparison with CUDA, llama.cpp, vLLM or TensorRT. Its predeclared protocol is in aien-dev/gpu-head-to-head, which pins this repository by commit; the raw logs and results will be published there. Chip-only times (GPU time per token) and end-to-end times (through the HTTP server, prompt included) are different measurements and are always labelled separately; figures other projects publish are theirs, not re-measured here.
| Part | What it does | Where |
|---|---|---|
| GPU toolchain | Writes sm_121 machine code (SASS) directly: a kernel builder with 59 instruction kinds, a register allocator, the instruction encoder, launch descriptors (QMD) | src/omega_blackwell_codegen*.c ,src/omega_blackwell_encoder.c ,src/omega_blackwell_qmd.c |
| GPU runtime | Opens the device, allocates memory, uploads code, launches kernels, records command lists, runs jobs. The job of CUDA's runtime and driver API | src/omega_gpu_session.* ,omega_gpu_cmdlist.* ,omega_gpu_engine.* ,omega_gpu_platform* |
| Libraries | Matrix multiply and matrix-vector (BF16, FP8, 4-bit, ternary), attention for decode and prefill, elementwise, mixture-of-experts routing. The job of cuBLAS and cuDNN | src/omega_gpu_*_api.* |
| Driver client | Talks to NVIDIA's open kernel module through its ioctls; replaces libcuda |
nvrm/ ,m16/ |
| NVIDIA interface headers | The open kernel module's public interface definitions (MIT, unchanged) | third_party/nvidia-open-580.173.02/ |
| Runtime API | The written, versioned API and its manifest check | docs/runtime/API.md ,src/omega_runtime_api.h |
| Figures | Every picture on this page, drawn from the repository's own data | tools/figures/make_figures.py ,docs/img/ |
I read what developers write about CUDA and its libraries on GitHub, Reddit, NVIDIA's forums, the PyTorch forum, Hacker News and Stack Overflow, and ranked the problems by how many people they hurt (GitHub issue counts times how badly they hurt). The full research, with 97 sources and every quote checked word for word against its page, is docs/PAIN_POINTS.md. How CUDA++ itself was built, reverse engineering included, is docs/HOW_I_DID_IT.md.
Each row below gives the pain in a developer's own words, how CUDA++ fixes it, and the command that shows it, with the output it printed on my DGX Spark and the receipt it came from. A row says FIXED only when that command and receipt are on this branch. Where CUDA is still better, docs/PAIN_POINTS.md section 4 says so.
1. Out of memory, and on the DGX Spark a frozen machine (26,390 GitHub issues say "CUDA out of memory")
The pain. "The Spark ships with zero swap. On unified memory a spike goes straight to a global OOM that hard hangs the whole box. No ssh, no ping, needs a power cycle." (dgx-spark-laguna FINDINGS.md)
How CUDA++ fixes it. On the GB10 the CPU and GPU share one memory, and every CUDA++ buffer is part of it. CUDA++ has no caching allocator, so what a process holds is what it asked for. Before any allocation reaches the driver, CUDA++ checks how much memory the machine has free. If the buffer would leave less than a reserve (4 GiB by default, CUDAPP_MEM_RESERVE_MIB to change it, 0 to turn it off), the allocation is refused with a sentence instead of freezing the box.
The command that shows it (host test of the runtime on a simulated driver, run on the Spark):
$ make build/test_omega_gpu_platform_linux && ./build/test_omega_gpu_platform_linux
...
refusal: dma_alloc: refused 2.0 MiB: 4.0 GiB of memory is free and CUDA++ keeps 4.0 GiB free (CUDAPP_MEM_RESERVE_MIB=4096), because the CPU and GPU share it and running it out freezes the machine. Free buffers or use a smaller batch.
session: dma_alloc: refused 2.0 MiB: 4.0 GiB of memory is free and CUDA++ keeps 4.0 GiB free (CUDAPP_MEM_RESERVE_MIB=4096), because the CPU and GPU share it and running it out freezes the machine. Free buffers or use a smaller batch.
PASS D1_MEMORY_RESERVE_REFUSES_BEFORE_THE_DRIVER
test_omega_gpu_platform_linux (reference only, simulated driver): 9 passed, 0 failed
Receipts: evidence/dx/d1-memory/platform-test-spark.txt, and evidence/dx/d1-memory/negative-control.txt (with the check removed, the test fails). FIXED in the runtime, shown on the host. Left: the same refusal on the chip itself, inside a GPU window (docs/issues/D1-memory.md).
The pain. "I don't know a right solution that works for all setups, happy to hear suggestions." (FlashAttention's maintainer, flash-attention #246)
How CUDA++ fixes it. There is no toolkit version to match: compiler, runtime and libraries are one repository at one commit, built with make and a C compiler. The one outside version is NVIDIA's kernel module, whose public headers CUDA++ is built against. make doctor checks it and everything else a first run needs. It reads /proc, /sys and /dev only and opens no device. Every problem gets a line saying what is wrong, why and the command that fixes it. The doctor wrapper refuses a module of another version; make hello-gpu runs that wrapper first. Direct execution of binaries and applications linking the library have no driver-version or PCI-ID admission guard. The diagnostic is not a sandbox or current-commit qualification; callers must perform the compatibility preflight themselves and have an authorized GPU window.
The command that shows it:
The following excerpts are historical doctor output; current output explicitly labels the M16 receipt as a historical summary with raw evidence unavailable.
$ make doctor
cudapp doctor: is this machine ready for CUDA++?
info system: Linux 7.0.0-1019-nvidia aarch64
ok NVIDIA kernel module: open module 580.173.02 (CUDA++ speaks 580.173.02)
ok device files: /dev/nvidiactl /dev/nvidia0 /dev/nvidia-uvm can be opened by this user
ok GPU 000f:01:00.0: NVIDIA GB10 (sm_121), qualified (receipt: evidence/m16-blackwell-native-path-requalification-receipt.json)
ok memory: 121.7 GiB shared by the CPU and the GPU, 61.3 GiB available now
info CUDA toolkit: not needed and not used by CUDA++ (one is installed at /usr/local/cuda/bin/nvcc; that is fine)
info compiler: this doctor was built by 13.3.0; CUDA++ needs only make and a C compiler
READY. Next: make hello-gpu
On a machine with the wrong module (one of the nine fake machines CUDAPP_DOCTOR_SHOW=1 sh tests/doctor/run.sh build/cudapp-doctor builds):
FAIL NVIDIA kernel module: version 575.64.05 is loaded; CUDA++ speaks the 580.173.02 interface
Why: CUDA++ builds its driver calls from the 580.173.02 open-module headers
(third_party/nvidia-open-580.173.02). The layout of those calls can change between module versions,
and a wrong layout fails in strange ways, so CUDA++ refuses instead of guessing.
This is the only version CUDA++ asks you to match: no toolkit, no framework build, no extension build.
Do this: install the 580.173.02 open kernel module, or port the client to your version
(CONTRIBUTING.md, "Port a driver version": copy the new headers, rebuild, run make test and make hello-gpu).
Receipts: evidence/dx/d2-doctor/make-doctor-spark.txt, evidence/dx/d2-doctor/fake-machines.txt, evidence/dx/d2-doctor/negative-control.txt. FIXED.
The pain. "In our experience the python stack traces for these types of exceptions are basically always incorrect and pretty worthless." (vLLM's maintainers, vLLM blog)
How CUDA++ fixes it. A CUDA++ launch waits for its own completion by default, so an error comes back from the call that caused it. And the same kernel program runs on CUDA++'s host warp simulator with one flag, where every global access is checked against the buffers the launch was given. A bad access stops the run at that launch and names the kernel, the launch, the instruction, the block and thread, the address, the buffer it missed and by how much, and what to do next.
The command that shows it (--bug=past-end hands the last launch one element too many, the classic off-by-one):
$ build/hello_gpu --sim --bug=past-end
FAULT at launch 8 of 8 of kernel vecadd (simulator): global load outside every buffer
launch: grid 1025 x 1 blocks of 128 threads, n = 131073, first element 917504
instruction: 11 of 19, LDG.E (4-byte load)
thread: block 1024, thread 0 (element 1048576 of the whole job)
address: 0xeba6fbc6f040 (4 bytes), 0 bytes past the end of buffer a = [0xeba6fb86f040, 0xeba6fbc6f040) (4194304 bytes)
simulator: MEM at pc 11: global access outside the declared buffers
next: launch 8 was given n = 131073, and buffer a holds 131072 elements from this launch's first element.
Fix the launch or the buffer, run hello_gpu --sim until it passes, then run hello_gpu on the GPU.
Receipts: evidence/dx/d3-faults/fault-report-spark.txt, evidence/dx/d3-faults/test-hello-sim.txt, evidence/dx/d3-faults/negative-control.txt (with bounds checking off, the fault checks fail). FIXED on the simulator. Left: on the chip a faulting kernel still shows up as a launch that never completes, and the process then refuses further GPU work. Reading the GPU's own fault record (docs/issues/D3-chip-faults.md) and recovering without a restart (Omega bar C5, ledger L386) are open.
The pain. "For reference I just compiled it on a 2018 Dell laptop (Quadro P1000 GPU) and the build time was about 15 hours." (flash-attention #1038)
How CUDA++ fixes it. No per-architecture copies and no template-heavy device compiler: CUDA++ generates each kernel's machine code when the program builds it (row 9), so the library is plain C.
The command that shows it (on the Spark, 20 cores, with other builds running: load average 31.31):
$ git clone -b dx/pain-points <this repository> cudapp && cd cudapp
$ time make -j20
exit 0, wall 2.034379307 s
$ ls -l build/libcudapp.a
1060126 build/libcudapp.a
$ nm -u build/libcudapp.a | grep -c -i cuda (CUDA symbols the library needs)
0
Receipt: evidence/dx/pain-points/d5-build-from-empty.txt. FIXED for what CUDA++ contains. This is not like for like: CUDA++'s libraries cover far less than cuBLAS and cuDNN (docs/PAIN_POINTS.md section 4).
The pain. "Very frustrating compared to most every other GPU release that has zero day support from cuda/nvcc/pytorch/tensorflow." (r/LocalLLaMA, on the DGX Spark)
How CUDA++ fixes it. CUDA++ writes sm_121 machine code and has a supported-target table. make doctor refuses other NVIDIA GPUs by name and PCI id, with what is missing and the steps to port them. make hello-gpu runs the doctor first. This is a wrapper-only compatibility check, not a library admission guard: direct binaries and library callers can bypass it. A table row is not proof that the current checkout passed its gates; the M16 evidence limitation above applies.
The command that shows it (CUDAPP_DOCTOR_SHOW=1 sh tests/doctor/run.sh build/cudapp-doctor, the fake machine with a GPU CUDA++ has not qualified):
FAIL GPU 000f:01:00.0: NVIDIA device 0x2b85 is not one CUDA++ has qualified.
Why: CUDA++ writes each chip's machine code itself and has run it only on the NVIDIA GB10 (sm_121),
so it refuses this GPU now instead of failing later with "no kernel image is available".
What is missing: a target row for this chip, a check that the encoder's instruction words are valid on it,
its launch descriptor and class numbers, and the hello-gpu qualification run with its receipt.
What to do: follow docs/runtime/PORTABILITY.md section 5 ("What a new GPU needs") and CONTRIBUTING.md ("Port a chip"),
or report it with the output of `make doctor` attached.
Receipt: evidence/dx/d2-doctor/fake-machines.txt. FIXED in the doctor wrapper only: the refusal. Direct library admission is not enforced. Not fixed: CUDA++ targets one chip; porting more is open to everyone (CONTRIBUTING.md).
The pain. "Completely reproducible results are not guaranteed across PyTorch releases, individual commits, or different platforms." (PyTorch docs)
How CUDA++ fixes it. CUDA++ adds in a fixed order (no atomics in its reductions, evidence/E1-CLOSURE/), so the same inputs give the same bits with no setting to turn on. Every hello-gpu run prints the hash of its output, and so do the chip paths of the bf16 matrix-vector, rmsnorm and decode-attention tests.
The command that shows it (10 separate processes on the GB10, nothing set, the caller holds the GPU lock; output from the receipt):
$ flock -w 900 /tmp/aien-gb10.lock tools/run_d7_same_bits_chip.sh gemv 10 /tmp/d7
... (one line per run, then:)
SAME gemv_f32_151936x2048 ed3805976ef5eb04a893ed9d3b9ad1cc8afa17e60a36b7d4dd9dd98e698f9e4b in 10 of 10 runs
d7 same-bits: PASS (1 labels x 10 runs)
Receipt: evidence/d7-same-bits-chip/2026-10-27/SUMMARY.md (chip) and evidence/dx/d3-faults/hello-sim-spark.txt (simulator). FIXED on the GB10 for four jobs, 10 separate processes each, driver 580.173.02: hello-gpu (vector add; the chip hash equals the simulator's), the bf16 matrix-vector 151936 x 2048 (warp-shuffle reduction), rmsnorm at three shapes (reduction across the block) and decode attention at eight context lengths up to 32768 (softmax reductions, combine across up to 128 splits). The checker goes red on a chip run with one element unwritten and on a one-bit hash change (negative-control/). Not shown: the other kernels (scan and sum-reduce build on main since #112 but are NOT_RUN in this check), other GPUs, other inputs (docs/issues/D7-same-bits-chip.md).
The pain. "The CUDA tax, which is 4.6 GB approximately per each model on a DGX spark." (r/LocalLLaMA)
How CUDA++ fixes it. A CUDA++ process loads no CUDA library: Aien's live Qwen3.6-35B-A3B server (mixture-of-experts, about 3B parameters active per token) on 2026-10-21 had only the C library, the maths library, libgcc_s and the mapped (Omega evidence/WHAT-OMEGA-IS-20261021/live-serve-no-cuda.txt).
NOT YET MEASURED. The memory a CUDA++ process holds beyond its own buffers still needs a number from a chip run (docs/issues/D8-process-tax.md).
The pain. CUDA's licence: "You may not reverse engineer, decompile or disassemble any portion of the output generated using SDK elements for the purpose of translating such output artifacts to target a non-NVIDIA platform." (CUDA EULA) And a developer: "I hope someone is working on hardware agnostic solutions, we need more GPUs, not less." (r/LocalLLaMA)
How CUDA++ addresses it. This candidate uses Apache-2.0 WITH LLVM-exception for first-party material and preserves the earlier MIT grant for its pinned historical snapshot. Reverse engineering CUDA++ output, translating it and porting it to any hardware are allowed under the applicable license (CONTRIBUTING.md). Third-party notices remain in force.
The command that shows it:
$ grep -e "Version 2.0, January 2004" -e "LLVM Exceptions" LICENSE
Version 2.0, January 2004
---- LLVM Exceptions to the Apache 2.0 License ----
The earlier MIT-era output is retained as a dated receipt at evidence/dx/pain-points/licence.txt; it describes the historical tree, not this candidate. The GPU's signed firmware stays NVIDIA's: the chip checks its signature.
9. Compiling at run time, and hangs while it compiles (517 GitHub issues match "JIT compilation" hang cuda)
The pain. "A deep prefill forces a live JIT for a huge novel shape mid-inference, and that compile or kernel deadlocks the GPU on sm_121a." (dgx-spark-laguna FINDINGS.md)
How CUDA++ fixes it. The kernel builder runs inside the program, calls no outside compiler, keeps no cache directory, and gives the same bytes every time.
The command that shows it:
$ build/hello_gpu --sim
kernel built in 22 us: 32 instructions, 12 registers, no compiler run, no cache
machine code sha256 6248b8ed338019f75e20b13a8ef7e63b4e35764229e0669ffbe54464353e87d1
$ build/hello_gpu --sim
kernel built in 23 us: 32 instructions, 12 registers, no compiler run, no cache
machine code sha256 6248b8ed338019f75e20b13a8ef7e63b4e35764229e0669ffbe54464353e87d1
Receipt: evidence/dx/d3-faults/hello-sim-spark.txt. FIXED for the kernels CUDA++ builds. The model's larger kernels are built the same way; their build times are not yet printed.
The pain. "Even worse, if someone accidentally created a kernel with "cutlass" in the name, the driver would apply optimizations that are not safe." (r/LocalLLaMA)
How CUDA++ fixes it. Every CUDA++ library is source in this repository, built by the same kernel builder you can call, with host references and a simulator.
NOT YET. CUDA++'s matrix multiply is shaped for the models Aien runs. Attention has a general fallback since runtime API 0.6.0 (head_dim 1..256, any GQA ratio, causal and non-causal, sliding window, any length), but it is a scalar f32 kernel; its fast fused paths are still model-shaped. Any-shape GEMM, scans and sorts (Omega bar C3) and my own disassembler (bar C6) are ledger L386 (docs/issues/D11-libraries.md, docs/issues/D13-disassembler.md).
The pain. NVIDIA's data-center segment was 92.5% of its revenue last quarter: "Data Center revenue of $89.0 billion" out of $96.2 billion (NVIDIA, Q2 fiscal 2027). A company paid for compute sold has no reason to make the machine you own do the same work with less.
How CUDA++ answers it. Measure the work, not the hardware. Every hello-gpu run prints the bytes it moved, the operations it did, and the least time those bytes need at this machine's own measured ceiling, so less work per answer is a number in front of the developer.
The command that shows it:
$ build/hello_gpu --sim
work: 12582912 bytes moved (8388608 read, 4194304 written), 1048576 floating-point operations
at the GB10's measured 195.1 GB/s read ceiling this needs at least 64.5 us
Receipt: evidence/dx/d3-faults/hello-sim-spark.txt; the ceiling, evidence/OWN30B-T3-TC-PEAK-BW-5faa0f70-20261013T2236Z/. FIXED on the simulator. Left: the measured GPU time printed next to the floor, from a chip run (the code prints it; it has not run on the chip yet).
Replacing a developer's CUDA libraries and NVIDIA's kernel module with CUDA++ is ledger item L401; its section here comes with it.
You need: a DGX Spark (GB10) running Linux with NVIDIA's open kernel module loaded (version 580.173.02 was used here; /dev/nvidia0 and /dev/nvidia-uvm must exist and be readable and writable by your user), plus git, make, gcc and bash; the zero-CUDA gate also uses readelf and nm from binutils, which comes with gcc. Nothing from the CUDA toolkit is needed or used, no environment variable needs to be set, and nothing is fetched besides this repository.
From an empty folder:
git clone https://github.com/aien-dev/CUDApp
cd CUDApp
make hello-gpu
make hello-gpu-control
make zero-cuda-gate
make hello-gpu builds the library and examples/hello_gpu, runs make doctor (which checks the machine and names any problem with its fix), then runs the program. It builds a vector-add kernel, adds two arrays of one million numbers on the GPU and checks every answer on the CPU. It prints PASS and exits 0 when all answers are right, exits 1 when any is wrong, and exits 2 when it cannot use the GPU. build/hello_gpu --sim runs the same kernel on the host simulator with no GPU. examples/hello_gpu/hello_gpu.c explains each of its five steps.
make hello-gpu-control checks the check: it builds a second copy, build/hello_gpu_control, with a migration-pending switch turned on ( CUDAPP_MIGRATION_PENDING_HELLO_CONTROL=1; it is off in build/hello_gpu, and its .om successor is #79), runs it with --bug=short, which gives the last launch one element too few (nothing is written outside a buffer), and passes only when the program catches that one missing answer, prints FAIL and exits 1. make zero-cuda-gate reads the built programs' ELF dynamic section and imports and fails on any CUDA library or CUDA symbol; it first runs its own negative control, a stub libcudart that must fail. It is a check on the binaries, not proof that NVIDIA's kernel modules or firmware are gone.
On a shared machine, hold the GPU lock your operator uses around these commands (on the AIEN Spark: flock -w 900 /tmp/aien-gb10.lock make hello-gpu). A run of exactly these commands by a brand-new Linux user with an empty environment is recorded in evidence/C2-CLEAN-USER/.
make # build/libomega_gpu.a (also as build/libcudapp.a)
make test # every host-only check: encoder goldens, simulators, refusals, mutation tests
python3 tools/figures/make_figures.py --check # the figures on this page match the code they are drawn from
make doctor # is this machine ready? one line per check, each problem with its fix; opens no device
make test opens no device. The chip gates (./build/gpu_*_test --out receipt.json) need a GB10; each test's comment in the Makefile says how it is run.
make test-gemm-selectors is the focused host-only CLI regression for PFGEMM,
QGEMM and FP8GEMM. Each validates --shape before host/simulator/chip work or
receipt creation; unknown, empty and case-mismatched selectors exit 2 with INVALID,
never PASS. PFGEMM/QGEMM accept gateup, down, qkv, o; omission runs all.
FP8GEMM also accepts d768 and all (the four standard shapes, not d768).
No selected mode/cases/checks yields NOT_RUN rather than a zero-case PASS.
Invalid FP8GEMM input leaves an existing --out file untouched. These refusal
tests and doctor tests do not qualify GPU execution. Their before/after output
is retained in evidence/security-host-20261026/; the full host suite and GPU
gates were NOT_RUN for this repair.
Ownership rule (ARCH-0042, PROPOSED): Omega owns computation, CUDA++ owns GPU execution, Omega Bridge owns compatibility.
- CUDA++ (this repository) owns GPU execution: the sm_121 encoder and GPU code generators, the cubin writer and artifact exporter, artifact admission, the disassembler, the driver client (
nvrm/), submission, sessions and the GPU runtime. - Omega owns Aien's seeded
.omlanguage, its compiler and the compiler pin, semantics, and the declared contract of what a kernel computes. The existing Omegese → IR → sm_121 lowering, its checker and host words stay in omega-mv. CUDA++ is the GPU execution component Omega consumes. - Omega Bridge owns compatibility: C and Python emitted from a checked Omega subset, native bindings for foreign callers, framework attachments (PyTorch, llama.cpp/GGML; planned) and GPU artifact negotiation (
gpu_adapter.om). Bridge runs no kernels itself: its GPU adapter only accepts or refuses an attachment request and opens no device. Execution goes through this repository.
The currently shipped CUDApp implementation still contains first-party C; the planned, test-gated replacement with native .om is tracked in #79. The code was split out of Omega on 2026-10-21 with its full history; docs/AUDIT.md records how, and what was checked before it was published.
Not yet true: Omega does not consume CUDA++ at a pin today. Omega main has no cudapp.lock; it still builds its own diverged copy of the GPU code (#106) and takes the driver client from physics-mv (#107). The disassembler is in Omega (#10) and a second, .om cubin writer is proposed there (#108). The .om work here takes its compiler through Omega Bridge's bootstrap rather than an Omega pin (omega-mv#870).
The C names still carry the omega_ prefix they had inside Omega, so Omega's work in progress carries over unchanged (tools/extract/carry.sh). docs/NAMING.md has the plan for the cudapp names.
First-party material at this revision is Apache-2.0 WITH LLVM-exception (LICENSE); third-party terms and notices are in THIRD_PARTY.md. The earlier MIT grant is documented in the licensing transition and the license at the pinned historical MIT commit.