Array Languages #10: X_eTaL on Metal and CUDA, Through Mojo

3015 words • 16 min read • Abstract

Why another way onto the same GPU #

The X_eTaL-gpu work reaches every GPU it uses through OpenCL, the M1 Max included. That was the right first choice: one runtime for old and new cards from every vendor. On Apple silicon, though, it has costs:

• OpenCL on macOS is a leftover. Apple deprecated it in 2018, with macOS 10.14, in favor of Metal. It still ships, frozen at version 1.2.
• It is slow to call. Post #7 measured about 1.3 ms of fixed cost on every call on the M1 Max, against about 0.2 ms on the RTX 5060 Ti, a large part of why the M1 Max lost to Apple’s tuned CPU library on small batches.
• Its profiling is hard to trust. Apple’s OpenCL reports kernel times of a few hundredths of a millisecond for everything, which no second clock has confirmed.

Why Mojo #

Mojo is Modular’s systems language, Python-like on the surface, built for high-performance code on CPUs and accelerators. One Mojo program can target NVIDIA, AMD and Apple GPUs. Two things made it a fit:

• It reaches Apple’s GPUs through Metal. Support began in September 2025, with v0.25.6, and by the v1.0 beta in May 2026 Apple Metal was “a much more capable Mojo target”.
• It speaks C. X_eTaL’s native extensions, frompost #4 , are shared libraries with a C interface. Mojo’s@export now takes anabi("C") effect, so a Mojo library can present exactly that boundary.

From X_eTaL source to Mojo #

Read the diagram from the top. An X_eTaL program, here the SAXPY demo from post #7’s Accel library, is lowered by X_eTaL-gpu into its accelerator IR, using X_eTaL’s own crates, so the libraries it uses are inlined and every array operation becomes an IR operation. From there the IR can go two ways. On the left is the path post #7 measured, an OpenCL emitter and runtime. On the right is the new one: a Mojo emitter, a library built by Mojo’s compiler, and Mojo choosing the GPU, Metal on the Mac and NVIDIA’s driver on the Arch box. Whichever way it goes, the answer is checked against the X_eTaL evaluator.

The program is the one post #7 ran, unchanged:

"ac:" u̲se< "Accel"
x ← (r̲ange 8) ÷ 4
y ← 1.0 ᵃᶜs̲cale 1.0 0.5 0.25 0.125 0.0 -0.125 -0.25 -0.5
(2.0 ᵃᶜs̲cale x) ᵃᶜv̲add y

The only change is the device: xetal-gpu run saxpy.xtl --device mojo in place of --device opencl:0. The Mojo backend fuses the divide, the two scales and the add into one kernel, and writes it as ordinary Mojo a person could read:

def stage0(
    b_l: Pointer[Float32, ImmutAnyOrigin],
    b_r_2: Pointer[Float32, ImmutAnyOrigin],
    b_add: Pointer[Float32, MutAnyOrigin],
    n: Int32,
):
    """Stage 0: %div, %mul, %mul_2, %add fused, one thread per element."""
    var i = Int(block_idx.x * block_dim.x + thread_idx.x)
    if i < Int(n):
        var v_l = b_l[unsafe_offset=i]
        var v_r_2 = b_r_2[unsafe_offset=i]
        var v_div = v_l / Float32(4.0)
        var v_mul = Float32(1.0) * v_r_2
        var v_mul_2 = Float32(2.0) * v_div
        var v_add = v_mul_2 + v_mul
        b_add[unsafe_offset=i] = v_add

Then mojo build --emit shared-lib makes it a library with C entry points, cached by the source’s hash, and the tool loads it and calls it. Mojo’s DeviceContext opens whatever GPU the machine has. The GPU prints 1.5 1.5 1.75 2.125 2.5 2.875 3.25 3.5, which is what the X_eTaL evaluator prints.

Five demos, two GPUs #

One run shows the whole path. The two terminals at the top run the same command, just run-mojo libs/Accel/demos/reduce-sum.xtl, on the Mac and on the Arch box. The command lowers the X_eTaL file, has the Mojo backend write and build its kernels, and runs them. The blue line names the GPU that ran it, api=metal and the M1 Max on the left, api=cuda and the RTX 5060 Ti on the right, and the green lines are the program’s five results: a sum, a largest value, a dot product, then a float sum and a sum of squares. The strip below is the X_eTaL evaluator running the same file on the CPU. All three print the same five numbers.

Both runs reused a library already built, which is why the build reads 0. The Arch box’s run is an unedited capture from that machine.

“Agrees” means the output matches the evaluator’s committed baseline for that source, within 1e-5 for floats. The other eight Accel demos are refused by name, each with the operation the Mojo backend does not have yet: a reduction along an axis, an inner product, a transpose, a lookup, a take or drop, a rotation. The OpenCL backend runs all thirteen.

The times say more about the machines than the GPUs. On the 5060 Ti, building a demo’s library took 4.25 to 4.78 s the first time, and the first call, which sets up the device, 389 to 593 ms. After that, with the library cached, a second call in the same process took 0.05 to 0.09 ms. On the M1 Max a build takes about 2.6 s, cached after, and a run 3.5 to 4.5 ms, including creating the device context. The two hosts differ in more than their GPUs, the Mac’s repository on an SSD and the Arch box’s on a hard disk, so whole-process times compare machines, not cards. Neither is yet a fair comparison with OpenCL’s per-call cost; that measurement is planned.

The first proof #

Before the backend moved into X_eTaL-gpu, it was a small standalone tool, rust-demo, in a fork of Modular’s repository: a Rust program that reads the IR’s text form, writes Mojo, builds it and runs it. It began with five programs written by hand in the IR, and ended with the five Accel demos above, lowered from X_eTaL source, checked against both its own Rust reference and the X_eTaL evaluator, on both GPUs. That tool is also where the evidence that the GPU, and not the CPU, did the work comes from.

A result that matches could have come from the CPU. So each rust-demo run prints the device the library opened, the grid it launched, and, from an extra kernel, which block and thread computed each element. The two terminals show the same program, reduce, the sum, largest and smallest of a thousand values, run by the same command on each machine. Read the orange line first: api=metal and the M1 Max on the left, api=cuda and the RTX 5060 Ti on the right. Everything below it matches, line for line, down to the last green line: three of three outputs agree with the reference.

The times differ, and they are not a race. The Mac’s run reused a kernel it had already compiled; the Arch box’s was its first. The assembly the device compiler received can be printed too: Metal AIR for air64-apple-macosx26.0.0 on the Mac, PTX for sm_120a on the NVIDIA card.

Which thread did which element

A GPU runs a kernel as a grid of blocks, each a group of threads that work side by side. The grid on the left is the reduce program’s thousand elements, one square each, colored by the block whose thread computed it. Three blocks took 256 elements each, and the fourth took the last 232; its other 24 threads found no element and did nothing. The two outlined squares are the ones the tool names: element 0, computed by block 0, thread 0, and element 999, by block 3, thread 231.

That map comes from the GPU, not from the plan. An extra kernel has every thread write down its own block and thread number beside the element it handled, and the tool reads those numbers back. Both GPUs drew the same map, which is what one generated source for two vendors should give.

How Mojo reaches each GPU #

• Apple silicon: Mojo compiles kernels to Metal’s intermediate form, AIR, and runs them through Metal. No OpenCL.
• NVIDIA: Mojo compiles kernels with its own LLVM backend to PTX, NVIDIA’s portable GPU assembly; its bundledptxas assembles that to machine code, and the CUDA driver library,libcuda , which comes with the driver, loads and launches it. No CUDA toolkit ornvcc is needed, only driver 580 or newer, or anotherptxas named byMODULAR_NVPTX_COMPILER_PATH . The 5060 Ti’s driver, 615.78, needed no override.
• AMD: through ROCm. Not tried here.

So on NVIDIA, Mojo does go through CUDA’s driver, even though no CUDA is written. The GPU work’s claim of running without CUDA rests on the OpenCL backend, and every result here names the backend it used.

The cards: Mojo or OpenCL #

Modular lists the RTX 30, 40 and 50 series and the Turing generation as compatible for Mojo development. Mojo’s compiler also has targets for the older Maxwell and Pascal generations, which the docs do not list as supported, and none for Kepler. My plan is to support the older GPUs through X_eTaL-gpu’s OpenCL path if I cannot configure Mojo for them. The picture puts each of my cards on NVIDIA’s line of generations, oldest on the left: gray where Mojo has no target at all, amber where its compiler has one that Modular does not list, green where it is listed. The two outlined cards, the RTX 5060 Ti and, below, the M1 Max, are the ones that have run X_eTaL through Mojo.

The M3 Pro is the other Mac to try; Mojo lists M1 through M5.

What is still to do #

The Mojo backend covers elementwise work and whole-array reductions. Still ahead, in order:

• Axes and structure: reductions along an axis, several shapes in one program, then tiled matrix products, transposes, lookups, slices, rotations and reshapes, which brings in the rest of the Accel demos and, after them, X_eTaL-ML’s models, which run from their own source through OpenCL today.
• The macro and the extension:--device mojo behindgp:a_ccel< and the GPU extension, so an X_eTaL program calls the GPU through Mojo itself, with both backends checked against the same answers.
• Results by GPU, beside the OpenCL numbers, including the cost of a call with everything already built.
• Wider Ints: Ints cross the C boundary as 64-bit floats for now, so values beyond 2^53 are not yet tested.

What about MLX? #

Apple’s own MLX is the other way onto Metal, and in some ways the closest thing to X_eTaL already on the Mac: whole-array operations, evaluation deferred so a computation can be compiled and fused, memory the CPU and GPU share, and a C interface. Every operation the lowered programs use has an MLX counterpart, so an MLX runner could call MLX for each operation instead of generating kernels. The trade-off is reach: MLX is for Apple silicon, and Mojo just ran the same programs on an NVIDIA card. Both are worth measuring on the same Macs.

And FPGAs? Mojo compiles for CPUs and GPUs; it has no way to describe hardware, so the Tang boards stay with X_eTaL-fpga.

Part 10 of the Array Languages series. View all parts

Comments or questions? SW Lab Discord or YouTube @SoftwareWrighter.