ANEForge compiles a tensor graph into one program for the Apple Neural Engine and runs it from Python, without CoreML. The same path trains on the engine, optimizer included. The compute unit is fixed, so a model never falls to CPU/GPU.
0.33 ms on the engine
6.1× faster, 2.0 ms
Simulations and learning, run and trained on the engine. #
Not just neural nets. Each of these compiles to ANE programs and runs end to end on the Neural Engine.
Training, LLMs, native layers, compression, and more. #
Apple exposes the Neural Engine only through CoreML, and only for inference. ANEForge dispatches through the same private aned
stack that CoreML uses internally, from a normal user process.
Training runs on the engine CIFAR-10 → 71%
The forward pass, backward pass, and Adam update all compile to ANE programs, so a model trains end to end on the engine.
LLMs on the engine 27B on a pure ANE
Prefill and KV-cache decode for Llama/Qwen, exact speculative decoding, Mixture-of-Experts from GGUF, and the hybrid Qwen3.5-27B (DeltaNet + attention), decoding end to end on the engine.
MLPerf on the engine passes the checker
The MLPerf reference ResNet-50 runs pure-ANE and passes the upstream MLCommons submission_checker
at the reference accuracy (fp16 76.44%, equal to fp32). One command reproduces it.
Layers CoreML can’t reach +19 bridge ops
af.sdpa
drives fused attention directly, what CoreML never emits. Sort, argmax
, topk
, geometry too.
Streaming weight compression 4× smaller
int8, int4-LUT, or sparse weights via the dequant path, accuracy-gated. KV-cache and optimizer state stay resident.
Run arbitrary workloads FFT, BLAS
Not just neural nets. FFTs, linear algebra, and fluid sims compile straight to the engine.
Cross-compile across silicon 28 targets
Lower and gate a graph for any M1 to M5 ANE from one machine, and estimate its latency without running it.
CoreML is the only public route to the engine, and all it decides is whether to use it. #
| Path | On the ANE | No CoreML | Trains on it |
|---|---|---|---|
| CoreML / coremltools | scheduler chooses | no | no |
| MLX, PyTorch (MPS) | no (GPU) | yes | on the GPU |
| ANEForge | yes (direct) | yes | yes |
Where it lands: #
the reference ResNet-50 against official MLPerf.
The MLPerf reference ResNet-50 on the engine, next to official MLPerf Inference results. Every competitor number is sourced to a public mlcommons/inference_results
submission; the ANE row is unofficial and self-measured.
| System | SingleStream p90 | Offline | Power class |
|---|---|---|---|
| Apple Neural Engineunofficial | 0.773 ms | 1,149/s | single-digit W · SoC block |
| NVIDIA Jetson AGX Orinv3.1 · edge | 0.640 ms | 6,424/s | 15–60 W module |
| NVIDIA H100, 1 GPUv4.0 · datacenter | — | 88,714/s | 350–700 W board |
The package, the paper behind it, and a guide to the engine. #
ANEForge
The Python package. Build a graph, compile it to one ANE program, then run or train it. It handles quantized weights, native attention, resident state, and cross-compilation for 28 targets.
[GitHub →](https://github.com/sbryngelson/ANEForge)
[Docs →](https://aneforge.readthedocs.io)
The paper
“Python for direct computation on the Apple Neural Engine.” The preprint behind the numbers on this page, with the dispatch path and the validation against reference in full.
The guide
A reverse-engineering account of the engine: the datapath, the compiler, the program format, the firmware, the kernel driver, and the cross-silicon target tables. Most of it is undocumented by Apple.
Apple Silicon, macOS 14+, Python 3.10+. #
The e5rt
dispatch shim builds against Apple frameworks on first use. Then browse examples/
, starting with the quickstart.