BaseRT

Apple's M5 generation introduces a dedicated Neural Accelerator on every GPU core, and BaseRT, a native Metal inference runtime, exploits these units to deliver up to 6.4× higher prompt-processing thr…

A new LLM inference runtime called BaseRT achieves up to 6.4x faster local inference on Apple Silicon than llama.cpp by writing directly to Apple's Metal GPU API, skipping intermediate frameworks like…

BaseCompute released BaseRT, a fast inference runtime for local AI on Apple Silicon, claiming up to 35% faster decode and 78% faster prefill on an Apple M4 Pro with 4-bit quantization. The runtime all…