Ferrox v0.9.1, a pure-Rust GGUF inference engine, shipped today with MoE prefill on Apple Metal 2.4x faster, closing the gap to llama.cpp from 2.62x behind to 1.11x on OLMoE-1B-7B. The update also imp…
A new C++ implementation of NVIDIA's Sana 0.6B text-to-image model, named Sana.cpp, runs approximately 4.8 times faster than the PyTorch pipeline on Apple Silicon CPU. The project, developed by GitHub…
Christopher 'CJ' Pais, creator of the desktop dictation tool Handy, released transcribe.cpp on June 30 as an MIT-licensed C/C++ library that unifies local speech-to-text inference across macOS, Window…
Transcribe.cpp, a C/C++ speech-to-text inference library, has been released supporting 16 model families and 60+ variants via GGUF models on the ggml runtime. It offers Metal, Vulkan, and CUDA backend…
Mozilla.ai announced the release of transcribe.cpp, an open-source C/C++ speech-to-text inference library with GPU-accelerated support for multiple STT models. Developed through the Builders in Reside…
OpenAI Whisper has 12 unique checkpoint files, but the Python package exposes 14 local model names due to aliases for 'large' and 'turbo'. Additional confusion arises from runtimes and converted forma…
Handy Computer released transcribe.cpp, an open-source ggml-based transcription library supporting over 60 ASR models with GPU acceleration via Vulkan, Metal, CUDA, and TinyBLAS. The library, created …
Embershard, a macOS chat app with its own LLM inference engine, has been released in beta v0.1.1 for Apple Silicon devices running macOS 14 or newer. The app bypasses llama.cpp for inference, instead …