A developer released AI-CLI, a tiny C terminal assistant that connects user requests to a local LLM and executes returned shell commands directly. The tool supports multiple platforms and LLM engines,…
A developer ported Google's Gemma-4 family (2B, 4B, 12B) to AWS Inferentia2, overcoming three architectural obstacles that break the vendor stack. The port achieved 44 tok/s for the 2B model, 33-39 to…
Mlx-optiq, a new open-source tool, enables per-layer mixed-precision quantization of large language models on Apple Silicon, allowing users to run, fine-tune, and serve LLMs locally on Macs without a …
A developer has published a guide for running Google's Gemma-4 12B instruction-tuned model on Windows Subsystem for Linux 2 (WSL2) using the llama.cpp framework. The process involves installing build …
A developer successfully ran Google's Gemma-4 E2B-it large language model on a Raspberry Pi 4 using llama.cpp, achieving text generation speeds of 1.5 to 1.8 tokens per second. The project involved co…