The author successfully ran Google's new Gemma 4 open models (2B and 4B parameters) on a laptop using Ollama during Google I/O 2026 week. To make the small 2B model function as a useful research agent…
Fully local AI system using Llama 3 and LangChain to enable natural language querying of an internal SQLite database, eliminating any data being sent to third-party servers and avoiding legal complian…
Detailed, step-by-step guide for building llama.cpp from source on a Dell Precision T5820 workstation equipped with an RTX 3090 Ti, achieving 42 tok/s with Qwen3.6-27B. The author emphasizes that the …
The GGUF file format consolidates all necessary model components—including weights, chat templates, and special tokens—into a single file, offering a more ergonomic alternative to the scattered JSON f…
Ollama, an AI model server, ran on a MacBook with no NVIDIA GPU by using GTAP software to intercept CUDA calls and forward them to a remote DGX Spark workstation with a 128 GB Blackwell GPU. The setup…
The article describes the author's successful setup for running local AI models on an M4 Mac with 24GB of memory, specifically highlighting Qwen 3.5-9B (Q4 quantized) as the best performing model at ~…
Based on the article, the author, a self-described "AI skeptic," successfully used local LLMs (GPT-OSS 20B and Qwen3 Coder 30B) to migrate blog comments, completing the task in a few evenings. The aut…
The author, a software developer, finds generative AI tools like Claude highly useful for certain tasks, such as Java Spring Boot development, but notes that local models are only adequate and that th…