How to use Ollama: run AI models on your own machine

Ollama installs and runs open AI models locally for free under the MIT license, serving an API on localhost:11434 that editors and agents can query, according to the tool's own README verified on 12 August 2026. The install is a single command per OS — `curl -fsSL https://ollama.com/install.sh | sh` on macOS and Linux, or `irm https://ollama.com/install.ps1 | iex` on Windows — followed by `ollama run qwen2.5-coder` to start a coding model. The two limits are that available RAM caps model size, with roughly 7B-class models on 8 GB, 14B on 16 GB, and 32B-class on 32 GB or more, and that Claude Code will not connect to Ollama without a translation layer.

The Primer · Tools & Stacks How to use Ollama: run AI models on your own machine Vendor-neutral Ollama runs open models on your own computer, for free, with one install command and one run command. For a vibe coder the appeal is blunt: inference that costs nothing, never leaves your laptop, and has no rate limit but your hardware. Here is the whole thing, install to editor, plus the plain catch about your RAM and the reason Claude Code will not point at it without help. Ollama is the shortest path from “I want to run an AI model on my own machine” to a model actually running. It is one install command and one run command, it is free and open source under the MIT license, and once it is up it quietly serves an API on your laptop that your editor and your agents can talk to. For a vibe coder watching the meter, that is the appeal in one line: local inference costs nothing per token and has no rate limit except your hardware. Here is the whole answer up front. Install Ollama, run ollama run qwen2.5-coder , and you have a coding model answering on localhost:11434 . Point an OpenAI-compatible editor or agent at that address and you are coding against a model that never leaves your machine. The two catches, both covered below: your RAM sets the ceiling on how good a model you can run, and Claude Code will not connect to it without a translation layer. Install it: one command Verified from Ollama’s own README on 12 August 2026. Pick your OS: - macOS and Linux: curl -fsSL https://ollama.com/install.sh | sh - Windows: irm https://ollama.com/install.ps1 | iex , or download the installer from ollama.com/download https://ollama.com/download That installs the ollama command and a background service. There is nothing else to configure to get started; the service is what serves the local API you will use later. The commands you actually need Ollama’s CLI is small enough to learn in a minute. These are the ones you will use daily: - ollama run <model downloads the model if needed and starts a chat with it. ollama run qwen2.5-coder is a fine first command. - ollama pull <model downloads a model without running it, for later. - ollama list shows what you have downloaded, and their sizes on disk. - ollama ps shows what is currently loaded in memory. - ollama rm <model deletes a model you are done with, because they are large. Everything else is a variation on those. The model names come from ollama.com/library https://ollama.com/library , and a name can carry a size tag, so qwen2.5-coder:14b pulls the 14-billion-parameter version specifically. Which coding model, by your RAM This is the question the search results are full of, and the answer is that your memory decides it. A model has to fit in RAM or GPU VRAM to run at a usable speed, and a rough rule is that a quantized model needs a bit more than its parameter count in gigabytes. So: - 8 GB of RAM or VRAM: run a 7B-class model, such as qwen2.5-coder:7b . Capable for autocomplete and small edits. - 16 GB: a 14B model like qwen2.5-coder:14b fits and is a real step up for reasoning about code. - 32 GB or more: you can run a 32B-class model, though it will be slower; this is where local starts to feel genuinely useful. The code-tuned models worth knowing, all in the library, are Qwen2.5-Coder and Qwen3-Coder, DeepSeek-Coder-V2, CodeLlama, CodeGemma, and StarCoder2. The one thing no local model will do is match a frontier hosted model on the hardest tasks, which is the trade you are making: you swap some capability for zero cost and full privacy. Whether that trade is worth it for your work is exactly the calculation our cost study https://okaneland.com/study/cut-your-ai-coding-bill/ frames. Wire it into your editor and agents A running model is only useful once your tools can reach it, and this is where Ollama earns its place. The background service exposes two APIs on localhost:11434 : its native one at /api/chat and /api/generate , and an OpenAI-compatible one at /v1 for example /v1/chat/completions , where the API key can be any placeholder like ollama . That OpenAI-compatible endpoint is the important part, because most agents already speak the OpenAI API. Point one at http://localhost:11434/v1 and it just works: OpenCode, Aider, Cline, and Zed all connect to a local Ollama model directly this way, which the agent field guide https://okaneland.com/study/ai-coding-agent-field-guide/ covers as the model-agnostic camp. That is the whole setup: pick the agent, set its base URL to your local address, name your model, and you are running an agent against a model on your own laptop. The exception is the one people search for most: Claude Code will not point at Ollama by itself. Claude Code speaks Anthropic’s API, not the OpenAI one, so connecting it to a local model needs a translation proxy in between a router or a tool like LiteLLM that presents an Anthropic-shaped endpoint and forwards to Ollama . It is possible, it is not native, and if your goal is simply “an agent using a local model,” reaching for OpenCode or Aider is the shorter road than bending Claude Code to it. When local is not enough: Ollama Cloud Some models are too large for any laptop, and Ollama’s answer is Ollama Cloud, which runs big models on their hardware while you keep the same commands. You ollama signin , then pull a cloud model by its -cloud suffix, such as ollama pull gpt-oss:120b-cloud , and run it through the exact same CLI, Python, JavaScript, or API you already use. There is a free tier with usage limits, and the appeal is that a 120-billion-parameter model runs from the same ollama run muscle memory as a 7B one on your machine. It is the escape hatch for the days your hardware is the bottleneck, without leaving the tool. Ollama versus the alternatives Two comparisons come up constantly, and both have short answers. LM Studio does much of what Ollama does with a graphical interface instead of a command line, so it suits people who would rather click than type; Ollama suits people who want a command and an API to script against. llama.cpp is the lower-level engine that does the actual model running, and Ollama is a friendlier layer on top of that lineage, trading some fine control for a much simpler setup. None of these is wrong; they sit at different points on the same convenience-versus-control line, and the vibe coding stack https://okaneland.com/primer/vibe-coding-stack/ places where each fits. Is it free, and the real limit Yes. Ollama is MIT-licensed and running models locally is free, unlimited, and fine for commercial use; there is no per-token bill because there is no vendor counting tokens. The cost you actually pay is threefold and worth naming: the hardware to run a decent model, the electricity while it runs, and the capability gap against a frontier hosted model on hard problems. For the large, cost-conscious slice of coding that is autocomplete, refactors, and boilerplate, a local model on Ollama is genuinely enough and genuinely free, which is why it belongs in the toolkit even if a hosted model still does your hardest thinking. Start with ollama run qwen2.5-coder , wire it into the agent you already use, and you will know within an afternoon whether your machine is fast enough to make it a habit. One email, when there's something worth sending Get the research in your inbox. No fixed schedule, no filler. You get an email when we've tested something, run the numbers, or found a tool worth your time. Free. Double opt-in, unsubscribe in one click. What are you running locally? Get the research → https://okaneland.com/subscribe/ Sources | Source | Link | |---|---| | Ollama README github.com/ollama/ollama, read 2026-08-12 : install on macOS and Linux with curl -fsSL https://ollama.com/install.sh \| sh, on Windows with irm https://ollama.com/install.ps1 \| iex; run a model with ollama run <model ; the model library is at ollama.com/library; the native REST API serves on localhost:11434 /api/chat, /api/generate, /api/tags . | github.com ↗ https://github.com/ollama/ollama | | Ollama OpenAI-compatibility docs docs.ollama.com/openai, read 2026-08-12 : Ollama exposes an OpenAI-compatible endpoint at http://localhost:11434/v1 including /v1/chat/completions , with the api key set to any placeholder such as "ollama"; this is what lets tools built for the OpenAI API talk to a local model. | docs.ollama.com ↗ https://docs.ollama.com/openai | | Ollama Cloud docs docs.ollama.com/cloud, read 2026-08-12 : sign in with ollama signin, then pull a cloud model with a -cloud suffix e.g. ollama pull gpt-oss:120b-cloud and run it through the same CLI, Python, JavaScript, or API; cloud runs models too large for local hardware, with a free tier and usage limits. | docs.ollama.com ↗ https://docs.ollama.com/cloud | | Ollama model library, coding models ollama.com/library, read 2026-08-12 : current code-tuned models include qwen2.5-coder and qwen3-coder, deepseek-coder-v2, codellama, codegemma, and starcoder2, in sizes from about 1B to 30B+ parameters. | ollama.com ↗ https://ollama.com/library | | Ollama license github.com/ollama/ollama/blob/main/LICENSE, read 2026-08-12 : MIT. Running models locally is free and has no per-token cost; the constraint is your own hardware. | github.com ↗ https://github.com/ollama/ollama/blob/main/LICENSE |