How to run local LLMs with Ollama and plug them into your coding agent

Ollama runs open models such as Gemma, Qwen, DeepSeek, Kimi and gpt-oss on your own machine, with a CLI, a local REST API and official Python and JavaScript libraries. The newest trick is the one most people miss: ollama launch wires a coding agent like Claude Code, Codex or OpenCode straight to Ollama models.

This guide goes from install to a local chat, then to calling the model from code, then to a coding agent backed by Ollama.

curl -fsSL https://ollama.com/install.sh | sh

irm https://ollama.com/install.ps1 | iex

Prefer not to pipe a script into your shell? There are manual downloads for macOS (Ollama.dmg) and Windows ( OllamaSetup.exe), manual Linux instructions at docs.ollama.com, and an official Docker image, ollama/ollama, on Docker Hub.

ollama run gemma4

The first run downloads the model, then you chat in the terminal. Download size and speed depend on the model and your hardware, so start small. The full list is at ollama.com/library.

Not sure where to start? Just run:

ollama

It prompts you to run a model or connect Ollama to your existing agents and apps, such as Claude Code, OpenClaw, OpenCode, Codex and Copilot.

ollama launch claude

Pick a model when prompted, then work in your repo as usual. Supported coding integrations listed in the README are Claude Code, Codex, Copilot CLI, DeepSeek Harness, Droid and OpenCode. Each link has the specific setup for that agent.

Be realistic about quality: small local models are much weaker at agentic coding than frontier models. Local is great for privacy, offline work and cheap experiments; give the agent bigger models for real multi-file tasks if your hardware allows.

There is also ollama launch openclaw, which turns Ollama into a personal assistant across WhatsApp, Telegram, Slack, Discord and more.

Ollama's REST API listens on localhost by default:

curl http://localhost:11434/api/chat -d '{
  "model": "gemma4",
  "messages": [{
    "role": "user",
    "content": "Why is the sky blue?"
  }],
  "stream": false
}'
pip install ollama
python
from ollama import chat

response = chat(model='gemma4', messages=[
  {'role': 'user', 'content': 'Why is the sky blue?'},
])
print(response.message.content)
npm i ollama
python
import ollama from "ollama";

const response = await ollama.chat({
  model: "gemma4",
  messages: [{ role: "user", content: "Why is the sky blue?" }],
});
console.log(response.message.content);

That is a full AI feature prototype with no API bill. When you need more, the API docs, CLI reference, Modelfile reference and importing models guide cover the rest.

Ollama lists llama.cpp as its supported backend. If you want lower-level control, such as an OpenAI-compatible server with your own flags, llama.cpp is the engine to look at directly.

Each page has who it is for, the steps, and a prompt you can paste into your coding agent.

Use case: Launch a coding agent backed by Ollama

Use case: Run an open model on your machine in one command

Use case: Call a local model from Python or JavaScript

Get one email every Monday with the week's hottest AI repos and exactly how to use them: subscribe free at dedupedai.com