Setting Up a Local AI Coding Agent with Ollama and Aider

A developer set up a local AI coding agent using Ollama on Windows and Aider inside Ubuntu on WSL2, running the Qwen2.5-Coder model entirely on the local machine without a dedicated GPU. The architecture keeps Ollama on Windows to avoid duplicate model installations while Aider connects via mirrored networking in WSL2. The setup enables code development with LLMs while keeping all data local and avoiding cloud APIs.

Over the past few months, I've experimented with several workflows for using LLMs in software development. In this article, I describe the local setup I've validated on my own PC: Ollama running on Windows, Aider inside Ubuntu on WSL2, and a code-focused model running entirely on the local machine. The machine I used for my first experiment runs Windows 11 with an Intel Core i7-1355U CPU, 48 GiB of RAM, and no dedicated NVIDIA GPU. It's generally a capable machine, but not particularly well suited for AI inference. Because it lacks a dedicated GPU, I had to run the model in CPU-only mode. The goal of this setup was to create an environment where both the source code and the data remain entirely on the local machine—far from the cloud-based frontier model approach. From an architectural standpoint, I decided to keep Ollama on Windows because I also use the installed models with other Windows applications, and I wanted to avoid maintaining duplicate model installations. Aider, on the other hand, runs inside WSL, where installation and configuration are simpler and the Linux command-line environment is more convenient. Ollama remains exposed on the local port 11434. Aider, running inside Ubuntu WSL2, connects to Ollama through the OLLAMA API BASE environment variable. This approach avoids duplicating models inside WSL while keeping development repositories in the Linux filesystem under ~/repos , where Git and other command-line tools work much more naturally. Since this machine is not an AI workstation, the solution had to be realistic. That meant avoiding massive GPU-hosted models, cloud APIs, and overly complex automation. The final architecture looks like this: Windows ├─ Ollama │ └─ qwen2.5-coder:14b │ └─ WSL Ubuntu ├─ Aider ├─ Git └─ Development repositories Keeping Ollama on Windows while running Aider inside WSL was a natural choice. I already use Ollama with several Windows applications, so duplicating the models inside WSL made little sense. At the same time, I much prefer working with the Linux command line for software development. I installed Ollama on Windows and downloaded the primary coding model: ollama pull qwen2.5-coder:14b I also installed the smaller model for quicker testing: ollama pull qwen2.5-coder:7b To verify the installation: ollama list Initially, Aider running inside WSL couldn't communicate with Ollama running on Windows. To make it work, I had to configure WSL to use mirrored networking . From PowerShell: notepad $env:USERPROFILE\.wslconfig Then I added the following configuration: wsl2 networkingMode=mirrored dnsTunneling=true firewall=true autoProxy=true Finally, I restarted WSL: wsl --shutdown After reopening Ubuntu, I tested the connection: curl http://127.0.0.1:11434/api/tags The response confirmed that WSL could successfully reach the Ollama server running on Windows at 127.0.0.1:11434 . Inside Ubuntu/WSL: sudo apt update sudo apt install -y git python3 python3-pip python3-venv pipx curl python3 -m pipx ensurepath exec $SHELL -l pipx install aider-chat To verify the installation: aider --version I also configured Git so I could version-control my experiments: git config --global user.name "xxxx" git config --global user.email "xxxxxxx@xxxxx.xx" I created a simple test project: mkdir -p ~/repos cd ~/repos mkdir my-project cd my-project git init Then I launched Aider: cd ~/repos/my-project OLLAMA API BASE=http://127.0.0.1:11434 aider --model ollama chat/qwen2.5-coder:14b --no-show-model-warnings When Aider displayed: Aider v0.86.2 Model: ollama chat/qwen2.5-coder:14b Git repo: .git I finally knew the basic setup was working correctly. For convenience, I also wanted to access the repository directly from Windows Explorer. From inside WSL: explorer.exe . This command opens the current WSL directory directly in Windows File Explorer. Once I had Aider working with a local Ollama instance, there was still one thing I wanted to understand better: what was happening under the hood. How many tokens were being processed? How long did inference take? What kind of throughput could I expect? We'll cover all of that in the next installment. 😉