I have very, very limited experience with AI agents.
I've used AI heavily while building software, debugging, writing, researching, and generally figuring things out as I go. But multi-agent systems? Local models? Orchestrating two separate models and letting them pass work back and forth without me stepping in?
That was new territory.
Which is exactly why I wanted to try it. heh
The overall question was a simple one. In my head, at least:
What happens if I give one local AI model the job of software developer, another local model the job of code reviewer, and then get out of their way?
No OpenAI API.
No Claude API.
No paid inference.
Just Ollama, Python, my PC, and two local models talking to each other.
My PC specs:
* CPU: Intel i5-12600KF
* GPU: NVIDIA RTX 3070 Ti
* RAM: 16 GB
* OS: Fedora Linux
* Runtime: Ollama
* Initial models: Qwen2.5-Coder 7B + Qwen3 8B
* Final models: Qwen2.5-Coder 3B + Qwen3 4B
The result is hilarious, honestly. I expected the experiment to either fail immediately or produce something surprisingly competent. Instead, it did both.
I wanted the simplest possible development team.
One agent would be the Builder. Its responsibility was to read a product request, inspect the current workspace, and create or modify the application.
The second agent would be the Reviewer. It would receive the original task, inspect what the Builder created, review the implementation, and either approve it or request changes.
The flow looked like this:
I called the project RelayLab, because the agents essentially relay the project between each other. (duh)
The orchestrator itself is just Python. For the first version, I intentionally kept it constrained — partly because I wanted to keep the test simple, and partly because I had no clue what I was doing. The agents couldn't run arbitrary shell commands or touch the rest of my computer. The Builder could only propose file writes and deletions inside an isolated experiment workspace. Every round was also logged so I could inspect exactly what happened afterward.
My first attempt was perhaps slightly ambitious. I started with:
Builder: qwen2.5-coder:7b
Reviewer: qwen3:8b
And my computer basically responded: absolutely not.
CPU usage shot to 100%, the machine became almost unusable, and I got my first lesson in running multiple local models. The issue wasn't just model size — Ollama was keeping both models loaded between turns, which meant the second model couldn't fit cleanly into GPU memory and started spilling work onto the CPU.
So I downsized. The eventual setup became:
Builder: qwen2.5-coder:3b
Reviewer: qwen3:4b
I also changed the Ollama requests to use:
{
"keep_alive": 0
}
That forces each model to unload after its turn — like the good little AI it is. Instead of two AI coworkers fighting over the same VRAM, the process became:
Much better.
Seeing 100% GPU in ollama ps while a completely local model was actively building an application was genuinely one of those little moments where the technology suddenly felt much more real. There's an unexplainable joy about running local AI models rather than using something like ChatGPT or Codex. This is all happening right now, on my computer?
Truth be told, I spend a lot of time using AI products. But running the actual models locally and watching them become parts of a system I wrote myself felt completely different.
Before any agents could actually talk, I hit an even stranger issue.
Ollama happily let me download several gigabytes of models. ollama list worked perfectly. Then I tried running one:
error starting llama-server:
llama-server binary not found
Huh?
My Ollama installation was incomplete — which is particularly funny, because from the outside everything looked fine. The models existed. The service existed. The CLI existed. The component responsible for actually performing inference did not.
After reinstalling Ollama properly, the models finally started running. RelayLab itself hadn't even had its first agent conversation yet, and I had already learned considerably more about local inference than I expected.
For my first real test, I deliberately avoided writing an extremely detailed specification. I gave the system this:
Build a polished single-page notes app where I can
create, complete, and delete notes.
Make it pleasant to use.
The Builder received that prompt. It generated an index.html. Lil ole RelayLab wrote the file.
Then the Reviewer inspected it.
[round 1] builder wrote 1 file(s); reviewer: changes_requested
Great, I thought.
The review went back to the Builder. Another revision.
[round 2] builder wrote 1 file(s); reviewer: changes_requested
Then another.
[round 3] builder wrote 1 file(s); reviewer: changes_requested
At this point I was sitting there watching two local models iterate on software without me writing the implementation. And yes, that was extremely cool.
Then the Reviewer broke the entire experiment.
I initially required the Reviewer to return exactly one of two statuses:
{ "status": "approved" }
or:
{ "status": "changes_requested" }
Seems reasonable. Except small local language models do not particularly care about your beautifully designed enum. The Reviewer returned something else.
RelayLab responded exactly as any lovingly over-strict software system should: AgentProtocolError. Crash.
The funny part was that the AI had probably made a completely sensible judgment. The system failed because it expressed that judgment using the wrong vocabulary.
So I made the orchestrator more tolerant. Statuses such as rejected, needs_changes, request_changes, and failed were normalized into changes_requested. Likewise, pass, accepted, and approve became approved.
That fixed the first problem. Then the Reviewer found a new way to break things — it returned valid JSON with no status field at all. The response was probably wrapped inside another structure, or used a completely different property name:
{
"review": {
"decision": "revision required",
"issues": [
"Notes do not persist",
"Delete button needs an accessible label"
]
}
}
It now normalizes that into something like:
{
"status": "changes_requested",
"feedback": "- Notes do not persist\n- Delete button needs an accessible label"
}
And if the Reviewer still produces something completely unusable, RelayLab no longer destroys the entire experiment. It safely assumes changes_requested, preserves the malformed response in the logs, and keeps going.
This might have been my favorite lesson from the experiment: the models weren't necessarily failing at reasoning. They were failing at protocol obedience. And those are very different problems.
Eventually, the system survived all six rounds.
[round 1] builder wrote 1 file(s); reviewer: changes_requested
[round 2] builder wrote 1 file(s); reviewer: changes_requested
[round 3] builder wrote 1 file(s); reviewer: changes_requested
[round 4] builder wrote 1 file(s); reviewer: changes_requested
[round 5] builder wrote 1 file(s); reviewer: changes_requested
[round 6] builder wrote 1 file(s); reviewer: changes_requested
Runtime: 3 minutes, 54 seconds
Final result:
{
"status": "max_rounds_reached",
"rounds_completed": 6
}
And this was the application.
Um. Wow.... Beautiful. Stunning. A triumph of modern artificial intelligence.
It looks like an HTML tutorial from 1998. 😭
The app worked well enough to create notes, and the agents had clearly spent several rounds modifying the implementation. But remember the original instruction: make it pleasant to use. The giant white page, browser-default input, browser-default buttons, and nearly nonexistent visual hierarchy were not exactly what I had in mind.
The even more interesting part was what the Reviewer cared about during the final round. Its feedback was:
Critical bug in completion toggle: the note text gets a trailing space when toggling between completed and not completed. The completion state should be stored as a boolean per note to avoid string manipulation.
And technically? That's a good review. The Reviewer found a legitimate implementation flaw.
But look at the screenshot again. There was a much bigger problem.
This became the most interesting result of the experiment.
The agents were cooperating. The Builder could implement changes. The Reviewer could find bugs. Feedback successfully traveled between them. But the system wasn't necessarily becoming a better product. Instead, it started becoming locally optimized.
The Reviewer found a specific issue. The Builder addressed that issue. The Reviewer found another specific issue. The Builder addressed that. Six rounds later, they were debating representation of completion state while the application still looked almost completely unfinished.
Neither agent consistently stepped back and asked: does this actually satisfy the user's overall request?
That's a different problem from code generation. It's a coordination problem. And simply adding another AI agent didn't magically solve it.
Before doing this experiment, it would have been easy for me to assume:
one AI developer = useful
therefore
developer AI + reviewer AI = more useful
But the interaction between agents matters just as much as the intelligence of the individual models.
My first Reviewer prompt essentially told the model: find problems. So that's what it did. Forever. There was no strong definition of what "done" actually meant. No acceptance rubric. No prioritization. No distinction between:
BLOCKER: The requested feature doesn't exist.
and:
MINOR: There's an extra whitespace character in the internal representation.
Without that structure, adding a Reviewer can actually create an endless optimization loop. The system needs a manager — even if that manager is deterministic code rather than another LLM.
RelayLab's next version will give the agents explicit acceptance criteria instead of relying on open-ended reviews. For this notes app, that might include functionality, usability, accessibility, visual quality, persistence, and whether the actual product still resembles the original request.
I also want to introduce browser verification. Right now, the Reviewer only sees the source code — it doesn't see what I saw when I opened the resulting file. That matters. Eventually I'd like the workflow to become:
Builder
↓
writes app
↓
browser harness
↓
opens application
↓
performs interactions
↓
captures screenshot
↓
Reviewer receives
├── source
├── test results
├── screenshot
└── acceptance criteria
↓
approve / revise
Then I'll give the system the exact same original prompt. That gives me something much more useful than simply making RelayLab better — it gives me an experiment. I can compare:
Version 1 — open-ended reviewer, static source inspection, 6 rounds, failed to converge
against:
Version 2 — explicit acceptance criteria, browser testing, visual feedback, same models, same prompt, same hardware
...and see what actually changes.
I went into this with extremely limited experience building agent systems and even less experience running language models locally. That turned out to be one of the best parts — instead of starting with assumptions about how agents were supposed to work, I got to watch the failure modes appear in real time.
The models fought over VRAM. The inference runtime broke. The agents couldn't follow my JSON protocol. The Reviewer learned how to say "no" approximately seventeen different ways. And once all of that finally worked, the two agents successfully collaborated for six rounds... only to produce a notes app that looked like this.
[Screenshot again, because honestly it deserves another appearance.]
But I don't consider the experiment a failure. Quite the opposite. The infrastructure worked. Two local models performed different roles, exchanged feedback, modified a shared artifact, survived multiple iterations, and did it entirely on my own computer without paid inference.
What failed was the assumption that conversation alone creates coordination. It doesn't. Agents need constraints. They need shared definitions of success. They need tools for observing the real environment. And apparently they occasionally need a Python script standing between them saying:
"I know you wrote 'rejected,' but what you meant was 'changes_requested.'"
I'm going to keep building RelayLab. Mostly because now I really want to know what happens in round two.
RelayLab: github.com/miflow13/Relay
The broader concept of multi-agent software development isn't new — projects such as ChatDev and MetaGPT have explored teams of specialized language-model agents before. My experiment is much smaller: I'm interested in seeing what happens when that idea is pushed onto consumer hardware using small, entirely local models and a deliberately simple orchestrator.