Hey everyone!
Like many developers, I’ve become increasingly dependent on AI coding agents.
They are incredibly productive. A single developer with a capable coding agent can now accomplish tasks that previously required several developers working together.
But there is a catch: the more I use them, the more I depend on expensive models. And despite their impressive capabilities, I still find myself dealing with unpredictable execution, hallucinations, limited control, and difficulty understanding what actually happened during a long-running task.
I started wondering:
Could we build a coding agent harness that makes local and affordable LLMs more useful, while making the overall system more reliable, controllable, and efficient?
That question led me to build CodeForge-v2.
#
There are already many excellent coding agents and agent frameworks out there. Rather than forking one of them, I decided to experiment with a different architectural approach.
My core design principle is:
LLM proposes. Deterministic systems decide authority and enforce execution.
In many agent architectures, the LLM plays a central role in deciding what happens next. The harness provides tools and an execution loop, but the reliability of the overall system still depends heavily on the model.
CodeForge takes a different approach by separating intelligence from execution authority.
The LLM handles:
The deterministic runtime kernel handles:
Policy enforcement and tool authorization. #
Task graph management and state transitions. #
Approval and budget management. #
Workspace revisions and checkpoints. #
Verification and completion decisions. #
Execution history and provenance.
The goal is to ensure that an LLM can propose an operation without automatically having the authority to execute it.
I believe this separation is particularly important when building autonomous agents that need to work on software projects over extended periods.
#
I don’t want to build just another ReAct loop with tool calling.
My longer-term vision is an autonomous software engineering assistant that can:
Work with local models through tools such as Ollama, as well as cloud models. #
Execute software engineering tasks with controlled permissions. #
Track progress and let developers monitor execution through a dashboard. #
Allow developers to inspect, approve, and control important operations. #
Learn from execution failures through structured feedback and evaluation. #
Expand its capabilities through controlled extensions.
Ultimately, I want to explore controlled self-evolution.
However, I don’t want an agent to freely rewrite its own governance kernel. I’d rather keep the core runtime responsible for authority and safety while allowing new capabilities to evolve independently under explicit controls.
This is an important distinction in the direction I’m exploring.
#
I also have a slightly different perspective on evaluating coding agents.
SWE-bench is important, and I plan to use it to evaluate CodeForge. But benchmark scores alone don’t tell us whether an agent can consistently produce and maintain good software.
I care about questions such as:
Can it create a clean and extensible architecture? #
Can it preserve architectural invariants during refactoring? #
Can it improve a codebase without breaking existing functionality? #
Can it avoid accumulating architectural debt? #
Can it maintain useful documentation as a project evolves?
CodeForge incorporates a failure taxonomy to distinguish model failures from harness, tool, and interaction failures.
The idea is to avoid changing the harness to compensate for limitations that actually belong to the model or execution environment.
I also want to use held-out evaluation sets to detect regressions after optimization.
This evaluation approach is still evolving, but I believe it is worth exploring.
#
The deterministic runtime kernel is implemented and serves as the foundation of the project.
There is still considerable work ahead before CodeForge becomes a comfortable, daily-use coding agent.
My main priorities are:
1. Developer experience
Improving the dashboard, VS Code integration, execution monitoring, and approval workflows.
2. Environment and toolchain intelligence
Making the agent better at identifying missing dependencies, requesting permission to install them, retrieving relevant technical information, and using CI/CD feedback to recover from failures.
3. Benchmarking and optimization
Running SWE-bench experiments, analyzing execution data, and systematically improving the harness.
I’m currently constrained by local storage and compute resources, so I cannot run large-scale experiments as quickly as I’d like.
#
I’m sharing CodeForge because I’d like to connect with people interested in autonomous software engineering and agent infrastructure.
I’m particularly interested in feedback from:
Developers building agent harnesses and autonomous coding systems. #
Researchers exploring agent reliability, governance, and evaluation. #
People experimenting with local LLMs and Ollama. #
Contributors interested in architecture, runtime systems, dashboards, and integrations.
If you have access to sufficient compute resources and are interested in helping run SWE-bench experiments, I’d also be very grateful. I can provide the setup and configuration; the resulting agent database is enough for me to analyze the execution traces and failures. You don’t need to contribute code to help. A technical review, a discussion, or simply trying the project with a local model would already be valuable.
I’m building this as an open-source project, and I’m looking forward to learning from the community.
Thanks for reading!