Judgment to the model, precision to the tool, decisions to humans
What stops you from using AI coding at scale in your work?
Besides the fact that AI generated code quality is not stable, another big problem is that AI never follows the process you told it to follow.
You tell it to do requirements analysis first, and it just runs off and starts writing code. You make a five step plan, and by step three it forgets what the first two steps decided. Open a new session, and it cannot even remember where it left off yesterday.
Over the past few months, our team used a Python script to fix this bad habit of AI. This script acts as a navigator for the AI coding workflow. It uses a graph based on a state machine, and this graph decides where each step should go. Since then, the AI coding workflow has never drifted again.
In this article, I will show you how we did this.
You can find all the source code for this article at the end**, feel free to grab it.**
What Problem I Ran Into #
I have been formally pushing AI coding in my team for more than half a year now.
At first, we only made the kind of demo used to show off a product idea. It looked cool, but it could not really be used.
Later, we brought AIDLC into our AI development workflow. This is a workflow skill that came from Amazon, and it is a set of workflows made to standardize the AI development life cycle. It fully follows the enterprise level SDLC (software development life cycle) process, and it guides the LLM to generate code through detailed spec files.
After a few months of tuning, we were finally able to get AI to build applications that run stably in production.
I also shared our experience using AIDLC online, and it got a warm welcome from many people.
But as the work went on, we found that AIDLC could only help us build small projects or small features. When the project got bigger, problems started to show up.
When the context got too long, or when we switched sessions, AIDLC would start to lose track of the spec documents. This caused the whole workflow to stop strictly following the spec files generated ahead of time, and in the end there was always a bug or some gap in understanding.
We also switched back to OpenSpec and Superpowers to test it out, and the result was the same. Both had the problem of workflow drift.
Why This Problem Happens #
The root cause is that these are all workflow frameworks based on SDD (spec-driven development).
Whether it is AIDLC's skills or OpenSpec's commands, they are both made up of dozens of Markdown files. These frameworks get loaded into the harness when the session starts, and this takes up 30k to 50k tokens.
At the same time, before coding even starts, an SDD based development workflow also generates a large number of spec files. These files also get loaded into the context, and they take up a lot of the context window.
Frontier models often have context windows of more than 1 million tokens. It looks like they can hold a lot, but the model does not pay the same amount of attention to every position in the context.
Once a key rule document falls into a low weight zone, the model ignores it, and the instructions start to drift. It is like a thick instruction manual. The further you flip through it, the less you remember about the details at the front.
This is especially true for models like deepseek-v4.1-flash, which have a very long context window and super cheap caching. To chase a high cache hit rate, people tend to stuff a lot of content into the context window. Then they find that once the session gets long, the coding quality of these models drops a lot, and their ability to follow instructions also goes down.
In an article I wrote last year, I explained why LlamaIndex at the time could not accurately carry out workflow tasks based on the handoff mechanism. The reason was that the context was too long, and this caused the process to drift.
Using a Graph Engine in AIDLC #
Now that we know where the problem comes from, the next step is to look at how to fix it.
The answer is not in the AI itself, but in traditional agent frameworks. These frameworks figured out one thing a long time ago: who should drive the workflow should not be left entirely to the LLM's own judgment.
But if the goal is the same, having agents hand off tasks to each other, why did everyone end up choosing the graph engine path? To answer that, we need to look back at a bit of history in multi agent frameworks.
In early 2025, OpenAI released an agent framework called OpenAI Swarm. The core idea of this framework was to let the LLM call a function called handoff. This let the main agent hand off a planned task to a sub agent to carry out, which created a planner-executor architecture. Does this sound a lot like how modern AI coding tools (harnesses) use the task method to call up sub agents?
But this framework had a big limitation. When to hand off which task to which agent was written in the prompt given to the LLM. Because of this, the framework only worked well on the most advanced GPT models of that time. It did not work with other open source models. The root cause was that open source models back then were not yet good enough at following instructions.
This is why LangChain's LangGraph, which came out later (along with Crew.ai from around the same time), used a different approach. It used a scripted execution engine based on a state machine and a directed acyclic graph (DAG).
This approach is actually simple when you say it out loud. You use a Python script to first define a traditional workflow based on a graph. Each node in the graph is an agent that runs its own logic and produces structured output. Each edge is a conditional edge, and it decides which node to route the flow to next, based on the output of the previous node.
The LLM itself works on probability, so in the graph it is only responsible for making judgments, in other words, updating the state. What really decides how the workflow runs is a traditional scripted process defined by LangGraph. Because the script runs in a very deterministic way, the whole process never drifts.
By now, you have probably already seen where this is going. Yes, to let the harness also carry out the workflow accurately, I decided to add a graph based script engine to AIDLC. And at each stage, I let the LLM output a structured state to guide each step the script engine takes.
Now that the logic behind this is clear, let me walk you through how I designed this script engine.
I split the design of the engine into two phases: the authoring phase and the runtime phase. The authoring phase is mainly about how to define the workflow, and the runtime phase is about how to use the workflow.
Step one: define the workflow metadata
Whether it is AIDLC or an SDD framework like OpenSpec, they both actually record the definition of the whole AI coding workflow in prose, inside skill files or prompt files. The LLM then follows this prompt definition to move agents and process nodes along the flow.
This creates a problem. The LLM can pull information out of text, but a script program cannot. Neither Python nor JS can understand the meaning of text without help from an LLM.
One way to do this is like LangGraph's API, where the whole workflow gets defined directly in code. When a new development task comes in, it gets routed strictly according to the code that was already written.
But this method has a big downside: it is almost impossible to customize. I want to turn AIDLC into an open framework, which means it also allows users to modify the workflow themselves. And while using it, I also want to be able to adjust the implementation details of the workflow. If every change means I have to touch the script engine, that is too risky.
So I thought, could we use a structured YAML text file to define the workflow, the way crew.ai does?
But I also did not want to throw away the workflow definitions already in the existing Markdown files and start a whole new spec from scratch. That would mean every time I adjust the workflow, I would need to change both the Markdown file and the YAML file.
So I came up with a solution. I kept the existing 14 stage Markdown prompt files of AIDLC unchanged. But at the top of each file, I added a frontmatter. This means I put a block of config information at the start of the Markdown file that only the script reads. Then I pulled all the workflow related definitions out of the prompt and put them into this config block.
Here is a typical frontmatter, using the requirements-analysis.md file as an example:
---
slug: requirements-analysis
phase: inception
execution: ALWAYS
condition: Always executes with adaptive depth — every request needs its intent and requirements assessed; depth (minimal/standard/comprehensive) scales with clarity, complexity, and risk
gate: approve-continue
produces:
- inception/requirements/requirements.md
- inception/requirements/requirement-verification-questions.md
consumes:
- artifact: inception/reverse-engineering/*
required: true
conditional_on: brownfield
- artifact: inception/product-design/product-design.md
required: false
requires_stage:
- workspace-detection
- reverse-engineering
- product-brainstorm
depth: adaptive
scopes:
classic: EXECUTE
bugfix: EXECUTE
refactor: EXECUTE
security-patch: EXECUTE
infra: EXECUTE
express: EXECUTE
---
This frontmatter looks pretty complex, but it is actually very simple to understand:
- Keys like
slug,phase,execution,condition,gate,producesare all metadata. They define the conditions needed before the current node runs, what phase it is in, what it produces, and so on. The script engine uses these keys to check if the current node is working correctly when it routes the state. consumesandrequires_stagedefine which upstream nodes the current node depends on, and which states of the upstream nodes it needs to consume.depthandscopesdefine which workflows the current node shows up in, and how deep the branch of the graph goes. This makes it clear exactly when a Markdown file needs to be loaded, instead of every single Markdown file right at the start.
I also wrote a stage-contract.md document under the reference/common directory of AIDLC. This document works as a meta spec, and it records in detail what each key in the frontmatter means and what values it can take. So the next time I adjust the workflow, or add or remove a stage, the LLM can use this document to help me write the frontmatter correctly, and it can update the dependencies between the existing frontmatters without making mistakes. This is very handy.
Step two: set up multiple workflows based on scenario
Next, let me focus on the scopes key. This key is used to let AIDLC generate different workflows for different scenarios. Yes, after bringing in the new script engine, AIDLC no longer has just one single workflow. Instead, it now has 6 completely different workflows.
What is the benefit here?
In the original AIDLC V1.0, there was only one workflow, and all 14 stages were marked with different activation conditions. Depending on whether it was a new project, how complex the project was, whether it was a bugfix or a feature request, and so on, it was decided which stages would run and which would be skipped.
But whether a stage would run or get skipped, the prompt defining it would still get loaded into the context as soon as the skill loaded. This wasted a lot of tokens.
So this time I split AIDLC's workflow by business scope, from one scope into six. Each business scope maps to a different set of stages to run. When the user types a prompt and this triggers the aidlc-workflows skill to load, the LLM figures out which scope the current iteration falls into, and it only loads the Markdown files it needs.
As your project moves through different stages, the system also adjusts which scope it picks on its own, instead of sticking with the same one scope forever.
Here is an example: say your project starts out as just an MVP you want to build quickly to test your idea in the market. In this case, AIDLC decides this is an express scope, so it only loads the fewest stages needed to quickly finish a product prototype.
Then, as your project wins over users and slowly grows into a product running stably in production, AIDLC's workflow uses the audit log and past spec files to shift the scope to classic. At this point, optional stages like user stories, non functional requirements, security, and scalability all get loaded in, helping you polish your product step by step.
Step three: turn the metadata into a workflow graph
As I mentioned earlier, a script engine cannot possibly build a workflow at runtime by reading natural language text. So we need a descriptive syntax to let the engine run according to a definition made ahead of time.
We used frontmatter to define AIDLC's metadata at each stage. We also used scope to sort out which stages should be used in which scenarios. Now, just like crew.ai, we need to use a structured syntax to turn these workflows into predefined workflows.
The approach here is actually pretty simple. I did not generate the workflow dynamically at runtime (meaning, while the user is using AIDLC). Instead, I generate the workflow graph while developing the AIDLC skill.
I placed a generate.py script under aidlc-workflows/scripts. Every time the workflow definition changes, you can have the harness run the generate.py script to regenerate the graph file.
The generate.py script also supports a --check flag. Every time the AIDLC skill gets updated, you can use this flag to run an integrity check. I also set up a GitHub CI process that automatically checks the integrity of the graph every time code gets committed. This is very handy.
In the end, the generate.py script produces a graph in JSON format, stored under the aidlc-workflows/scripts/data/ directory. This graph defines what the workflow architecture looks like under each scope, what its entry conditions are, what it is responsible for, what it outputs, and so on. At runtime, the script engine reads this graph and uses it to build the final workflow.
Step four: use the script engine to run the workflow precisely
Now that we have talked about how the workflow gets defined, let's talk about how I use the script engine to make sure the workflow runs precisely.
Other frameworks also use scripted mechanisms to make sure the workflow runs precisely, but they all need hooks from the harness to listen to the different states of the workflow as it runs. This often means an SDD framework needs to work with more than twenty different harnesses, and it needs a custom plugin for each one.
But in an enterprise level AI coding environment, this approach does not work very well, because we always build our own harness on top of an open source harness. So I would rather use a workflow framework that is not tied to any specific harness.
The AIDLC skill I built was designed from the start to be harness agnostic. It still follows the AgentSkill spec, and it keeps the script engine as a CLI script stored under the skill directory's scripts folder, with the script named engine.py.
The nice thing about this is that you do not need to care at all about how this script is built or used. You just copy the skill directory into your project's root folder like before, and then you can use it.
Let me give a quick intro to how the engine.py script works. As a CLI script, engine.py's most important commands are these:
init. This command sets up a state file when the workflow starts. All the different states of the workflow get recorded in this file later on, so the script engine can use the state machine to route each node of the workflow.next. This command reads the state file, figures out which stages are already finished and what those finished stages produced, and uses this to decide what the next node is and routes to it.report. After each stage finishes, this command gets used to update the state file. This way, whether you are continuing after the context got compressed or starting a new session, you can always pick up from the last finished stage.status. This reads the global workflow properties set in the state file. For example, which scope is in use right now, whether optional stages are turned on, whether team collaboration mode is turned on, and so on, can all be read through this command. This way, every time a session resumes and reloadsaidlc-workflows, there is no need to ask the user the optional questions all over again.
Step five: use a state file to precisely record the workflow state
As I mentioned earlier, the script engine I designed is basically a graph engine based on a state machine, similar in principle to LangGraph. So besides the graph file that defines the workflow, the state file is another very important file.
This file gets generated each time a new workflow starts. It still uses the original aidlc-state.md file from AIDLC V1.0, but this file has been heavily reworked, and now it is split into two parts.
The first part is the workflow related config. Whether it is a new project or an old project, which scope is in use, which stages need to run, where the artifact documents are stored, which extended features are turned on, all of this gets recorded here. This way, whether it is a new session or picking up from a break in the session, there is no need to scan through the artifact documents one by one. You just read this file and can pick up the process from where it left off.
The second part is the task checklist that engine.py generates based on the config. While the workflow is running, the engine follows this checklist strictly, and after finishing, it goes back to this checklist to update the task status.
The script engine uses the two XML tags <!-- BEGIN ENGINE-STATE> and <!-- END ENGINE-STATE> to mark the content of the task checklist. Only the script engine is allowed to change the text wrapped inside these tags, not even the LLM is allowed to edit it directly.
Also, every time the task status is updated, a sha256 hash gets generated for the whole checklist. If the sha256 hash changes during a run, this means the node status has been tampered with, either by a human or by the LLM. The script engine will then raise a warning, and it will rebuild a correct checklist by working backward from the artifact files that were already generated. This greatly reduces the chance of workflow drift during a long running task, or cases where the LLM planned a task but never actually carried it out.
## Project Information
- **Project Type**: Greenfield
- **Start Date**: 2026-10-08T04:10:57Z
## Workspace State
- **Existing Code**: No
- **Reverse Engineering Needed**: No
- **Workspace Root**: D:\Documents\PythonProject\b15-dogfood2-chess
...
<!-- BEGIN ENGINE-STATE | do not hand-edit — maintained by scripts/engine.py -->
State Version: 1
Engine: aidlc-workflows/scripts/engine.py
## Stage Progress
### inception
- [x] workspace-detection
- [S] reverse-engineering
- [x] product-brainstorm
...
## State Digest
sha256: 33cc76d5ae995526968640649c234282291b00e42949b40c826722fc5ee5ac02
<!-- END ENGINE-STATE -->
Alright, that covers the whole implementation of the script engine. You might feel a bit anxious thinking this is so complex, do I need to spend time learning a whole new framework again?
Actually, no. This new script engine, just like the aidlc-workflows skill itself, is harness agnostic. You just drop the skill into your project's root folder and it runs on its own. You do not need to worry at all about how to use this skill.
Next, let me use a simple project to show you how the script engine works within a workflow.
Demo the Script Engine with a Small Project #
I will use a small request, "build a bookkeeping app", to show you how this script engine works.
Since this is just a demo project, I used a one sentence request. But if you want to use this in a production project, it is best to prepare a full product design document ahead of time, writing out clearly what features your product should have. Or you can first use OpenCode's plan mode to talk through your idea with AI, and write it down as a design document, then use build mode to start the aidlc-workflows and kick off the development process.
Since this starts from a new session, once I submitted my development intent to OpenCode and aidlc-workflows finished , the script engine first ran the status command to find out what state the workflow was currently in. Of course, the result was that this is a brand new, empty project.
So it used the init command to set up the new workflow.
After each stage finishes, the script engine uses the report command to update the workflow state. You can see that the LLM is no longer involved in moving the workflow forward at all. This is all handled by the skill calling the CLI.
That is pretty much it. The script engine compiles every node that the whole iteration needs to move through into a flow graph, and no matter how long the session stretches, it never drifts off track.
All you need to do is, at the approval gate set up for each stage (the gate configured for that stage), take a quick look at what it produced and give it a thumbs up. Once it passes review, the engine calls the next command to move you forward to the next stage.
And at every important point, like writing a document, calling an outside tool, or moving between states, the script engine uses the park command to leave a note. So even if your workflow gets cut off unexpectedly for whatever reason, the AI can pick back up from the spot where the last note was left.
If you start a new session to pick up a development process that was not finished before, the script engine also uses status first to read the current state and continue from there. There is no need to ask the user to choose again, and no need to scan through every single artifact file that has already been saved. This saves a lot of time.
That covers the demo of the core features. As you can see, while the script runs, every step follows the predefined process exactly, and even in a long session, nothing drifts off track, because the state file keeps everything in line.
You also do not need to give any special instructions. Just tell the AI your intent normally, and you do not need to worry about how the workflow is actually built.
Conclusion #
That is the story of how I used a script engine to make AIDLC run exactly according to the predefined workflow. This script engine borrows a core idea from Amazon's AIDLC v2.0: "judgment to the LLM, precision to the tool, decisions to humans." It also draws on the design wisdom of traditional agent workflow frameworks like LangGraph and Crew.ai.
The new aidlc-workflows skill, now with the script engine, has been running as a trial in my team for more than two months. The biggest thing I have noticed is that the workflow almost never drifts off track because of a long session anymore, and this precise execution of the workflow has further raised the quality of the code produced by AI coding.
I enjoyed writing this article. A lot of people say that AI technology moves so fast that whatever you learn today might be useless in just a few months. But we do not need to be too pessimistic about this. Take the solution shown in this article as an example. Even though the technology keeps changing, the principles behind the technology can still help you solve problems you have not even seen yet.
My story of improving AIDLC is not over yet. In my next article, I will walk you through how the AIDLC workflow can help you start from a "one sentence wish" and iterate step by step, eventually growing into a large, sustainably run project. Stay tuned.
Thanks for reading and subscribing. You can find the source code for this article at the end of the post, and try it out yourself. If you have any questions, feel free to leave a comment anytime, and I will reply as soon as I can.
Further Reading #
Source Code #
All the source code for this article is here, feel free to grab it:
Here's the source code for this article. Sign up now to get it for free.