AI agents don't usually become unreliable all at once. They degrade quietly. One session the agent repeats a step it already completed. The next, it loses track of where it was mid-run and starts over from scratch. Eventually it produces output with nothing in place to verify whether it actually succeeded.
The problem is structural. Harness components get built as one-off scripts instead of versioned, reusable units. Your state file lives in one project, a slightly different version lives in another, and when you fix a bug in one place the fix stays local. Teams end up maintaining the same harness logic across multiple projects and debugging whichever version happens to be in front of them.
This tutorial shows how to fix that at the source. Instead of writing harness logic as inline scripts, you'll package it as versioned components in a shared Bit scope. Fix a bug once, tag and export, and Ripple CI (Bit's built-in component CI/CD pipeline) propagates the update to every project that depends on it automatically. By the end, your harness stops being something you rebuild per project and becomes something you install with a single command.
Before you start, make sure you have the following in place so the setup steps run cleanly:
Node.js 18 or higher installed on your machine. Run node --version
to check.
A bit.cloud****account. You'll create a scope during setup. If you don't have one yet, head to bit.cloud/signup.
Basic familiarity with TypeScript. The components in this tutorial use TypeScript. You don't need to be an expert, but you should be comfortable reading typed function signatures.
A terminal and a code editor. The tutorial runs entirely from the command line with a few file edits along the way.
The three components are a state tracker, a context and a verifier. Each one targets a specific failure mode in the agent loop.
harness/state tracks what the agent has done, what it's currently working on and what comes next. It reads at session start and writes at session end.
harness/context pre-loads a structured map of your components and their relationships before the agent takes its first action.
harness/verifier evaluates agent output against a done condition before the loop continues, returning one of four verdicts: NO
, YES
, MAYBE
or IFF
.
Once all three are released to a shared scope, your entire harness installs into any agent project with a single command:
bit install @your-username/agent-harness.harness.state @your-username/agent-harness.harness.context @your-username/agent-harness.harness.verifier
Here's how the three components fit into the agent loop:
Create a new scope and give it any name you like. For this tutorial, the scope will be named agent-harness
. You can also use an existing scope."
With your scope ready, install Bit's version manager by running this in your terminal. It handles Bit installations and keeps your version up to date across projects:
npx @teambit/bvm install
Next, initialize your workspace. This creates a workspace.jsonc
configuration file at your project root and sets agent-harness
as the default scope for every component you create in this workspace:
bit init --default-scope your-username.agent-harness
Replace your-username
with your bit.cloud username. A successful run looks like this:
successfully initialized a bit workspace.
Open workspace.jsonc
and uncomment the Node environment line. This tells Bit which runtime to use when building and compiling your components:
"bitdev.node/node-env": {}
Then run this to pull the Node environment and resolve its full dependency tree. This is a one-time step per workspace:
bit install
This will take a few minutes on first run. You'll see pnpm working through several hundred packages before it completes.
Before creating any components, run this to see the available templates for your environment:
bit templates
The template you want is module
, listed under bitdev.node/node-env
. Now run this to scaffold all three harness components at once:
bit create module harness/state harness/context harness/verifier
Each command generates a component folder with a TypeScript entry file, a test file and the necessary Bit configuration. You'll see output confirming all three components were created under your scope, each assigned the Node environment automatically:
One thing worth noting before you move on: harness/state
, harness/context
and harness/verifier
are the component paths inside your workspace. When Bit publishes them to your scope, it generates a fully qualified package name by combining your scope and component path, for example @your-username/agent-harness.harness.state
.
Your workspace is ready. Three component folders now exist under agent-harness/harness/
, the Node environment is configured and you have a scope waiting to receive them once the implementations are complete.
An agent without externalized state forgets everything between sessions. Every run starts blind: no record of what was completed, no awareness of what's in progress and no queue of what comes next. This is the failure mode that production systems are designed to prevent at the infrastructure level. The state component fixes that by reading a structured JSON file at session start and writing back to it before the session ends.
Open the agent-harness/harness/state/state.ts
file generated by bit create module
in the previous step and replace the generated content with this:
import { readFileSync, writeFileSync, existsSync } from 'fs';
import { resolve } from 'path';
export type HarnessState = {
done: string[];
inProgress: string[];
next: string[];
};
const DEFAULT_STATE: HarnessState = {
done: [],
inProgress: [],
next: [],
};
export function loadState(statePath = 'harness-state.json'): HarnessState {
const abs = resolve(statePath);
if (!existsSync(abs)) return { ...DEFAULT_STATE };
const raw = readFileSync(abs, 'utf-8');
return JSON.parse(raw) as HarnessState;
}
export function saveState(state: HarnessState, statePath = 'harness-state.json'): void {
const abs = resolve(statePath);
writeFileSync(abs, JSON.stringify(state, null, 2), 'utf-8');
}
HarnessState
has three fields. done
holds everything the agent has completed. inProgress
holds whatever the agent is currently working on. next
holds the queue of work still to come.
loadState
reads the state file at the path you specify, defaulting to harness-state.json
at the project root. If no file exists yet, it returns an empty default state rather than throwing an error. saveState
writes the updated state back to the same path before the session ends. Between those two calls, your agent has a persistent, structured record of exactly where it is in the work, regardless of how many sessions it takes to get there.
Without pre-loaded context, your agent rediscovers the same dependency relationships from scratch on every run. It wastes the first part of every session figuring out what it already knew. The context component solves that by querying your Bit workspace for component dependencies and dependents before the agent takes its first action, handing it a structured map it can reason over immediately.
Open the agent-harness/harness/context/context.ts
file generated by bit create module
in the previous step, and replace the generated content with this:
import { exec } from 'child_process';
import { promisify } from 'util';
const execAsync = promisify(exec);
export type ComponentMeta = {
id: string;
description?: string;
dependencies: string[];
dependents: string[];
};
export type DependencyMap = Record<string, ComponentMeta>;
async function runBitShow(componentId: string): Promise<{ id: string; description?: string; dependencies: string[] }> {
const { stdout } = await execAsyncbit show --json ${componentId}, { timeout: 30_000 });
const data = JSON.parse(stdout);
const deps: string[] = (data?.dependencies ?? []).map((d: { id: string }) => d.id);
return {
id: componentId,
description: data?.description,
dependencies: deps,
};
}
async function runBitDependents(componentId: string): Promise<string[]> {
try {
const { stdout } = await execAsyncbit dependents ${componentId}, { timeout: 30_000 });
return stdout
.split('\n')
.map((l) => l.trim())
.filter((l) => l.length > 0 && !l.startsWith('┌') && !l.startsWith('└') && !l.startsWith('│') && !l.startsWith('─'));
} catch {
return [];
}
}
export async function buildDependencyMap(componentIds: string[]): Promise<DependencyMap> {
const map: DependencyMap = {};
await Promise.all(
componentIds.map(async (id) => {
const [meta, dependents] = await Promise.all([runBitShow(id), runBitDependents(id)]);
map[id] = { ...meta, dependents };
})
);
return map;
}
There are two things worth understanding about how this works before you move on.
runBitShow
shells out to bit show --json
for a given component ID and parses the JSON output into a structured object containing the component's ID, description and direct dependencies. runBitDependents
shells out to bit dependents
and parses the plain-text output, filtering out the table-border characters Bit uses in its CLI output to leave you with a clean list of dependent component IDs.
buildDependencyMap
takes an array of component IDs and runs both queries concurrently for each one using Promise.all
. The result is a DependencyMap
: a keyed record where each entry gives your agent a full picture of what a component depends on and what depends on it, all resolved before the first agent action fires.
Most agent loops use the same context to generate and judge output. That creates a real failure mode: the model can sound confident and still be wrong. A separate verifier keeps the check outside the generation loop and forces a second pass before the agent continues. The verifier component does that by sitting in a completely separate component, isolated from the generator, and evaluating output against a done condition you define before the loop starts.
Addy Osmani argues for keeping the maker away from the checker. The verifier is where that principle lives in your harness.
Open the agent-harness/harness/verifier/verifier.ts
file generated by bit create module
and replace the generated content with this:
export type DoneCondition = 'NO' | 'YES' | 'MAYBE' | 'IFF';
export type VerifyResult = {
verdict: DoneCondition;
reason: string;
condition?: string;
};
export type VerifierOptions = {
agentOutput: string;
doneKeywords?: string[];
failKeywords?: string[];
iffPattern?: RegExp;
};
export function verify(opts: VerifierOptions): VerifyResult {
const { agentOutput, doneKeywords = [], failKeywords = [], iffPattern } = opts;
const normalized = agentOutput.toLowerCase();
if (failKeywords.some((kw) => normalized.includes(kw.toLowerCase()))) {
return {
verdict: 'NO',
reason: Output contains a failure signal.,
};
}
if (doneKeywords.length > 0 && doneKeywords.every((kw) => normalized.includes(kw.toLowerCase()))) {
return {
verdict: 'YES',
reason: All done keywords found in output.,
};
}
if (iffPattern) {
const match = agentOutput.match(iffPattern);
if (match) {
return {
verdict: 'IFF',
reason: Output satisfies pattern but requires conditional verification.,
condition: match[0],
};
}
}
if (doneKeywords.some((kw) => normalized.includes(kw.toLowerCase()))) {
return {
verdict: 'MAYBE',
reason: Some but not all done keywords found.,
};
}
return {
verdict: 'NO',
reason: No done signals detected in output.,
};
}
VerifierOptions
takes four inputs:
agentOutput
is the raw string output from the agent.
doneKeywords
is a list of terms that must all appear in the output for it to pass.
failKeywords
is a list of terms that immediately fail the output if any one of them appears.
iffPattern
is a regular expression that triggers a conditional verdict when it matches.
The verify
function evaluates in a fixed priority order. Failure signals are checked first: if any failKeyword
appears in the output, the function returns NO
immediately without evaluating anything else. If all doneKeywords
are present, it returns YES
. If iffPattern
matches, it returns IFF
along with the matched string as the condition the agent needs to resolve before continuing. If only some doneKeywords
are present, it returns MAYBE
, pausing the loop for human review. If none of those conditions are met, it returns NO
.
The four verdicts map directly to actions in your agent loop. YES
updates state and continues. NO
retries the action. MAYBE
s and flags for human review. IFF
checks the named dependency before deciding either way.
All three components are built. Next, you version them and release them to your scope so Ripple CI can pick up the export automatically.
Tag all three components with a single command. This versions the modified components selected for tagging in one command:
bit tag --message "initial implementation of harness components"
You'll see output confirming all three components were tagged at version 0.0.1
:
Now push all three components to your scope on bit.cloud:
bit export
You'll see Bit indexing your components and confirming a successful push:
You may see a warning about node-env
not being loaded during export. Run bit install
locally to clear it.
The moment bit export
completes, Ripple CI picks up the push and kicks off a remote build job automatically. No configuration required. Head to the URL in the export output to watch compilation, tests and documentation generation run against all three components in the cloud.
Here's what a successful build looks like:
Ripple CI dashboard showing three harness components built successfully in 2 minutes 41 seconds
With your components live on bit.cloud, install all three into any agent project with a single command:
bit install @your-username/agent-harness.harness.state @your-username/agent-harness.harness.context @your-username/agent-harness.harness.verifier
If you're working outside a Bit workspace, install via npm with Bit's registry instead:
npm install @your-username/agent-harness.harness.state \
@your-username/agent-harness.harness.context \
@your-username/agent-harness.harness.verifier \
--registry https://node-registry.bit.cloud
With all three components installed, here's how they wire together at the boundaries of an agent session:
import { loadState, saveState } from '@your-username/agent-harness.harness.state';
import { buildDependencyMap } from '@your-username/agent-harness.harness.context';
import { verify } from '@your-username/agent-harness.harness.verifier';
// At session start
const state = loadState();
const context = await buildDependencyMap(['harness/state', 'harness/context', 'harness/verifier']);
console.log('Resuming from:', state);
console.log('Dependency map:', context);
// Agent does its work here
const agentOutput = 'scaffold auth module completed. tests passing.';
// Verify before continuing
const result = verify({
agentOutput,
doneKeywords: ['completed', 'tests passing'],
failKeywords: ['error', 'failed'],
});
console.log('Verdict:', result.verdict);
// Update state based on verdict
if (result.verdict === 'YES') {
state.done.push('scaffold auth module');
state.inProgress = [];
state.next = ['write tests for auth module'];
}
// At session end
saveState(state);
The context loads before the first action. The state picks up where the last session ended. The verifier evaluates output before the loop continues. When any of those three components changes in any project, tag and export from that project, and Ripple CI propagates the update to every downstream dependent automatically. That's the harness: three components, one install command, zero copy-paste.
You now have a production harness that lives outside your agent, versioned and shared across every project that needs it. When something breaks, you fix it in one place, tag it, export it and Ripple CI propagates the change automatically.
That's the difference between a harness you maintain and a harness that maintains itself.
The approach scales further than this tutorial goes. You can extend HarnessState
to track token usage, session duration or retry counts. You can add a fourth component that handles context window management, trimming what gets loaded based on what the state says is already done. You can wire the verifier into a CI step so no agent output merges without passing a a done condition first.
Every extension is just another versioned component added to the same scope. Once the pattern is in place, growing the harness stops being a rewrite and starts being an addition.