Most agent frameworks treat execution as ephemeral scripts. You run a loop, it calls tools, maybe it logs to a file, then it exits. ZenML's new Kitaru project takes a different approach: wrap agents in the same durable infrastructure you'd use for batch ML pipelines. That means versioned artifacts, replayable execution graphs, and observability hooks borrowed from MLOps tooling.
Hamza Tahir, ZenML's founder, discussed the project on the Practical AI podcast. The core idea is simple: if you can version a trained model and replay its training run, you should be able to version an agent's state and replay its decision path. The gap between agent demos and production systems is mostly plumbing, and MLOps already solved half of it.
A harness is not a framework. It's a runtime wrapper that sits between your agent code and the execution environment. Think of it as a sidecar that captures state transitions, tool invocations, and decision points without forcing you to rewrite your agent logic.
Core responsibilities:
The harness doesn't dictate your agent architecture. You can run ReAct loops, plan-and-execute patterns, or custom state machines. The harness just ensures you can inspect, debug, and reproduce what happened.
ZenML built its reputation on pipeline orchestration for batch ML. Kitaru ports three primitives to agent execution:
In MLOps, a pipeline is a directed acyclic graph of steps. Each step consumes artifacts, runs code, and produces new artifacts. Agents break this model because they loop and branch based on runtime conditions.
Kitaru's solution: treat each agent iteration as a pipeline step. The loop becomes a meta-pipeline that spawns child pipelines. You get the same artifact tracking and caching semantics, but the graph can grow dynamically.
Every tool call result, every LLM response, every state transition gets written to an artifact store. This is not just logging. Artifacts are immutable, versioned, and queryable. You can diff two agent runs to see where their decision paths diverged.
If you replay an agent run and the inputs haven't changed, Kitaru can skip expensive tool calls and reuse cached results. This is critical for debugging. You don't want to re-run a $50 API call just to test a downstream logic branch.
| MLOps Primitive | Batch ML Semantics | Agent Semantics | Kitaru Adaptation |
|---|---|---|---|
| Pipeline | Static DAG | Dynamic loop with branches | Meta-pipeline spawns child steps per iteration |
| Artifact | Model weights, datasets | Tool outputs, LLM responses, state snapshots | Immutable, versioned, diffable |
| Caching | Skip training step if inputs match | Skip tool call if context matches | Content-addressed cache with replay flag |
| Observability | Metrics, logs, lineage | Traces, decision logs, tool call spans | Unified trace format for existing APM tools |
Kitaru doesn't replace your agent framework. It wraps it. You keep using LangChain, AutoGPT, or custom code. The harness intercepts execution at three points:
1. Initialization
The harness loads the agent's configuration (model, tools, prompts) from a versioned artifact store. It sets up observability hooks and initializes the replay cache.
2. Execution Loop
Each iteration of the agent loop becomes a pipeline step. The harness captures:
3. Termination
The harness writes a final artifact containing the full execution trace, all intermediate states, and a replay manifest. You can load this artifact later to resume or debug.
from kitaru import AgentHarness
from your_agent import MyReActAgent
harness = AgentHarness(
agent=MyReActAgent(),
artifact_store="s3://my-bucket/agents",
enable_replay=True,
cache_tool_calls=True
)
result = harness.run(
task="Find the cheapest flight to Berlin",
max_iterations=10
)
harness.replay(
run_id="abc123",
from_step=5,
override_tool_results={"search_flights": cached_results}
)
Replay is not just re-running the same code. It's reconstructing the exact decision path the agent took, then optionally diverging at a specific step.
What Kitaru snapshots:
Replay modes:
Kitaru emits OpenTelemetry traces. Each agent iteration is a span. Tool calls are child spans. LLM requests are nested spans with token counts and latency.
This plugs into your existing observability stack. If you're running Jaeger, Honeycomb, or Datadog, you get agent traces alongside your API traces. You can correlate an agent's decision with the downstream services it called.
Key metrics Kitaru tracks:
Agent harnesses introduce new failure surfaces:
| Failure Mode | Symptom | Mitigation |
|---|---|---|
| Artifact store latency | Slow agent startup, high write latency | Use local cache with async upload |
| Replay divergence | Replayed run produces different results | Snapshot external API responses, not just tool outputs |
| State explosion | Artifact store fills with GB of traces | Prune old runs, compress snapshots, sample traces |
| Cache invalidation | Stale tool results break replay | Content-address cache keys, include tool version in hash |
The biggest risk is assuming replay will always be deterministic. If your agent calls a live API that returns different results, replay will diverge. Kitaru can snapshot API responses, but you need to decide whether that's acceptable for your use case.
Good fit:
Bad fit:
Use Kitaru if you replay agent runs at least twice per week for debugging, your artifact storage budget is under $500 per month, or you need compliance audit trails showing exact decision paths. It's a strong fit when agents run 10+ iterations per task and you're already using ZenML or similar MLOps tooling. The harness overhead (5-15% latency, 2-10 GB storage per 100 runs) pays off when you can skip re-running expensive tool calls during debugging.
Avoid it if your agents are stateless request-response systems where standard API logging suffices. Skip the harness if you run agents in ephemeral containers without persistent storage, or if you're still in prototype phase with fewer than three production agents. The replay infrastructure adds complexity that only matters when you're debugging non-deterministic failures or need to reconstruct past decisions for auditing.
The decision hinges on replay frequency and storage cost. If you're not looking at old agent runs to understand failures, you don't need the harness. If you are, and you can afford 100-500 MB per agent run, Kitaru gives you the same debugging experience you'd expect from batch ML pipelines.