Every engineering organization experimenting with coding agents eventually hits the same wall. It rarely happens on day one—day one is a demo where an agent fixes a unit test in thirty seconds. It happens on day thirty, when an autonomous loop hits an unexpected error, hallucinates a workaround, pushes to the wrong branch, leaks a token into a build log, or burns $4,000 in model tokens overnight retrying a broken migration.
Most pitches for AI software factories lead with speed: how many agents you can fan out in parallel, or how fast a prompt becomes a pull request.
At Islo, we took the opposite starting point: in enterprise engineering, speed without hard boundaries is a liability. Fast often implies cutting corners on isolation, credential scoping, or human oversight. What actually unlocks autonomous throughput for an enterprise is safety enforced outside the agent’s decision loop.
Recently, the Agent Baseline project published The Secure Software Factory: A Reference Architecture. Its core thesis states the architectural requirement clearly:
“This architecture keeps what an agent decides apart from what it may do: the agent proposes, policy authorizes, and infrastructure enforces.”
Agent Baseline — The Secure Software Factory Reference Architecture
As teams move from Level 1 (Human-Orchestrated) to Level 2 (Workflow-Orchestrated) and Level 3 (Agent-Orchestrated) autonomy, an agent’s plans must never become its permissions. Four boundaries from the reference architecture—plus a fifth operational boundary around how the factory itself is versioned and budgeted—have to hold from outside the agent’s own decisions, whether the agent cooperates or not:
- Machine (Sandbox) : Where its code and services run
- Network : Which systems and operations it can reach
- Supply chain : Which components and sources it can use
- Identity and access : Who the agent acts for and what it may do
- Pinned versions & budget bounds : How Lines, Agents, and Machine builds evolve—and how much compute or model spend a run can consume
In this technical deep dive, we walk through how we architected the Islo platform around those five boundaries:
- Network, supply chain, and identity controls at the egress gateway (provider credentials stay out of the Machine)
- Strong hardware and perimeter isolation (dedicatedIslo Machines , in Islo Cloud or in your own cloud / on-prem via BYOC)
- Full audit and traceability (tamper-evident observability linked by run)
- Not spending money without bound (bounded stage transitions, model gateway caps, and real-time USD budget ceilings)
- Safest to make a change to (combiningPrompt ,Script , andDecision Agents inside version-pinnedLines on pinnedMachine builds , backed by outer-loop approval & steer controls and empirical reward-hacking defenses)
System Overview: Enforcing Policy Outside the Agent Loop #
To ensure an agent cannot alter the rules governing its own execution, Islo separates workflow orchestration, runtime scope authorization, isolated Islo Machines, and outbound network enforcement into distinct layers that map directly to the Agent Baseline Reference Architecture:
Following the flow from top to bottom:
- Trigger → Line, Agents & Machine Builds (
5) → Durable Execution Layer : A run starts from aTrigger (webhook, ticket, schedule, or CLI) and loads a version-pinnedLine declaring the stages, whichAgent (Prompt ,Script , orDecision ) runs at each stage, the pinnedMachine versions , bounded stage transitions, loop caps, and USD budget. Thedurable execution layer checkpoints state, enforces timeouts, and runs each stage on its configuredIslo Machine . - The Inner Loop — Islo Machine (
1) : InsideThe Factory (running either inIslo Cloud or inyour own BYOC environment), stages run onIslo Machines —dedicated microVMs with their own Linux kernel and your real stack. Inside the Machine, theStage Agent (Prompt · Script · Decision) executes its inner loop in an untrusted environment withzero real tokens , using only its configuredSkills & MCP , pinnedMachine build ,Scoped knowledge , and schema-validatedTyped outputs . - Runtime Authorization & Vault (
4) and Egress Gateway (2) : The stageproposes scope for its task, whileRuntime authorization & vault authorizes that task scope, holds real credentials in the vault, and passes theauthorized scope to theEgress gateway . Every outbound connection from the Machine passes through the default-deny gateway, which enforces:- Supply chain rules (
3) : Read-only package registries and approved hosts, paths, and HTTP methods. - Model gateway & budgets (
5) : Approved models, token caps, and real-time spend ceilings. - Connected integrations : Scoped access to source control, issue trackers, chat, and internal APIs—whileeverything else is denied .
- Supply chain rules (
- Observability : Every prompt, command, file diff, trajectory turn, and network call across the factory is captured out-of-band in a tamper-evident record linked by run.
- The Outer Loop — Pull Request → Approval & Steer → Merge : Verified stage outputs open aPull request for automated checks or humanApproval & steer . Review feedback or operator steering drives the next stage in the outer loop before approved changesMerge .
1. Network, Supply Chain, and Identity Controls at the Egress Gateway #
In a conventional CI runner or developer workstation, giving an agent access to GitHub, Slack, Linear, or a model provider means dropping GITHUB_TOKEN, SLACK_BOT_TOKEN, or ANTHROPIC_API_KEY into environment variables or files on disk.
Once a raw secret enters the execution environment, you no longer have a network boundary—you have a wish. Any prompt injection hidden in an issue body, malicious dependency install script, or hallucinated curl command can read the environment and exfiltrate the credential to an external server, or use that token against any repository or endpoint the token’s scope permits.
How the Egress Gateway Evaluates Traffic and Injects Credentials on the Wire
In Islo, real provider credentials never enter the Machine. Instead, the Islo Machine (1), Egress Gateway ( 2), Supply Chain Rules ( 3), and Connected Integrations & Vault ( 4) work together so the Stage Agent never touches a raw secret:
As shown in the gateway policy flow above:
- Islo Machine · MicroVM (
1) with Zero Real Credentials : Inside the untrusted environment, theStage Agent and standard CLIs (git,gh,npm,claude,codex,cursor) run with a non-sensitive placeholder token. If an agent dumps its environment variables or inspects configuration files on disk, there is nothing to steal. All outbound traffic from the microVM is routed at the host boundary into the egress gateway—an agent cannot bypass it via raw sockets, custom DNS, or direct IP connections. - Gateway Profile & Connected Integrations Vault (
4) : A declarativeGateway profile configures default-deny rules, allowed hosts, paths, HTTP methods, and content filters for the Machine. Real OAuth and API credentials stay inside theConnected integrations & vault , which scopes a token strictly to the authorized rule. - Three-Step Layer 7 Inspection inside the Egress Gateway (
2) :- 1. On-demand TLS & host identity : Rather than relying on SNI passthrough (which leaves encrypted payloads blind to inspection), the gateway terminates TLS on demand using a per-Machine CA pre-installed in the VM’s trust store and binds the request to the host-verified Machine ID.
- 2. Host, path & method rules : Evaluates the decrypted request against prioritized allow and deny rules in the Machine’s gateway profile.
- 3. Content filters & LLM judge : Evaluates payload regular expressions, MIME types, size limits, or an independent outboundLLM judge to block secret leaks or reward-hacking cheat queries—and injects the real scoped token on the wire only when the request is authorized.
This produces four deterministic outcomes on the wire:
- Authorized Integration (
api.github.com/repos/acme/*) :GETandPOSTrequests are allowed, and the real scoped token from the vault is injected on the wire outside the Machine. - Supply Chain Rules (
3·registry.npmjs.org) : Restricted toGET only—the Stage Agent can fetch packages to install dependencies, but cannot publish or mutate packages. - Disallowed Method or Scope (
DELETE /repos/acme/*or any other org) : Immediately rejected at the gateway withdenied (403). - Everything Else : Unlisted hosts and leaks are
deniedby default—and for verification or grading stages that require zero external connectivity, internet access can be disabled completely.
2. Strong Isolation: Dedicated Islo Machines and BYOC #
Container-based sandboxes share a single Linux kernel with the host and with adjacent workloads. For deterministic microservice code you wrote yourself, shared-kernel containers are reasonable. For an autonomous agent that compiles untrusted code, runs arbitrary shell scripts, starts Docker containers, and installs third-party packages, a shared kernel is a single kernel vulnerability away from a cross-tenant breakout.
Islo Machines: Real Developer Computers on Dedicated MicroVMs (1)
An Islo Machine is a complete, hardware-isolated developer computer—not a stripped-down container that can only run a single script:
- Dedicated Linux Kernel & Real Application Stack : Each Machine is a dedicated microVM with its own Linux kernel. Inside one Machine, agents can run your repositories, dependencies, PostgreSQL, Redis, Docker (
docker compose up), a real headless browser, and your full build toolchain—the exact same setup your engineers run locally, while the hypervisor boundary isolates the host. - Versioned Machine Builds (
source,build, andverify) : Machines are built and published as immutableMachine builds across progressive readiness levels—from repositorysourcecheckouts, tobuildenvironments with pre-compiled toolchains, to full-stackverifyenvironments with your application and services up and healthy. Warm pools and copy-on-write disk clones boot a Machine from its pinned build in milliseconds. - Out-of-Band Host Orchestration & Zero Lateral Movement : Host-to-guest command execution, file sync, and health probes travel over an isolated host-only channel rather than a shared TCP management network. Host firewall rules drop all Machine-to-Machine traffic by default, so one Machine can never ping, scan, or connect to another.
- Flexible Machine Scoping Across Stages : Each Agent binds to a Machine template, and stages in aLine can either share a persistent Machine across the run (preserving the checked-out branch, warm build artifacts, and running services from stage to stage) or run on separate Machines when you want a clean environment boundary or a different readiness level between stages—while only declared, schema-validatedTyped outputs drive transitions from one stage to the next.
Data Residency and Private Perimeter Isolation: Bring Your Own Cloud (BYOC)
For many enterprise security teams, hardware isolation between VMs is only half the question. The other half is where those Machines—and your source code, builds, and internal network traffic—physically reside. When an agent works on sensitive proprietary repositories, it often needs to reach systems that are never exposed to the public internet: self-hosted GitHub Enterprise or GitLab instances, internal package registries, private databases, and staging APIs. Sending source code to a third-party cloud—or opening inbound firewall holes into your private network so an external runner can reach staging—creates an unacceptable attack surface.
With Islo BYOC (Bring Your Own Cloud), the entire factory execution and network enforcement plane runs inside your cloud account or on-prem environment, while Islo Cloud operates the managed control plane:
As shown in the BYOC architecture diagram:
- Islo Cloud · Managed Control Plane (
operated by Islo) :- Lines & Agents (
5) : Manages version-pinned Lines,Prompt, Script & Decision Agents , triggers, and USD budget ceilings. - Gateway profiles & policies (
4) : Manages workspace RBAC, scoped gateway rules, and the tamper-evident audit viewer. - Managed fleet operations : Delivers zero-downtime host upgrades underneath running Machines and centralized capacity monitoring—without requiring your team to self-host or maintain a fork.
- Lines & Agents (
- Outbound-Only Connection · Zero Inbound Ports : Your hosts initiate an outbound-only connection to Islo Cloud. No inbound ports are opened into your network, andMachines keep running even if the control-plane link is interrupted .
- Your Cloud Account or On-Prem · BYOC (
owned by you) :- Islo Machines · MicroVMs (
1·Machines on your hosts) : Run on your own hosts with their own Linux kernel and zero real tokens.Warm pools ,Machine builds ,Source code , andWorking state stay inside your perimeter on local disk. - Local Egress Gateway (
2) : Enforced directly in your network—TLS inspection, host/path rules, and wire-level token injection happen locally soinjected credentials never pass through Islo Cloud . - Private Internal Systems (
3) : Agents reachGitHub Enterprise / GitLab ,Internal package registries , andPrivate DBs & staging APIs directly over your private network. - Observability : Streams out-of-band execution logs, trajectories, and the gateway audit trail from your hosts.
- Islo Machines · MicroVMs (
3. Full Audit and Traceability: Tamper-Evident Observability Linked by Run #
In the Agent Baseline Reference Architecture—and in both architecture diagrams above—the Observability bar spans the factory floor, linking every action across boundaries into a single tamper-evident record linked by run.
When an enterprise security team asks, “What did this agent do at 02:14 UTC, why did it touch that repository, and which policy authorized that outbound call?”, grepping standard output logs inside a container is not an audit strategy—especially because an untrusted process inside the Machine can modify or delete local files. (As we demonstrated in our verifier tampering research, any file inside the workload’s write boundary can be overwritten by a detached background process.)
Out-of-Band Collection Outside the Machine Write Boundary
Islo’s Observability pipeline is architected so that nothing running inside the Machine can forge, alter, or erase the audit trail:
- Out-of-Band Command & Execution Logging : The host captures every command execution, exit code, and terminal output stream directly from outside the guest VM. The Machine has no network route or credentials to the audit store, and the host stamps every record with verified workspace, user, Machine, and stage identifiers.
- Unified Agent Trajectories & File Diffs : Whether a stage runs aPrompt Agent (Claude Code ,Codex ,Cursor , orOpenCode ), a deterministicScript Agent , or aDecision Agent , Islo captures the execution stream from outside the guest and normalizes every turn into a unified timeline: prompts, model reasoning steps, tool calls, file diffs, token usage, and terminal recordings.
- Correlated Gateway Audit Trail : Every outbound HTTP/TLS request, matched gateway rule, content filter or LLM judge verdict, and wire-level token injection decision is linked to the exact Line run and stage that triggered it.
If an incident occurs—or if an engineer wants to inspect why a verification stage rejected a change—the entire chain of custody from the initial Trigger to the final Pull request is preserved in an append-only record outside the Machine.
4. Not Spending Money Without Bound: Model Gateway and Line Budget Ceilings (5) #
Autonomous agents fail differently from traditional CI pipelines. When a deterministic build script fails, it exits 1 in seconds. When an autonomous agent fails—especially inside a multi-stage workflow with review-and-rework loops—it can reason itself into an infinite retry cycle, spawning subagents, reading megabytes of repository context on every turn, and burning thousands of dollars in model tokens and compute over a weekend.
In our reference architecture diagram, financial and compute bounds (5) are enforced at two independent control points outside the agent: orchestration limits in Line, Agents & Machine builds (5) / Durable execution layer, and wire-level limits in Model gateway & budgets (5):
1. Bounded Stage Transitions and Loop Caps (Line, Agents & Machine builds)
Every Line bounds both individual stage retries and feedback loops between stages. A review-and-rework transition from a verification stage back to an implementation stage (when verification outputs needs_changes or failed) carries a hard loop cap (for example, at most 3 review revisions or 2 fix cycles). Once that transition reaches its cap, the loop cannot fire again—preventing an oscillation between coding and review stages from running forever.
2. Line-Wide Iteration, Timeout, and USD Budget Ceilings (Durable execution layer)
Above individual transitions, the durable execution layer enforces a global envelope on every Line run:
- Maximum total iterations : Caps the total number of stage executions across the entire Line.
- Line USD budget ceiling : Tracks cumulative model and Machine spend across every stage attempt in real time.
- Stage and approval timeouts : Enforces hard wall-clock duration ceilings on every stage execution as well as on humanApproval & steer s.
If a run crosses its loop cap, exceeds its timeout, or reaches its USD budget ceiling, the durable execution layer immediately halts further stage scheduling and parks the run before another Machine or model turn can start.
3. Model Gateway & Budgets (5 · Wire-Level Model Enforcement)
Because all model inference traffic from a Stage Agent passes through the Egress gateway (2) into Model gateway & budgets ( 5), Islo enforces model governance on the wire:
- Approved Models Only : Agents can only call model providers and model IDs permitted by policy.
- Token Caps & Real-Time Spend Ceilings : The model gateway meters input, output, and cached tokens on every turn against the run’s and workspace’s real-time spend ceilings, cutting off runaway inference immediately if a ceiling is reached.
4. Workspace Machine Quotas
At the workspace level, hard capacity ceilings on concurrent Islo Machines, vCPUs, memory, and disk allocation ensure that an external webhook storm or a misconfigured schedule can never exhaust your compute fleet.
5. Safest to Make a Change To: Purpose-Built Agents (Prompt · Script · Decision), Pinned Machine Builds, and Outer-Loop Control ( 5) #
There is a popular narrative in AI circles around “self-improving software factories”—systems where agents autonomously rewrite their own system prompts, edit their own skills, alter workflow routing, and mutate their own execution environments after every run.
In a research prototype, unconstrained self-modification looks magical. In an enterprise production environment, an unconstrained self-modifying factory is a security and reliability nightmare.
Consider what happens if an agent processes an untrusted external issue, pull request, or log payload, and that run is allowed to mutate the factory’s shared prompts, skills, or routing rules for future runs:
- Persistent Cross-Run Prompt Injection : Untrusted input from one ticket can plant instructions into shared state that silently alter how future runs behave across unrelated repositories.
- Zero Reproducibility : When a production workflow breaks on Thursday after succeeding on Tuesday, nobody can tell which silent self-mutation changed the factory’s behavior.
- Unreviewed Privilege Escalation : As theAgent Baseline Level 3 architecture warns, when an agent chooses the next step,its plans must never become its permissions.
At Islo, we believe the safest software factory is the one that is safest to make a deliberate, reviewable, deterministic change to.
1. Three Purpose-Built Agent Types: Prompt, Script, and Decision
A common mistake when building agent workflows is handing every step to an open-ended coding agent—asking an LLM to run the build, decide whether tests passed, and choose what to do next. When a model controls steps that should be deterministic, it can rationalize a test failure or hallucinate a passing check.
In Islo, each stage of a Line runs one of three explicit Agent types (Stage Agent (Prompt · Script · Decision) in the architecture diagram), matching the authority of the worker to the nature of the step:
- Prompt Agents (for work that needs engineering judgment ): An AI coding agent (Claude Code ,Codex ,Cursor , orOpenCode ) follows your prompt on its assignedMachine with its configured model, reasoning effort,Skills & MCP servers, andScoped knowledge . Prompt Agents handle tasks like planning a change, implementing a fix, reviewing a diff, or driving a browser to reproduce a bug.
- Script Agents (for deterministic checks—no AI, same result every time ): Runs your shell commands in exact order on the Machine and stops at the first failure. Builds, linters, typechecks, and test suites execute as Script Agents so a model can never skip, mock, or talk its way past a red check.
- Decision Agents (for structured questions from text or images ): Evaluates text or an image (such as a UI screenshot or verification artifact) against named, schema-typed questions (booleans, enums, or structured fields) without open-ended shell or code-editing access—turning evidence into clean, auditable routing decisions.
2. Declarative Lines and Pinned Machine Builds (5)
In the top row of our architecture diagram (5 · Line, Agents & Machine builds), factory processes are never stored in mutable agent memory. They are declarative, version-controlled definitions validated before deployment:
- Explicit Stage Contracts & Typed Outputs : Each stage declares its input parameters and schema-validatedTyped outputs (for example,
result = "approved" | "needs_changes" | "blocked"alongside the structured report). Conditional transitions compare those typed enum outputs directly rather than parsing free-form prose, so a phrase quoted inside a report can never accidentally trigger the wrong transition. - Pinned Machine Builds & Readiness Gates : Every Agent binds to anIslo Machine , and every Line declares its required Machine readiness level (
source,build, or full-stackverify). When a Line runs, it pins the publishedMachine build version across the run—and if a Machine is still upgrading to the requiredverifylevel, runs wait cleanly rather than executing against an incomplete environment. - Separating Scoped Knowledge from Procedural Control : Inside each Machine (
1),Scoped knowledge (domain memories and reference notes scoped by repository) is kept strictly separate fromSkills & MCP and Line routing rules, ensuring ambient notes can never hijack how a Line executes.
3. Outer-Loop Human & Automated Controls: Pull Request → Approval & Steer → Merge
At the bottom of the architecture diagram, the Outer Loop keeps humans and deterministic checks in command of what ships:
- Stop & Inspect : Because thedurable execution layer checkpoints state at every stage boundary, operators can halt any in-flight Line run cleanly and inspect its fullObservability trail.
- Approval & Steer : High-impact transitions or generatedPull requests for automated verification checks or a human operator. Operators (or a boundedLine Manager agent operating under explicit transition caps) can approve, ask a human question, orsteer a stopped run back to a stage with type-checked parameter overrides—driving the next stage in the outer loop until the change is ready toMerge .
4. Safely Adopting New Agents and Frontier Models: Lessons from Our Reward-Hacking Research
The hardest moment to keep a software factory safe is when you upgrade a Prompt Agent to a newly released frontier model or swap in a new coding harness. More capable models don’t just write better code—they are also dramatically more resourceful at finding shortcuts around naive guardrails when a test fails or a deadline approaches.
Our security research on RewardHackBench and verifier tampering directly shaped how Islo makes new models and agents safe to roll out:
- Models Take Shortcuts Without Being Asked : In ourRewardHackBench evaluations of frontier models on SWE-bench and CyBench tasks, agents cheated on8% of unprompted control runs —autonomously searching the web for published writeups or upstream patches—and on100% of runs when nudged by time pressure. In a production factory, the equivalent shortcut is an agent commenting out a failing test, weakening a linter rule, or mocking out a broken integration just to make the stage pass.
- Why Static Blocklists Fail and Gateway Judges Succeed : When we tested static hostname blocklists and regex content filters against frontier agents, agents bypassed them by cloning sparse git pack streams, hitting alternative mirrors, or fetching JSON-wrapped page data that evaded source-code regexes. Only anoutbound LLM judge at theEgress gateway (
2) —evaluating counterfactual intent on every request—drove the cheat rate to0% while preserving every honest solve. - Why Verification Needs an Independent Boundary (
1) : In ourverifier tampering study , we showed that when an untrusted coding step shares a mutable runtime with the grading harness, a detached background process can overwrite the verifier’s score file every 50 milliseconds—turning a failed test suite into a passing verdict.
Because of these findings, Islo Lines are designed so no agent grades its own homework:
- Independent Stage Agents & Machine Boundaries (
1) : Implementation, code review, and verification run as distinct stages with separateAgents (implementer,reviewer,verifier, or deterministicScript /Decision Agents ). Stages can share a Machine when fast local state continuity is desired, or bind to separate Machines and distinctgateway profiles when you want complete runtime isolation between coding and verification. - Verification on
verify-Level Machines Under Locked-Down Profiles (2&3) : Verification stages run against a full-stackverifyMachine with your real services up, verify that the running application matches the exact reviewed commit SHA, and execute with internet disabled or behind an outbound LLM judge so the verifier’sTyped outputs (passedvs.failed) cannot be tampered with. - Evaluating New Models Before Promotion (
5) : Because model and harness settings live on theAgent while your engineering process lives in theLine , you can swap a single Prompt Agent to a new frontier model, benchmark it against historical tickets andDoubleAgent stateful fakes, and inspect itsObservability trajectory before promoting it to production.
Why “Safest” Is What Actually Makes You Fast in the Enterprise #
When startups evaluate coding agents, they often optimize for how fast a single developer can spin up a prototype. When enterprises evaluate a software factory, the bottleneck is never raw token generation speed—the bottleneck is institutional trust:
| Reference Architecture Boundary | Enterprise Question | How Islo Enforces the Answer Outside the Agent |
|---|---|---|
| 1 · Machine | “Can untrusted code escape the sandbox—or force us to ship code outside our network?” | Dedicated Islo Machines + BYOC. Runs on hardware-isolated microVMs with their own Linux kernel, your real stack, and zero real tokens—in Islo Cloud or inside your own cloud / on-prem network viaBYOC . |
| 2 · Network | “Can a prompt injection or malicious script exfiltrate data or bypass egress rules?” | Default-deny egress gateway. Host-enforced Layer 7 TLS inspection, host/path/method rules, regex content filters, and outbound LLM intent judges; everything else is denied. |
| 3 · Supply chain | “Can an agent publish packages, push to unapproved repos, or pull from untrusted sources?” | Supply chain rules at the gateway. Read-only package registries (GET only) and explicit repository/organization path and HTTP method allowlists. |
| 4 · Identity & access | “Can an agent steal or misuse our GitHub, Slack, or cloud credentials?” | Runtime authorization & vault. Zero real credentials inside the Machine; scoped OAuth/API tokens are injected on the wire only for requests matching the stage’s authorized scope. |
| 5 · Pinned versions & budget bounds | “What stops silent workflow drift, hallucinated test passes, or a runaway loop burning $10,000 overnight?” | Lines, Prompt/Script/Decision Agents & Model gateway. Deterministic Script Agents for checks, Decision Agents for structured questions, pinned Machine builds, bounded transition loops, model gateway spend ceilings, and outer-loopApproval & steer controls—backed by tamper-evidentObservability . |
Speed without safety forces engineers to babysit every terminal session, turning developers into human firewalls for their agents. By enforcing Machine isolation, network and supply-chain policy, runtime authorization, spend bounds, and version-pinned Lines across Prompt, Script, and Decision Agents outside the agent loop, Islo lets engineering teams delegate real work with confidence.
If you are designing or evaluating a secure software factory for your engineering organization, read the Agent Baseline Reference Architecture, explore our documentation, or reach out to us at [email protected].