# Fly.io AI Agents: Moving from LLMs to Virtual Machines

> Source: <https://promptcube3.com/en/news/2823/>
> Published: 2026-07-24 16:53:12+00:00

# Fly.io AI Agents: Moving from LLMs to Virtual Machines

## The "Sandboxed OS" Architecture

Standard prompt engineering for agents usually involves giving the LLM a set of tools (functions) it can call. The problem is that these tools are brittle. If a tool fails or returns an unexpected format, the agent loops or crashes. By deploying agents into their own dedicated VMs, the agent gains a full POSIX-compliant environment.

Instead of calling a `read_file()`

function, the agent simply executes `cat filename.txt`

in a bash shell. This shifts the burden of reliability from the tool-definition layer to the operating system layer.

To implement this locally for testing before deploying to a cloud provider, you can simulate this "agent-in-a-box" using Docker. Here is a basic configuration for a Python-based agent environment that restricts the agent's capabilities while providing a full shell:

```
# docker-compose.yml for Agent Sandbox
services:
  agent-env:
    image: python:3.11-slim
    volumes:
      - ./agent_workspace:/home/agent/workspace
    environment:
      - OPENAI_API_KEY=${OPENAI_API_KEY}
    working_dir: /home/agent/workspace
    deploy:
      resources:
        limits:
          cpus: '0.5'
          memory: 512M
    command: /bin/bash
```

## Why VMs Beat Function Calling

When you move from tool-calling to a VM-based AI workflow, you change the error-handling paradigm. In a traditional LLM agent setup, a syntax error in a generated script usually requires a complex feedback loop to the model. In a VM environment, the agent receives the actual stderr from the shell.

**State Persistence:** A VM maintains a filesystem. The agent can install a library via`pip`

, run a script, check the output, and then modify the script based on the result without the developer having to explicitly manage a "state" object in the prompt.**Dependency Management:** If an agent needs a specific version of a tool (e.g.,`ffmpeg`

for video processing), it can attempt to install it via`apt-get`

rather than the developer needing to pre-build every possible tool into the agent's API.**Security Isolation:** Running an agent in a VM provides a hard boundary. If the LLM generates a destructive command like`rm -rf /`

, it only kills the ephemeral instance, not the host server.

## Real-World Deployment Logic

For those building an LLM agent deployment, the key is managing the lifecycle of these VMs. You cannot keep a VM running 24/7 for every single user query due to cost. The optimal pattern is "Spin up -> Execute -> Snapshot/Destroy."

Here is a conceptual Python snippet showing how to manage an ephemeral agent session via a CLI interface:

``` python
import subprocess

class AgentSession:
    def __init__(self, session_id):
        self.session_id = session_id
        self.vm_id = None

    def start_vm(self):
        # Example command to spin up a lightweight Firecracker VM or Docker container
        cmd = f"docker run -d --name agent_{self.session_id} agent-base-image"
        self.vm_id = subprocess.check_output(cmd, shell=True).decode().strip()

    def execute_command(self, command):
        # Executing a command directly in the agent's "body"
        exec_cmd = f"docker exec agent_{self.session_id} /bin/bash -c '{command}'"
        result = subprocess.run(exec_cmd, shell=True, capture_output=True, text=True)
        return result.stdout if result.returncode == 0 else result.stderr

    def terminate(self):
        subprocess.run(f"docker rm -f agent_{self.session_id}", shell=True)

# Usage
session = AgentSession("user_123")
session.start_vm()
print(session.execute_command("ls -la"))
session.terminate()
```

## The Performance Trade-off

The shift to VM-based agents introduces latency. Cold-starting a VM takes longer than a simple API request. However, for complex tasks—like autonomous coding or data analysis—the time spent booting a VM is negligible compared to the time spent in a "hallucination loop" where a tool-based agent fails repeatedly because it lacks the proper environment context.

This approach turns the agent from a "chatbot with plugins" into a "remote operator." By providing a real filesystem and a real shell, we stop trying to simulate a computer inside a prompt and simply give the AI an actual computer.

[Claude Code Workflow: Open Weights vs. Closed Models 1h ago](/en/news/2807/)

[Claude Code: My Experience with Local Credential Scanning 2h ago](/en/news/2787/)

[Claude Code Workflow: Leveraging Open Weights for Local Dev 2h ago](/en/news/2773/)

[Oracle AI Pivot: 21,000 Layoffs to Fund Infrastructure 3h ago](/en/news/2749/)

[DeepSQL: Self-Hostable DBA Agent for Postgres & MySQL 4h ago](/en/news/2739/)

[AI Chips: The Great Hardware Sprint 5h ago](/en/news/2719/)

[Next Claude Code Workflow: Open Weights vs. Closed Models →](/en/news/2807/)

## All Replies （3）

[@CameronWizard](/en/users/CameronWizard/)Probably. New leadership usually means a hard reset on the roadmap, especially when the tech landscape flips this fast.