Pi Agent Harness: What a Unified LLM API and Agent Loop Reveal About Tool-Calling Boundaries

Pi hit 1.0 after nearly a year of development by the Gatsby team. It's trending at #8 on GitHub with 100K+ stars, positioned as a self-extensible coding agent with a unified multi-provider LLM API. The interesting part is not the coding agent itself. It's the runtime layer underneath: how Pi normalizes tool-calling across OpenAI, Anthropic, and Google, manages state across multi-step workflows, and explicitly punts on permission boundaries.

This is a case study in agent runtime design. Pi exposes the plumbing between reasoning (LLM calls) and execution (tool invocation), and it forces you to make a choice: convenience or isolation.

Every major LLM provider has a different tool-calling schema. OpenAI uses tools

with function

objects. Anthropic uses tools

with input_schema

. Google uses function_declarations

. Pi's @earendil-works/pi-ai

package abstracts this into a single interface.

Here's what that looks like:

import { createLLM } from '@earendil-works/pi-ai';

const llm = createLLM({
  provider: 'openai', // or 'anthropic', 'google', custom endpoint
  model: 'gpt-4',
  apiKey: process.env.OPENAI_API_KEY,
});

const response = await llm.chat({
  messages: [{ role: 'user', content: 'What is the weather in SF?' }],
  tools: [
    {
      name: 'get_weather',
      description: 'Get current weather for a location',
      parameters: {
        type: 'object',
        properties: {
          location: { type: 'string' },
        },
        required: ['location'],
      },
    },
  ],
});

The abstraction hides three things:

The impedance mismatch is real. OpenAI returns tool_calls

as an array. Anthropic returns content

blocks with tool_use

types. Google returns functionCall

objects. Pi's abstraction layer maps all of these to a single ToolCall

type with name

, arguments

, and id

.

The @earendil-works/pi-agent-core

package sits on top of the LLM API. It manages the agent loop: prompt, tool call, tool execution, result injection, repeat.

The runtime tracks:

When a tool call happens, the runtime:

tool

role message.This is a synchronous blocking loop. If a tool takes 30 seconds to run, the agent waits 30 seconds. If a tool fails, the error message goes back to the LLM, and the LLM decides what to do next (retry, skip, abort).

Pi does not have built-in retry logic or circuit breakers. If a tool throws an exception, the runtime catches it, serializes the error message, and appends it to the conversation history as a tool result with an error flag.

The LLM sees:

{
  "role": "tool",
  "tool_call_id": "call_abc123",
  "content": "Error: ENOENT: no such file or directory, open '/tmp/missing.txt'"
}

The LLM can:

This is a design choice. Pi treats the LLM as the orchestrator. The runtime is just plumbing. If you want retries, timeouts, or fallback logic, you implement them in your tool handlers or wrap the agent loop.

Pi's documentation is explicit: "Pi does not include a built-in permission system for restricting filesystem, process, network, or credential access. By default, it runs with the permissions of the user and process that launched it."

This is not an oversight. It's a trade-off. Adding a permission system means:

Pi punts on this. If you need isolation, you containerize. The docs outline three patterns:

The Gondolin pattern is interesting. It runs Pi on the host but executes tools inside a browser extension sandbox. The extension has limited filesystem access and no direct network access. Tool results flow back to Pi over a message-passing bridge.

Docker Compose is the middle ground. You define a docker-compose.yml

with volume mounts for the directories Pi needs to read/write, and you use Docker's network isolation to block outbound connections except to specific hosts (LLM APIs, internal services).

Kubernetes is the heavy option. You use pod security policies to drop capabilities, network policies to enforce egress rules, and resource quotas to prevent runaway tool execution.

Pi is four packages:

@earendil-works/pi-ai

@earendil-works/pi-agent-core

@earendil-works/pi-coding-agent

@earendil-works/pi-tui

The flow:

User input → TUI → Agent Core → LLM API → Provider (OpenAI/Anthropic/Google)
                        ↓
                   Tool Registry
                        ↓
                   Tool Handlers (filesystem, shell, etc.)
                        ↓
                   Tool Results → Agent Core → LLM API → Provider

The agent core is stateful. It holds the conversation history in memory. If the process crashes, you lose the session. There is no built-in persistence layer. If you need durable state, you wrap the agent core and snapshot the conversation history to disk or a database after each turn.

The coding agent can modify its own tools. It has a create_tool

tool that generates a new tool definition and registers it at runtime. The tool definition is TypeScript code. The agent writes it, saves it to disk, and dynamically imports it.

This is powerful and dangerous. The agent can:

There is no approval gate. If the LLM decides to create a tool, the tool gets created. If you're running Pi with your AWS credentials in the environment, the agent can create a tool that reads them and sends them to an external server.

The mitigation is containerization. If Pi runs in a container with no AWS credentials, no network access, and a read-only filesystem except for a scratch directory, the blast radius is limited.

Pi includes a @earendil-works/pi-telemetry

package. It defines vendor-neutral telemetry contracts: structured logs, traces, and metrics. The reference adapter writes to stdout in JSON format.

You can plug in your own adapter to send telemetry to Datadog, Honeycomb, or an OpenTelemetry collector. The telemetry schema includes:

This is useful for debugging multi-step workflows. You can trace a failed tool call back to the LLM response that triggered it, see the arguments that were passed, and inspect the error message.

Pi's failure modes are predictable:

The common thread: Pi does not hide errors. It surfaces them to the LLM and lets the LLM decide what to do.

Use Pi when:

Avoid Pi when:

Pi is plumbing. It solves the provider abstraction problem and gives you a basic agent loop. Everything else (security, persistence, observability, error handling) is your responsibility. That's a feature, not a bug. It keeps the runtime small and forces you to think about the boundaries that matter for your deployment.