AX: Google's Open Agentic Orchestrator Explained — Building Production AI Agent Workflows

Originally published at adityarawas.in

Every framework promising to "orchestrate agents" eventually collapses into the same three problems: state management across long-running tasks, tool-call reliability, and observability when something silently breaks at 3am. AX — Google's open agentic orchestrator — is the first framework in this space that treats those three problems as first-class citizens instead of afterthoughts bolted onto a prompt-chaining library. If you've shipped anything with LangChain or a homegrown agent loop, you already know why that matters.

This is a practical breakdown of what AX actually does, how it's architected, and how to wire it into a real Node.js/TypeScript backend with Docker for deployment.

AX is an open-source orchestration layer for AI agents that sits between your LLM calls and your application logic. Unlike prompt-chaining libraries that treat agents as a sequence of function calls, AX models agents as stateful graphs with explicit transitions, retries, and checkpoints.

The core primitives:

This is closer to a workflow engine (think Temporal or AWS Step Functions) than a chatbot SDK — which is exactly the gap it fills. Most agent frameworks are great for demos and terrible for anything that needs to survive a network blip or a rate-limited API three steps into a 20-step task.

If you're building agentic features into a product — not a research notebook — you care about things AX explicitly designs for:

That last point is the sleeper feature. You don't need to rewrite your Go microservices in Python to make them agent-callable — AX treats any HTTP/gRPC endpoint as a valid executor.

npm install @google/ax-orchestrator
npm install @google/ax-store-redis

A minimal agent graph in TypeScript:

import { Graph, Executor, RetryPolicy } from '@google/ax-orchestrator';
import { RedisStateStore } from '@google/ax-store-redis';

const store = new RedisStateStore({
  url: process.env.REDIS_URL!,
});

const fetchData: Executor = {
  id: 'fetch-user-data',
  type: 'tool',
  run: async (ctx) => {
    const res = await fetch(`https://api.internal/users/${ctx.input.userId}`);
    if (!res.ok) throw new Error(`Upstream failed: ${res.status}`);
    return res.json();
  },
  policy: RetryPolicy.exponentialBackoff({ maxAttempts: 3, baseMs: 500 }),
};

const summarize: Executor = {
  id: 'summarize-with-llm',
  type: 'llm',
  model: 'gemini-2.5-pro',
  prompt: (ctx) => `Summarize this user profile in 2 sentences:\n${JSON.stringify(ctx.state.fetchData)}`,
};

const graph = new Graph({ store })
  .addNode(fetchData)
  .addNode(summarize)
  .connect('fetch-user-data', 'summarize-with-llm');

export async function runAgent(userId: string) {
  const run = await graph.start({ input: { userId } });
  return run.result;
}

Notice there's no manual try/catch retry loop, no ad-hoc setTimeout backoff — that's the whole point. The policy is declared once, attached to the node, and the orchestrator enforces it.

The architectural bet AX makes is that agents are workflows, not chat sessions. That reframing is why it plugs so cleanly into existing DevOps tooling — checkpoints are just rows in Postgres, traces are just OTel spans, and scaling is just adding more orchestrator replicas behind a load balancer.

Real-world use cases rarely involve a single LLM call. Here's a research-and-report pipeline with conditional branching:

import { Graph, Executor, Condition } from '@google/ax-orchestrator';

const searchWeb: Executor = {
  id: 'search-web',
  type: 'tool',
  run: async (ctx) => searchAPI(ctx.input.query),
};

const validateResults: Executor = {
  id: 'validate-results',
  type: 'llm',
  model: 'gemini-2.5-flash',
  prompt: (ctx) => `Are these search results relevant to "${ctx.input.query}"? Answer YES or NO.\n${JSON.stringify(ctx.state.searchWeb)}`,
};

const draftReport: Executor = {
  id: 'draft-report',
  type: 'llm',
  model: 'gemini-2.5-pro',
  prompt: (ctx) => `Write a report using:\n${JSON.stringify(ctx.state.searchWeb)}`,
};

const refineQuery: Executor = {
  id: 'refine-query',
  type: 'llm',
  model: 'gemini-2.5-flash',
  prompt: (ctx) => `Rewrite this search query to be more specific: "${ctx.input.query}"`,
};

const graph = new Graph()
  .addNode(searchWeb)
  .addNode(validateResults)
  .addNode(draftReport)
  .addNode(refineQuery)
  .connect('search-web', 'validate-results')
  .connectConditional('validate-results', {
    onTrue: 'draft-report',
    onFalse: 'refine-query',
  })
  .connect('refine-query', 'search-web'); // loop back

This loop-back edge is where most naive agent implementations fall apart — infinite loops without a cap. AX solves it with built-in cycle guards:

const graph = new Graph({
  maxCycles: 5,
  onCycleLimitExceeded: 'fail-with-partial-result',
});

AX ships OpenTelemetry instrumentation out of the box. Every node execution emits a span with input/output/token-usage metadata:

import { AXTracer } from '@google/ax-orchestrator';

const tracer = new AXTracer({
  exporter: 'otlp',
  endpoint: process.env.OTEL_COLLECTOR_URL,
});

graph.attachTracer(tracer);

Pipe that into Grafana/Tempo and you get a flame graph of your agent's decision tree — which node retried, how many tokens each LLM call burned, and where latency actually lives. This is the difference between debugging an agent by re-reading chat transcripts versus debugging it like you'd debug a distributed system, because that's what it is.

Because the orchestrator is stateless, containerizing it is straightforward:

FROM node:20-alpine AS builder
WORKDIR /app
COPY package*.json ./
RUN npm ci
COPY . .
RUN npm run build

FROM node:20-alpine
WORKDIR /app
ENV NODE_ENV=production
COPY --from=builder /app/dist ./dist
COPY --from=builder /app/node_modules ./node_modules
EXPOSE 3000
CMD ["node", "dist/server.js"]

docker-compose.yml for local development with Redis as the state store:

version: "3.9"
services:
  ax-orchestrator:
    build: .
    ports:
      - "3000:3000"
    environment:
      - REDIS_URL=redis://redis:6379
      - OTEL_COLLECTOR_URL=http://otel-collector:4318
    depends_on:
      - redis
  redis:
    image: redis:7-alpine
    ports:
      - "6379:6379"
  otel-collector:
    image: otel/opentelemetry-collector:latest
    ports:
      - "4318:4318"

Scale horizontally by just increasing replicas — since state lives in Redis/Postgres, any orchestrator instance can pick up a checkpointed run.

docker compose up --scale ax-orchestrator=3

The pattern that separates toy agents from production agents is explicit failure handling per node:

const paymentExecutor: Executor = {
  id: 'process-payment',
  type: 'tool',
  run: async (ctx) => chargeCard(ctx.state.amount, ctx.state.cardToken),
  policy: {
    retry: { maxAttempts: 2, baseMs: 1000 },
    onFailure: 'compensate',
    compensate: async (ctx) => {
      await refundIfCharged(ctx.state.transactionId);
    },
  },
};

That compensate hook is essentially the Saga pattern from distributed transactions, applied to agent workflows. If you've built payment or booking systems, this should feel immediately familiar — because it's solving the same problem.

AX is overkill for:

It earns its complexity when you have:

Pick based on workflow duration and payload size — don't default to Postgres for a workflow that completes in under 10 seconds; Redis will save you real latency.

compensate hook makes AX suitable for workflows with real side effects like payments or bookings.