AI Harness: the worst and the best buzzword in the industry

---
title: "AI Harness: the worst and the best buzzword in the industry"
published: false
tags: [ai, harness, middleware, finops, aws, bedrock, opensource]
series: "TokenOps on AWS"
cover_image: # TODO: circuit-breaker / middleware diagram
---

"El mercado habla de 'AI Harness' como si fuera magia. El verdadero arnés de un LLM es un Proxy Inverso y un Middleware Transaccional determinístico. Es el código tradicional (styrr-llm y sayay-guard) el que confina, audita y presupuesta la inferencia probabilística antes de que toque tu infraestructura en la nube."

— TokenOps raw research, Turno 8

"Harness" is the most polarizing word in AI engineering right now. Depending on who you ask it's either the industry's worst buzzword or the best technical concept ever packaged badly. It's both — and the difference is whether you can name the actual engineering underneath.

requests

script or an Express server that wraps the OpenAI or Bedrock API.Strip the LinkedIn marketing and the original test harness metaphor becomes genuinely powerful for generative AI: electrical isolation of uncertainty.

An LLM is a highly unstable, probabilistic component. You cannot wire it directly into a bank's production database. You need a physical code "harness" that isolates it. When the model goes crazy — spewing corrupt text or prompt injections — the harness acts as a circuit breaker / thermal fuse that absorbs the impact and cuts the current.

A good harness guarantees the LLM never holds control logic. The model only processes text. The harness handles:

styrr-llm

does)sayay-guard

does)Post 2 of TokenOps on AWS. In post 1 we established the ontology as the grounding ledger; here we name the harness for what it is — transactional middleware. Post 3 completes the picture by stripping every buzzword down to infrastructure primitives.

The real "AI Harness" is three composable packages, all published, all zero hard dependencies:

npm install @carloscortezcloud/sayay-guard   # budget control (the financial circuit breaker)
npm install @carloscortezcloud/styrr-llm      # physical routing (the reverse proxy)
npm install @carloscortezcloud/tinkuy-agent   # format translation (the schema middleware)

Budget is checked before inference, recorded after. On block, it raises a native TokenBudgetExceededException

that Step Functions matches in its Catch block — the harness cuts the current before the retry bill grows:

import { SayayGuard, DynamoStorage } from '@carloscortezcloud/sayay-guard';

const guard = new SayayGuard({
  storage: new DynamoStorage({ tableName: 'sayay-ledger' }),
  budget: { dailyUsd: 5 },
});

// Throws TokenBudgetExceededException on block → ASL ErrorEquals catch
const decision = await guard.checkOrThrow('user-42', 0.005);
// ASL: the harness's circuit breaker trip
"Catch": [{ "ErrorEquals": ["TokenBudgetExceededException"], "Next": "HandleBudgetExceeded" }]

Physical routing of inference — decide in microseconds which endpoint gives the best cost per compute unit:

import { StyrRouter } from '@carloscortezcloud/styrr-llm';

const router = new StyrRouter({
  apiKey: process.env.OPENROUTER_API_KEY!,
  models: [
    { id: 'anthropic.claude-3-sonnet-20240229-v1:0', provider: 'bedrock' },
    { id: 'meta-llama/llama-3.3-70b-instruct:free', provider: 'openrouter' },
  ],
});

1. Isolation, not magic. A harness's job is to isolate the black box: the model stays behind the middleware, the production system stays in front. Nothing else.

2. Separation of responsibilities. Auth, routing, budget, format translation — four traditional software jobs, four layers, zero "agentic" smoke.

3. TokenOps renames the harness. From the raw research: "Al llamarlo por su nombre técnico (Middleware, Proxies, Circuit Breakers), educas a la comunidad de AWS y demuestras que el control de la IA no se logra con más IA, sino con ingeniería de software robusta y tradicional."

block

, not warn

, is what stops the bill.npm install @carloscortezcloud/sayay-guard

npm install @carloscortezcloud/styrr-llm

TokenBudgetExceededException

pattern"The No-Buzzwords Manifesto: your AI stack is Buffers, Load Balancers, and State Machines"

Built by Carlos Cortez — AWS Community Hero, Lima, Perú. Part of the TokenOps open-source ecosystem.