{"slug": "your-agent-s-free-text-output-is-an-api-you-never-designed", "title": "Your Agent's Free-Text Output Is an API You Never Designed", "summary": "A developer argues that agent pipelines fail at the boundary where free-text model output is parsed by host code, effectively creating an undocumented API, and proposes validating a typed decision object (status, confidence, findings) before any state change. The pattern, illustrated with a TypeScript assertDecision guard and a ReviewDecision schema, separates contract violations from judgment errors and implementation bugs, and is described as the basis for structured judgment tools exposed over MCP. The author maintains JevCases, an independent index of Jev use cases, and notes it is unaffiliated with TypeSafe.", "body_md": "Most agent demos fail in the same place, and it is not the model.\n\nIt is the boundary where the model's output leaves the model and enters your program. Up to that point it is a string. Strings do not have a schema.\n\nSo you end up with code like this:\n\n``` js\nconst reply = await llm.complete(prompt);\n\n// please work\nif (reply.toLowerCase().includes(\"approve\")) {\n  await merge();\n} else {\n  await requestChanges();\n}\n```\n\nThis works until the model writes *\"I would not approve this yet\"* and your substring check matches `approve`. Now a change that should have been blocked got merged.\n\nAny time you parse meaning out of generated text, you have declared an API. You just did not write it down.\n\nThat interface has properties you probably did not intend:\n\nThe fix is not a better prompt. Prompt engineering narrows the failure rate; it does not remove the parser.\n\nAn agent step usually needs two different outputs, and they should not be the same output:\n\nThe decision should come from a closed set that your code already understands:\n\n```\ntype ReviewDecision = {\n  status: \"pass\" | \"review\" | \"fail\";\n  confidence: \"low\" | \"medium\" | \"high\";\n  findings: Array<{ file: string; note: string; severity: \"info\" | \"warn\" | \"block\" }>;\n};\n```\n\nNow the failure modes separate cleanly:\n\n| What broke | Where to look | \n|---|---|\n| `status` came back as`\"probably fine\"` | Contract violation — reject before acting | \n| `status` was valid but wrong | Judgment problem — improve evidence or prompt | \n| `status` was right but the wrong branch ran | Implementation bug in your own code | \n\nWithout the typed boundary, all three look like \"the AI did something weird,\" and you have no way to tell them apart.\n\nThe important part is not the schema. It is that the schema is checked *before* anything changes.\n\n``` js\nconst ALLOWED = new Set([\"pass\", \"review\", \"fail\"]);\n\nfunction assertDecision(value: unknown): ReviewDecision {\n  if (typeof value !== \"object\" || value === null) {\n    throw new Error(\"Decision is not an object\");\n  }\n  const d = value as Record<string, unknown>;\n  if (typeof d.status !== \"string\" || !ALLOWED.has(d.status)) {\n    throw new Error(`Illegal status: ${String(d.status)}`);\n  }\n  if (!Array.isArray(d.findings)) {\n    throw new Error(\"findings must be an array\");\n  }\n  return {\n    status: d.status as ReviewDecision[\"status\"],\n    confidence: d.confidence === \"high\" || d.confidence === \"medium\" ? d.confidence : \"low\",\n    findings: d.findings as ReviewDecision[\"findings\"],\n  };\n}\n```\n\nA thrown error is a feature here. It is a loud, recoverable failure that happens before a merge, a payment, or a deploy — instead of a silent wrong branch.\n\nThis is the pattern behind structured judgment tools, like the [Choice and Score interfaces exposed over MCP in this case](https://jevcases.com/cases/typesafe-mcp/): the model supplies the judgment, the typing layer fixes its shape, and the calling program gets a value it can compare or gate on instead of prose it has to interpret.\n\nA typed result is not automatically trustworthy. `status: \"pass\"` with no support is just a shorter guess.\n\nSo the decision record should carry:\n\nThat gives you a replayable record. When a bad decision ships, you can tell whether the right evidence was missing, the wrong rule was applied, or the validation layer was too loose.\n\nTyping the output does **not**:\n\nIt solves one narrow problem: the host no longer has to guess a control signal out of decorative prose.\n\nThat is worth a lot. It is much easier to reason about a system where you can see the allowed choices, the selected choice, the evidence, and the resulting action.\n\nBefore an agent step is allowed to change state, I want answers to these:\n\nIf those are all \"no,\" the demo can still look impressive. It is just not yet an instrument.\n\n*Disclosure: I maintain [JevCases](https://jevcases.com/), an independent index of Jev use cases and experiments. It is not affiliated with TypeSafe.*", "url": "https://wpnews.pro/news/your-agent-s-free-text-output-is-an-api-you-never-designed", "canonical_source": "https://dev.to/ruixuan_jiang_663eb75f3fa/your-agents-free-text-output-is-an-api-you-never-designed-3mb9", "published_at": "2026-09-25 02:40:58+00:00", "updated_at": "2026-09-25 02:58:52.635876+00:00", "lang": "en", "topics": ["ai-agents", "agent-protocols", "ai-tools", "developer-tools"], "entities": ["JevCases", "TypeSafe", "MCP"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/your-agent-s-free-text-output-is-an-api-you-never-designed", "markdown": "https://wpnews.pro/news/your-agent-s-free-text-output-is-an-api-you-never-designed.md", "text": "https://wpnews.pro/news/your-agent-s-free-text-output-is-an-api-you-never-designed.txt", "jsonld": "https://wpnews.pro/news/your-agent-s-free-text-output-is-an-api-you-never-designed.jsonld"}}