{"slug": "your-ai-is-in-production-that-doesn-t-mean-it-s-production-ready", "title": "Your AI Is \"In Production.\" That Doesn't Mean It's Production-Ready.", "summary": "StackRail has published the AI Production Readiness Framework (APRF), a vendor-neutral, gated methodology for determining whether an AI application is safe to operate in production. Unlike maturity scores, APRF uses mandatory pass/fail checks that block releases if any fail, and capability is measured by the weakest pillar rather than an average. The framework covers eight domains including security, safety, data, model lifecycle, agents, reliability, cost, and governance.", "body_md": "Stop shipping LLM features like landing pages. APRF is a gated, machine-readable production readiness framework—with code, YAML gates, and CI you can wire up this week.\n\nMost teams ship an LLM feature the same way they ship a landing page: merge the PR, watch the demo, celebrate.\n\nThen reality shows up.\n\nNone of those failures look like \"the model wasn't smart enough.\"\n\nThey look like **production systems without production gates.**\n\nThis post is the developer cut of that argument—plus the parts you can implement: allowlists, approval gates, YAML policy, CI, and a machine-readable attestation. Canonical version lives on StackRail: [Your AI Is \"In Production.\" That Doesn't Mean It's Production-Ready.](https://stackrail.io/articles/ai-in-production-not-production-ready/)\n\nNIST AI RMF tells you how to *think* about risk.\n\nISO/IEC 42001 tells you how to *manage* an AI system.\n\nSOC 2 tells auditors how to *trust your company*.\n\nUseful. Necessary. Incomplete for the engineer on call.\n\nThe question that actually decides whether you sleep at night is simpler:\n\nCan this AI application safely operate in production?\n\nThat's the question behind the **AI Production Readiness Framework (APRF)** — a vendor-neutral working draft published by [StackRail](https://stackrail.io/aprf/).\n\nIt's not a certification.\n\nIt's not a partner network.\n\nIt's not a 0–100 \"readiness score\" you put in a board deck.\n\nIt's a **gated** methodology: mandatory checks either pass or they block you. Recommended controls never average into the gate.\n\nIf you've ever been sold an \"AI maturity score,\" you already know the failure mode:\n\nAPRF forbids that trade.\n\n```\nvanity_score = mean(all_controls)          # ❌ averages away a missing kill switch\ngate_result  = ALL(mandatory_checks.pass)  # ✅ one fail = blocked\ncapability   = min(pillar_levels)          # ✅ weakest pillar wins\n```\n\nMandatory checks are **pass/fail**.\n\nFailures are **blockers**.\n\nCapability attainment is the **minimum** across pillars — not a mean.\n\nYou never publish a single overall percentage.\n\nIf that sounds strict: good. Production is strict.\n\n```\n+---------------------------+        +----------------------------------+\n|         Demo Path         |        |            APRF Path             |\n+---------------------------+        +----------------------------------+\n\n+---------------+                    +----------------------+\n| Prompt works  |                    | Pin APRF version     |\n+-------+-------+                    +----------+-----------+\n        |                                       |\n        v                                       v\n+---------------+                    +----------------------+\n|   Merge PR    |                    | Run mandatory checks |\n+-------+-------+                    +----------+-----------+\n        |                                       |\n        v                                       v\n+-------------------------+          +----------------------+\n| Ship in production      |          | All gates pass?      |\n+-------------------------+          +-----+-----------+----+\n                                          |           |\n                                      No  |           | Yes\n                                          |           |\n                                          v           v\n                                +----------------+  +----------------+\n                                | Block release  |  | Attest & ship  |\n                                +----------------+  +--------+-------+\n                                                             |\n                                                             v\n                                                  +----------------+\n                                                  | Observe & drill|\n                                                  +----------------+\n```\n\n| Piece | What you get |\n|---|---|\n| 8 domains | Security, safety, data, model lifecycle, agents, reliability, cost, governance |\n| 27 pillars | Focused control areas under those domains |\n| Core Profile | 40 gates for Tier‑2 customer-facing AI |\n| Regulated Profile | 61 gates for Tier‑3 / regulated systems |\n| Lenses | Extra mandatories for RAG, Agents, Voice, Coding agents |\n| Spec + attestation | Machine-readable JSON + downloadable self-attestation |\n| Crosswalks | NIST AI RMF, ISO 42001, OWASP LLM Top 10, SOC 2, AWS WA, SLSA — informative only\n|\n\nMachine-readable source of truth: [https://stackrail.io/aprf/spec/](https://stackrail.io/aprf/spec/)\n\nAPRF doesn't congratulate you. It asks (Core + Agents lens territory):\n\n| Gate | Requirement (paraphrased) | Artifact you should have |\n|---|---|---|\n`TOL-M1` |\nTool calls authorized server-side, not by model output alone | Gateway authz tests + deny logs |\n`TOL-M2` |\nPer-agent tool allowlist; unknown tools denied | Allowlist config + negative tests |\n`TOL-M3` |\nHigh-impact tools behind approval / dual control / policy | Impact inventory + bypass tests |\n`HUM-M1` |\nHigh-impact actions inventoried and gated | Gate wiring evidence |\n`AGN-*` / cost gates |\nStep budgets, kill switch, spend ceilings | Configs, drills, billing alerts |\n\nIf you can't demonstrate those with **artifacts**, you don't get a soft yellow score. You get **gate fail**.\n\n```\nUser\n │\n │ Natural language goal\n ▼\nAgent Runtime\n │\n │ proposed_tool + args\n ▼\nTool Gateway\n │\n ├─ Validate allowlist\n ├─ Validate JSON Schema\n │\n ├── Invalid?\n │      └──► DENY (logged)\n │\n └── Valid\n        │\n        ├── High-impact?\n        │      │\n        │      ├── Yes → Request approval\n        │      │            │\n        │      │            ├── Denied → Stop\n        │      │            └── Approved → Execute tool\n        │      │\n        │      └── No → Execute with scoped credentials\n        │\n        ▼\n Tool (CRM / Shell / Deploy)\n        │\n        ▼\n Sanitized result\n        │\n        ▼\nAgent Runtime\n```\n\nYou don't need to \"adopt APRF\" as a religion on day one. Wire the same ideas into your stack.\n\n``` js\nimport { z } from \"zod\";\n\nconst tools = {\n  search_docs: {\n    impact: \"read\",\n    schema: z.object({ query: z.string().min(1).max(500) }),\n    run: async ({ query }: { query: string }) => searchDocs(query),\n  },\n  update_crm_contact: {\n    impact: \"write\",\n    schema: z.object({\n      contactId: z.string().uuid(),\n      fields: z.record(z.string().max(200)).refine(\n        (f) => Object.keys(f).length <= 10,\n        \"too many fields\",\n      ),\n    }),\n    run: async (args: { contactId: string; fields: Record<string, string> }) =>\n      updateCrm(args),\n  },\n} as const;\n\ntype ToolName = keyof typeof tools;\n\nexport async function invokeTool(\n  name: string,\n  rawArgs: unknown,\n  ctx: { agentId: string; approvalToken?: string },\n) {\n  const allowlist = await loadAllowlist(ctx.agentId); // e.g. [\"search_docs\"]\n  if (!allowlist.includes(name as ToolName) || !(name in tools)) {\n    await audit({ event: \"tool_deny\", reason: \"not_allowlisted\", name, ctx });\n    throw new Error(\"TOOL_DENIED\");\n  }\n\n  const tool = tools[name as ToolName];\n  const args = tool.schema.parse(rawArgs); // throws → no side effects\n\n  if (tool.impact !== \"read\") {\n    await requireApproval({ tool: name, args, token: ctx.approvalToken });\n  }\n\n  return tool.run(args as never);\n}\n```\n\nThe failure mode to kill: UI has \"Approve\", but the agent HTTP path calls the tool directly.\n\n```\nHIGH_IMPACT = {\"update_crm_contact\", \"refund_order\", \"shell_exec\"}\n\ndef execute_tool(agent_id: str, name: str, args: dict, approval_id: str | None):\n    if name not in allowlist_for(agent_id):\n        raise PermissionError(\"not_allowlisted\")\n\n    if name in HIGH_IMPACT:\n        decision = approvals.get(approval_id)\n        if not decision or decision.status != \"approved\":\n            audit(\"ungated_attempt\", agent_id=agent_id, tool=name)\n            raise PermissionError(\"approval_required\")\n        if decision.tool != name or decision.args_hash != hash_args(args):\n            raise PermissionError(\"approval_mismatch\")\n\n    return TOOLS[name](args)\n```\n\nBypass test you should actually run in CI:\n\n```\n# Expect 403 / TOOL_DENIED — never a CRM write\ncurl -sS -X POST \"$GATEWAY/tools/update_crm_contact\" \\\n  -H \"Authorization: Bearer $AGENT_TOKEN\" \\\n  -d '{\"contactId\":\"...\",\"fields\":{\"email\":\"attacker@example.com\"}}' \\\n  | grep -E 'approval_required|TOOL_DENIED|403'\n```\n\nPin the framework version and declare which gates you claim for this service:\n\n```\n# aprf/policy.yaml\naprfVersion: \"0.10.0\"\nprofileId: aprf-profile-core\ncriticality: 2\nlenses: [agents]          # adds agent-specific mandatories\n\nsystem:\n  name: support-assistant\n  description: Customer chat with RAG + CRM tools\n\ngates:\n  # Map check IDs → how CI proves them\n  TOL-M1:\n    evidence: tests/gateway/authz_deny.test.ts\n  TOL-M2:\n    evidence: config/agents/*/tools.allowlist.json\n  TOL-M3:\n    evidence: tests/gateway/high_impact_requires_approval.test.ts\n  HUM-M1:\n    evidence: docs/high-impact-actions.md\n  COST-M1:                 # example: spend ceiling / DoW controls\n    evidence: infra/budgets/openai.tf\n\n# Recommended checks can live here but MUST NOT influence gate pass/fail\nrecommended:\n  OBS-R2:\n    evidence: dashboards/agent-traces.json\n# .github/workflows/aprf-gates.yml\nname: APRF gates\non:\n  pull_request:\n  push:\n    branches: [main]\n\njobs:\n  gates:\n    runs-on: ubuntu-latest\n    steps:\n      - uses: actions/checkout@v4\n\n      - name: Pin & fetch APRF spec\n        run: |\n          curl -fsSL https://stackrail.io/aprf/spec/ -o aprf-spec.json\n          jq -e '.version == \"0.10.0\"' aprf-spec.json\n\n      - name: Unit / contract tests for tool gateway\n        run: npm test -- tests/gateway\n\n      - name: Policy evidence exists for every mandatory gate\n        run: |\n          python scripts/check_aprf_evidence.py \\\n            --policy aprf/policy.yaml \\\n            --spec aprf-spec.json\n\n      - name: Negative: unknown tool is denied\n        run: npm test -- tests/gateway/unknown_tool_denied.test.ts\n```\n\nEvidence checker sketch:\n\n``` python\n# scripts/check_aprf_evidence.py\nimport json, sys, pathlib, yaml\n\npolicy = yaml.safe_load(open(\"aprf/policy.yaml\"))\nspec = json.load(open(\"aprf-spec.json\"))\n\n# Resolve Core (+ lenses) mandatory IDs from the pinned spec in real code.\n# Here we only verify declared gate evidence paths exist.\nmissing = []\nfor check_id, meta in policy[\"gates\"].items():\n    path = pathlib.Path(meta[\"evidence\"])\n    if not path.exists():\n        missing.append(f\"{check_id} → {path}\")\n\nif missing:\n    print(\"APRF gate evidence missing:\")\n    print(\"\\n\".join(missing))\n    sys.exit(1)\n\nprint(f\"OK: {len(policy['gates'])} gate evidence paths present (aprf {policy['aprfVersion']})\")\n```\n\nSelf-attestation is **not** certification. It *is* a reproducible artifact for PRs, change tickets, and audits.\n\nMinimal shape (see [attestation schema 0.6](https://stackrail.io/aprf/attestation-schema/0.6/) and [samples](https://stackrail.io/aprf/samples/)):\n\n```\n{\n  \"$schema\": \"https://stackrail.io/aprf/attestation-schema/0.6\",\n  \"type\": \"aprf-self-attestation\",\n  \"aprfVersion\": \"0.10.0\",\n  \"certificationLevel\": \"self-attestation\",\n  \"assessedAt\": \"2026-07-25T12:00:00.000Z\",\n  \"subject\": {\n    \"organization\": \"Your Co\",\n    \"systemName\": \"support-assistant\"\n  },\n  \"assessor\": { \"name\": \"platform-oncall\", \"role\": \"Platform engineer\" },\n  \"input\": {\n    \"criticality\": 2,\n    \"profileId\": \"aprf-profile-core\",\n    \"lensIds\": [\"agents\"],\n    \"outcomes\": [\n      { \"checkId\": \"TOL-M1\", \"passed\": true, \"evidenceRef\": \"tests/gateway/authz_deny.test.ts\" },\n      { \"checkId\": \"TOL-M2\", \"passed\": true, \"evidenceRef\": \"config/agents/support/tools.allowlist.json\" },\n      { \"checkId\": \"TOL-M3\", \"passed\": false, \"evidenceRef\": \"MISSING: approval bypass tests\" }\n    ]\n  },\n  \"result\": {\n    \"gate\": \"fail\",\n    \"blockers\": [\"TOL-M3\"]\n  },\n  \"statement\": \"Self-attestation against APRF Core + agents lens; not third-party certification.\",\n  \"disclaimer\": \"Crosswalks to NIST/ISO/SOC2 are informative alignment only.\"\n}\n```\n\nOne failed mandatory → **gate fail**. No averaging. No \"87% ready.\"\n\nWe published a Core / Regulated self-assessment with optional lenses. Download the attestation JSON when you're done.\n\nAnyone looking for a badge that says \"we're compliant with everything.\"\n\nAPRF won't pretend. That's the point.\n\n*APRF is a working draft.\n\nPublisher today: StackRail.\n\nIntended long-term steward: a neutral working group via public RFCs.\n\nContribute: [stackrail.io/aprf/rfc](https://stackrail.io/aprf/rfc/).*\n\n*Originally published on StackRail (set as canonical above).*", "url": "https://wpnews.pro/news/your-ai-is-in-production-that-doesn-t-mean-it-s-production-ready", "canonical_source": "https://dev.to/prasoonanand/your-ai-is-in-production-that-doesnt-mean-its-production-ready-3ka", "published_at": "2026-07-25 10:39:25+00:00", "updated_at": "2026-07-25 11:02:30.098064+00:00", "lang": "en", "topics": ["ai-safety", "ai-infrastructure", "ai-policy", "mlops", "developer-tools"], "entities": ["StackRail", "NIST", "ISO", "SOC 2"], "alternates": {"html": "https://wpnews.pro/news/your-ai-is-in-production-that-doesn-t-mean-it-s-production-ready", "markdown": "https://wpnews.pro/news/your-ai-is-in-production-that-doesn-t-mean-it-s-production-ready.md", "text": "https://wpnews.pro/news/your-ai-is-in-production-that-doesn-t-mean-it-s-production-ready.txt", "jsonld": "https://wpnews.pro/news/your-ai-is-in-production-that-doesn-t-mean-it-s-production-ready.jsonld"}}