{"slug": "ship-gate-the-pre-deploy-checklist-most-ai-features-skip", "title": "Ship Gate: The Pre-Deploy Checklist Most AI Features Skip", "summary": "AI architect Wasim Sheikh published a pre-deployment checklist called \"Ship Gate\" that he argues most AI features skip before going live. The five-gate framework covers threat modeling, data boundaries, eval sets of at least 20 cases, human override for irreversible actions, and observability that lets an engineer reconstruct a bad turn in under five minutes. Sheikh cites a case where an AI email summarizer shipped into customer-facing replies and invented a discount a customer had never been offered, with no eval set, human gate, or kill switch in place.", "body_md": "Most AI features don't fail in the model.\n\nThey fail at the gate.\n\nSomeone demos a shiny prompt. Leadership loves it. It ships. Then a customer pastes something weird, an agent calls the wrong tool, or a \"helpful\" answer invents an offer that never existed. The postmortem is always the same: we optimized for the demo, not for production.\n\nI call the missing step **Ship Gate** — a short, boring checklist you run before any AI feature leaves the sandbox. Not a 40-page risk memo. A gate you can clear in an afternoon.\n\n**Situation.** A product team shipped an AI email summarizer into customer-facing replies. Staging looked clean. Leadership wanted it live before the quarter closed.\n\n**What broke.** Day one, the model invented a discount a customer had never been offered. Support scrambled. There was no eval set of \"known trap\" emails, no human gate on outbound AI text, and no kill switch short of a full deploy rollback.\n\n**The fix (Ship Gate).**\n\nThen they added the ops layer every Ship Gate implies: canary traffic, a one-click kill switch, and a documented rollback owner before the flag flipped on.\n\nShip Gate isn't bureaucracy. It's the minimum checklist so an AI feature earns production — instead of learning in front of customers.\n\nDemos are curated. You pick the happy path. You soft-prompt around the edge cases. You never paste a raw customer email with secrets still in it.\n\nProduction is the opposite. Users paste junk. Models invent confident nonsense. Tools fire with the wrong args. Logs capture more than you think.\n\nIf your only test was \"it looked good in the meeting,\" you didn't ship AI — you shipped a vibe.\n\nShip Gate exists to make that gap visible before customers feel it.\n\nRun these in order. Fail any one and you don't ship. Pass all five and you can sleep.\n\nBefore code, write three sentences:\n\nIf you can't name the top three failure modes, you're not ready to build — you're ready to be surprised.\n\nAnswer out loud:\n\nThen enforce it in code, not in a slide. Strip secrets before the model. Scope tools to the minimum. Prefer retrieval over stuffing the whole corpus into context. If the model doesn't need a field, don't give it the field.\n\nPick 20–50 real-ish cases: happy path, hostile prompts, empty input, long paste, \"ignore previous instructions,\" the ticket that always breaks things.\n\nScore each on three axes:\n\nShip Gate rule: you don't tune the UI until the eval set is green enough that you'd trust a teammate to use the feature unsupervised.\n\nEvery autonomous or semi-autonomous path needs an escape hatch:\n\nIf the only recovery path is \"hope the model was right,\" you don't have a product. You have a liability with a chat box.\n\nWhen it breaks at 2 a.m., can you answer:\n\nIf the answer is \"we'd have to dig through JSON for an hour,\" fix the logs before you grow the feature. Ship Gate isn't complete until a tired engineer can reconstruct a single bad turn.\n\n| Gate | Pass criteria | \n|---|---|\n| Threat model | Top 3 failure modes named + owner | \n| Data boundary | See / write / never-leave documented + enforced | \n| Eval set | ≥20 cases; safe + useful bar met | \n| Human override | Irreversible actions confirmed; stop path clear | \n| Observability | One bad turn reconstructable in <5 minutes | \n\nNo green across the board → no ship.\n\nAsk one question before any AI feature goes live:\n\nWhich of the five gates is still red — and who owns fixing it?\n\nIf nobody owns the red gate, you don't have a launch plan. You have a calendar date.\n\nI'm Wasim Sheikh — AI Architect. I build systems teams trust and organizations depend on: not demos, not proofs of concept — production.\n\nCanonical: [https://sheikhwasim.com/insights/ship-gate-pre-deploy-checklist/](https://sheikhwasim.com/insights/ship-gate-pre-deploy-checklist/)", "url": "https://wpnews.pro/news/ship-gate-the-pre-deploy-checklist-most-ai-features-skip", "canonical_source": "https://dev.to/anciwasim/ship-gate-the-pre-deploy-checklist-most-ai-features-skip-1do6", "published_at": "2026-09-17 15:35:12+00:00", "updated_at": "2026-09-17 15:53:01.929468+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "mlops", "ai-products", "developer-tools"], "entities": ["Wasim Sheikh", "Ship Gate"], "alternates": {"html": "https://wpnews.pro/news/ship-gate-the-pre-deploy-checklist-most-ai-features-skip", "markdown": "https://wpnews.pro/news/ship-gate-the-pre-deploy-checklist-most-ai-features-skip.md", "text": "https://wpnews.pro/news/ship-gate-the-pre-deploy-checklist-most-ai-features-skip.txt", "jsonld": "https://wpnews.pro/news/ship-gate-the-pre-deploy-checklist-most-ai-features-skip.jsonld"}}