How We Built an Autonomous Incident War Room with Switch AI

Developer Krish Punia built OpsSwitch, an open-source multi-agent incident-response system on the Switch AI platform that runs a four-agent squad inside team chat to triage, investigate, patch, and document production incidents. The agents—sentinel, sleuth, releaser, and scribe—are defined in a YAML template and communicate over SSE and HTTP, with a human-in-the-loop gate requiring explicit approval before any production deploy. In a demo against a cart-service KeyError bug, the relay moved from alert to root-cause commit identification and a proposed hotfix, with a post-mortem drafted after human sign-off.

How We Built an Autonomous Incident War Room with Switch AI Published for the WeMakeDevs × DevTools Academy "Flip the Switch" Hackathon By Krish Punia Imagine getting paged at 3:15 AM on a Saturday. Production is throwing HTTP 500s on the checkout endpoint. You log onto Slack, ping three teammates, search Datadog for error traces, sift through dozens of recent GitHub commits, and debate whether to rollback or hotfix. By the time the patch is live and the post-mortem is written, over 45 minutes have elapsed. What if your team chat wasn't just a place where humans panic, but an active War Room where autonomous AI agents collaborate alongside you to diagnose, reproduce, patch, and document the incident in under 90 seconds? In this article, I'll walk you through how I built OpsSwitch using Switch AI—an open-source platform that brings AI coding agents directly into team chat. Most AI coding tools Claude Code, Cursor, Codex, OpenCode live isolated inside a developer's terminal or IDE. But real engineering happens in teams, and team coordination happens in chat Slack, Mattermost, Microsoft Teams, Discord . Switch solves this by providing: Chat as a Room: Every channel becomes an environment where multiple agents and humans coexist. First-Class Agent Identity: Agents get their own avatars, names, and addressing rules addressing: anyone or addressing: owner-agents . Agent Protocol: Agents communicate via Server-Sent Events SSE and HTTP, allowing any custom agent or LLM framework to participate. Human-in-the-Loop Governance: Built-in gates ensure that dangerous actions like production deploys require explicit human sign-off. In OpsSwitch, we created a specialized four-agent squad: PagerDuty Alert │ ▼ @ops-sentinel https://dev.toCommander ──► Assesses SEV, summarizes impact │ ▼ @ops-sleuth https://dev.toInvestigator ──► Git blame, commit diff, root cause │ ▼ @ops-releaser https://dev.toRemediation ──► Generates patch, verifies tests │ ▼ Human Engineer https://dev.toGate ──► Replies "Approve" │ ▼ @ops-scribe https://dev.toChronicler ──► Markdown Post-Mortem delivered The Switch Template template.yaml Switch allows you to define your entire multi-agent topology in a clean YAML template: version: 1 params: team: type: string default: ops repo path: type: string default: "{$agents dir}/demo-service" agents: name: "{team}-sentinel" description: Incident Commander addressing: anyone instructions: | Triage incoming error reports. Assign SEV-1, SEV-2, or SEV-3. Hand off code investigation to @{team}-sleuth with the exact stack trace. name: "{team}-sleuth" description: Git & Code Investigator addressing: owner-agents instructions: | Inspect the repository. Run git log -n 5 and git blame. Pinpoint the exact commit and line causing the failure. Hand off to @{team}-releaser. name: "{team}-releaser" description: Remediation & Deployment addressing: owner-agents instructions: | Prepare the hotfix and run local tests. SAFETY: Never deploy without explicit "Approve" from {$creator}. name: "{team}-scribe" description: Incident Chronicler addressing: owner-agents instructions: | Compile the comprehensive post-mortem report once remediation is confirmed. room: name: "{team}-incident-war-room" agents: "{team}-sentinel", "{team}-sleuth", "{team}-releaser", "{team}-scribe" users: "{$creator}" The Incident In our test microservice cart-service , a junior engineer pushed commit 8bae34d: discount = COUPONS cart.coupon code Raises KeyError when coupon is invalid The Autonomous Relay in Action Detection: An error alert hits incident-war-room: CRITICAL: 500 Spike on /api/v2/checkout KeyError: 'INVALID PROMO' Triage: @ops-sentinel declares SEV-2, notes that unauthenticated shoppers with promo codes are blocked, and addresses @ops-sleuth. Investigation: @ops-sleuth runs git blame and catches commit 8bae34d within 15 seconds. It notes that COUPONS.get cart.coupon code, 0.0 was needed. Remediation & Human Gate: @ops-releaser prepares the patch, executes python3 test service.py to confirm 2/2 tests pass, and tags me: " @krish https://dev.to/krish Punia: Hotfix verified locally. Reply 'Approve' to deploy." Approval & Deploy: I review the one-line diff and type "Approve". Documentation: @ops-scribe posts a complete Markdown post-mortem with 5 Whys, timeline, and action items. Total resolution time: 91 seconds. Multi-Agent Specialization Beats Monolithic Prompts: Giving each agent a distinct role Sentinel vs. Sleuth vs. Releaser vs. Scribe prevented context pollution and hallucinations. Human Gates Are Mandatory for Enterprise Trust: Autonomous agents must be safe by design. Switch's addressing model makes human sign-offs natural and reliable. Chat is the Natural Home for AI Teammates: Instead of switching between 5 browser tabs and terminals during an emergency, having your agents live in your team chat keeps everyone aligned. You can explore the complete repository, Switch template, and reproduction harness here: 👉 GitHub: kpunia306/ops-switch Special thanks to WeMakeDevs, DevTools Academy, and the Switch AI team for hosting this fantastic hackathon