Switch from zero: ship a human-in-the-loop data quality agent to your team chat

A developer built Shield, a human-in-the-loop data quality agent, during the Flip the Switch hackathon on Switch 0.31.0, running it in a GitHub Codespace with fictional data. The agent checks a daily table, reports findings in a team chat channel, and only applies fixes after a person approves. The writeup documents self-hosting Switch via Docker Compose, registering a custom always-on agent through POST /agents, and connecting over a Server-Sent Events stream keyed by a client-generated connection id with a 6-second heartbeat TTL.

I spent the Flip the Switch hackathon building Shield , an agent that checks a daily table, explains what it finds in a team channel and only fixes data after a person approves. This post is the guide I wish I had on day one: how to self-host Switch with no desktop app, how a custom agent joins a room, and the three things that will block you the first time. Everything here was run on Switch 0.31.0 in a GitHub Codespace. All the data is fictional. Shield at a glance Watch it work: 3-minute demo https://github.com/jenniferevangelista-de/shield-for-data/releases/download/v0.1.0/shield demo.mp4 · 77-second explainer https://github.com/jenniferevangelista-de/shield-for-data/releases/download/v0.1.0/shield explainer.mp4 Switch https://github.com/sandbox-quantum/switch turns a chat channel Slack, Teams, Discord, Telegram or Mattermost into a room where people and AI agents work together. Switch ships no agents and no models. It gives you the room, the membership, the shared documents and a wire protocol, and your agent brings the brains. That split is why it fits a "human in the loop" workflow so well: the approval happens where the team already talks. There are two ways in: You need Docker with Compose v2. The server, its database, the Gateway admin surface and a bundled Mattermost all come from one compose file in the Switch repo. git clone https://github.com/sandbox-quantum/switch.git && cd switch cp .env.example .env bash deploy/local/generate-env-secrets.sh .env local fills every password and token sed -i 's/^SWITCH VERSION=. /SWITCH VERSION=0.31.0/' .env docker compose -f deploy/local/standalone-docker-compose.yml --project-directory . \ --profile collab --profile gateway up -d Three things worth knowing: SWITCH VERSION . collab profile is Mattermost; gateway is the admin surface. setup container creates a Mattermost admin, a normal user called user , a team called switch , and links Mattermost as the default bridge. When docker wait on that container returns 0, you are ready. An agent is registered once, out of band, with the server's AGENT REGISTRATION TOKEN from .env . For a process that is always running rather than a session Console starts on demand , use the lower-level POST /agents with an always on connection model: python import requests r = requests.post f"{SWITCH}/agents", headers={"Authorization": f"Bearer {REGISTRATION TOKEN}"}, json={ "name": "shield", what people type after @ "description": "Data quality teammate", "connector type": "custom-python", "integration profile": { "connection model": "always on", "message exchange": True, "pre invocation mediation": , "post invocation mediation": , "event reporting": , "command capabilities": {"reset": "unsupported", "compact": "unsupported", "interrupt": "unsupported"}, }, "tools": , "models": , "metadata": {}, "overwrite": True, } agent id, api key = r.json "id" , r.json "api key" the key is shown once: store it The convenience route POST /agents/register-known only knows Claude Code, Codex, OpenCode and Antigravity, so a custom agent goes through POST /agents . Down is one Server-Sent Events stream, up is plain HTTP. The part that trips people up is that the connection is identified by an id you generate , not by the socket: conn = str uuid.uuid4 generate once, reuse on every reconnect stream = session.get f"{SWITCH}/agents/{agent id}/events", params={"connection id": conn, "scope": "single", "filter": "addressed", "rooms": room id}, headers={"Accept": "text/event-stream"}, stream=True connection state . Read the rooms and cursor from it rather than assuming. POST /agents/{id}/connection/beat and {connection id, cursor} . The TTL is 6 seconds. A 404 or 409 on the beat means "reopen the stream", not "back off". filter=addressed means you only receive messages that mention your agent. That is what you want for a chat agent. rooms= . A room subscribed after the stream opens can miss its buffered events. Shield's whole client is about 150 lines: shield/switch client.py https://github.com/jenniferevangelista-de/shield-for-data/blob/main/shield/switch client.py . Operations are POST /agents/{id}/ops/{name} with the arguments as the body, plus an X-Switch-Connection-Id header when the operation needs a connection. create room on the default bridge creates the Mattermost channel for you: room = op "create room", { "name": "data-quality", "description": "Data quality incidents for sales.orders", "agent names": "shield" , include yourself, or you are not a member "user names": "user" , Mattermost usernames "channel type": "channel public", "instructions": "Shield never changes data without an approval in this room.", }, connection id=conn To answer in the room, POST /agents/{id}/message with {room id, content} needs no connection at all. Markdown renders in Mattermost. 1. "You're not permitted to direct messages to me in this room." A new agent answers only its owner, and an agent registered with the registration token is owned by a bootstrap account, not by you. Nobody can talk to it. The Gateway admin credentials in .env can clear the policy: s = requests.Session s.post f"{SWITCH}/gateway/auth/login", json={"email": ADMIN EMAIL, "password": ADMIN PASSWORD} s.put f"{SWITCH}/gateway/agents/{agent id}/addressing-policy", json={"policy": None} With no policy, room membership decides who can reach the agent. In production, set a policy that names the people allowed to approve. 2. "I'm currently offline: my connector isn't reporting in." Switch answers on your agent's behalf until it sees heartbeats. If your test posts a message in the first seconds after the agent starts, the agent never sees it. My end-to-end test simply asks @shield help until the real answer comes back. 3. Nothing happens and there is no error. Messages that do not @mention the agent are delivered to nobody when you use filter=addressed . Check the Switch logs docker logs <project -switch-1 : lines from switch core.delivery.addressing tell you exactly why a message was or was not delivered. This is the part that makes it a workflow rather than a bot. Shield follows one rule: a scan only records findings; only an approval changes data. Each finding gets an id and three possible answers, all plain messages in the room: @shield approve F-1 runs the proposed actions, checks again, and posts the result. @shield edit F-1 hold dashboard+notify owners changes the plan before approving. @shield reject F-1 known test load closes it with a reason and touches nothing. Switch passes the sender's name with every message, so the audit trail records who decided. Shield writes that trail to a room document create room document , then update room document , which every member of the room can read later. The main check is the business-date gate from my article Your pipeline ran green and shipped last week's data anyway https://dev.to/jenniferevangelistade/your-pipeline-ran-green-and-shipped-last-weeks-data-anyway-1il3 : the greatest attempt per job and date wins, and a date is ready only when every required job's newest attempt succeeded for that date. The other five checks schema, row count, nulls, duplicates, late files are a Great Expectations suite, and owners come from a catalog adapter shaped like OpenMetadata's API. Shield's language model is a small open model, Gemma 3 4B, served on the same machine by Ollama https://ollama.com through its OpenAI-compatible API, so it costs nothing and no data leaves the host. It has two jobs: @shield are today's numbers ok to publish? runs a scan. @shield sure, go ahead with F-1 does That boundary is the design choice I would repeat in any agent that touches production data: let the model read and explain, and keep every write behind an exact, human command. Mattermost has a plain REST API, so an end-to-end test can log in as the demo user, post @shield break stale business date , @shield scan , @shield approve F-1 , and wait for each reply. In my repo, that runs on a fresh GitHub Actions runner on every push: start Switch, register Shield, play every scenario. If the test passes, the README is telling the truth. The code, the setup script and the end-to-end test are here: github.com/jenniferevangelista-de/shield-for-data https://github.com/jenniferevangelista-de/shield-for-data .