# Switch from zero: ship a human-in-the-loop data quality agent to your team chat

> Source: <https://dev.to/jenniferevangelistade/switch-from-zero-ship-a-human-in-the-loop-data-quality-agent-to-your-team-chat-4cm9>
> Published: 2026-11-03 22:40:24+00:00

I spent the Flip the Switch hackathon building **Shield**, an agent that checks a daily table, explains what it finds in a team channel and only fixes data after a person approves. This post is the guide I wish I had on day one: how to self-host Switch with no desktop app, how a custom agent joins a room, and the three things that will block you the first time.

Everything here was run on Switch 0.31.0 in a GitHub Codespace. All the data is fictional.

**Shield at a glance**

**Watch it work:** [3-minute demo](https://github.com/jenniferevangelista-de/shield-for-data/releases/download/v0.1.0/shield_demo.mp4) · [77-second explainer](https://github.com/jenniferevangelista-de/shield-for-data/releases/download/v0.1.0/shield_explainer.mp4)

[Switch](https://github.com/sandbox-quantum/switch) turns a chat channel (Slack, Teams, Discord, Telegram or Mattermost) into a **room** where people and AI agents work together. Switch ships no agents and no models. It gives you the room, the membership, the shared documents and a wire protocol, and your agent brings the brains. That split is why it fits a "human in the loop" workflow so well: the approval happens where the team already talks.

There are two ways in:

You need Docker with Compose v2. The server, its database, the Gateway (admin surface) and a bundled Mattermost all come from one compose file in the Switch repo.

```
git clone https://github.com/sandbox-quantum/switch.git && cd switch
cp .env.example .env
bash deploy/local/generate-env-secrets.sh .env local   # fills every password and token
sed -i 's/^SWITCH_VERSION=.*/SWITCH_VERSION=0.31.0/' .env
docker compose -f deploy/local/standalone-docker-compose.yml --project-directory . \
  --profile collab --profile gateway up -d
```

Three things worth knowing:

`SWITCH_VERSION`.` collab` profile is Mattermost; `gateway` is the admin surface.`setup` container creates a Mattermost admin, a normal user called `user`, a team called `switch`, and links Mattermost as the default bridge. When `docker wait` on that container returns 0, you are ready.
An agent is registered once, out of band, with the server's `AGENT_REGISTRATION_TOKEN` from `.env`. For a process that is always running (rather than a session Console starts on demand), use the lower-level `POST /agents` with an `always_on` connection model:

``` python
import requests

r = requests.post(f"{SWITCH}/agents", headers={"Authorization": f"Bearer {REGISTRATION_TOKEN}"}, json={
    "name": "shield",                      # what people type after @
    "description": "Data quality teammate",
    "connector_type": "custom-python",
    "integration_profile": {
        "connection_model": "always_on",
        "message_exchange": True,
        "pre_invocation_mediation": [], "post_invocation_mediation": [], "event_reporting": [],
        "command_capabilities": {"reset": "unsupported", "compact": "unsupported", "interrupt": "unsupported"},
    },
    "tools": [], "models": [], "metadata": {}, "overwrite": True,
})
agent_id, api_key = r.json()["id"], r.json()["api_key"]   # the key is shown once: store it
```

The convenience route `POST /agents/register-known` only knows Claude Code, Codex, OpenCode and Antigravity, so a custom agent goes through `POST /agents`.

Down is one Server-Sent Events stream, up is plain HTTP. The part that trips people up is that **the connection is identified by an id you generate**, not by the socket:

```
conn = str(uuid.uuid4())   # generate once, reuse on every reconnect
stream = session.get(f"{SWITCH}/agents/{agent_id}/events",
                     params={"connection_id": conn, "scope": "single", "filter": "addressed",
                             "rooms": room_id},
                     headers={"Accept": "text/event-stream"}, stream=True)
```

`connection_state`. Read the `rooms` and `cursor` from it rather than assuming.`POST /agents/{id}/connection/beat` and `{connection_id, cursor}`. The TTL is 6 seconds. A 404 or 409 on the beat means "reopen the stream", not "back off".`filter=addressed` means you only receive messages that mention your agent. That is what you want for a chat agent.`rooms=`). A room subscribed after the stream opens can miss its buffered events.
Shield's whole client is about 150 lines: [`shield/switch_client.py`](https://github.com/jenniferevangelista-de/shield-for-data/blob/main/shield/switch_client.py).

Operations are `POST /agents/{id}/ops/{name}` with the arguments as the body, plus an `X-Switch-Connection-Id` header when the operation needs a connection. `create_room` on the default bridge creates the Mattermost channel for you:

```
room = op("create_room", {
    "name": "data-quality",
    "description": "Data quality incidents for sales.orders",
    "agent_names": ["shield"],          # include yourself, or you are not a member
    "user_names": ["user"],             # Mattermost usernames
    "channel_type": "channel_public",
    "instructions": "Shield never changes data without an approval in this room.",
}, connection_id=conn)
```

To answer in the room, `POST /agents/{id}/message` with `{room_id, content}` needs no connection at all. Markdown renders in Mattermost.

**1. "You're not permitted to direct messages to me in this room."** A new agent answers only its owner, and an agent registered with the registration token is owned by a bootstrap account, not by you. Nobody can talk to it. The Gateway admin (credentials in `.env`) can clear the policy:

```
s = requests.Session()
s.post(f"{SWITCH}/gateway/auth/login", json={"email": ADMIN_EMAIL, "password": ADMIN_PASSWORD})
s.put(f"{SWITCH}/gateway/agents/{agent_id}/addressing-policy", json={"policy": None})
```

With no policy, room membership decides who can reach the agent. In production, set a policy that names the people allowed to approve.

**2. "I'm currently offline: my connector isn't reporting in."** Switch answers on your agent's behalf until it sees heartbeats. If your test posts a message in the first seconds after the agent starts, the agent never sees it. My end-to-end test simply asks `@shield help` until the real answer comes back.

**3. Nothing happens and there is no error.** Messages that do not `@mention` the agent are delivered to nobody when you use `filter=addressed`. Check the Switch logs (` docker logs <project>-switch-1`): lines from `switch_core.delivery.addressing` tell you exactly why a message was or was not delivered.

This is the part that makes it a workflow rather than a bot. Shield follows one rule: **a scan only records findings; only an approval changes data.**

Each finding gets an id and three possible answers, all plain messages in the room:

`@shield approve F-1` runs the proposed actions, checks again, and posts the result.`@shield edit F-1 hold_dashboard+notify_owners` changes the plan before approving.`@shield reject F-1 known test load` closes it with a reason and touches nothing.
Switch passes the sender's name with every message, so the audit trail records who decided. Shield writes that trail to a **room document** (`create_room_document`, then `update_room_document`), which every member of the room can read later.

The main check is the business-date gate from my article [Your pipeline ran green and shipped last week's data anyway](https://dev.to/jenniferevangelistade/your-pipeline-ran-green-and-shipped-last-weeks-data-anyway-1il3): the greatest attempt per job and date wins, and a date is ready only when every required job's newest attempt succeeded for that date. The other five checks (schema, row count, nulls, duplicates, late files) are a Great Expectations suite, and owners come from a catalog adapter shaped like OpenMetadata's API.

Shield's language model is a small open model, Gemma 3 4B, served on the same machine by [Ollama](https://ollama.com) through its OpenAI-compatible API, so it costs nothing and no data leaves the host. It has two jobs:

`@shield are today's numbers ok to publish?` runs a scan. `@shield sure, go ahead with F-1` does That boundary is the design choice I would repeat in any agent that touches production data: let the model read and explain, and keep every write behind an exact, human command.

Mattermost has a plain REST API, so an end-to-end test can log in as the demo user, post `@shield break stale_business_date`, `@shield scan`, `@shield approve F-1`, and wait for each reply. In my repo, that runs on a fresh GitHub Actions runner on every push: start Switch, register Shield, play every scenario. If the test passes, the README is telling the truth.

The code, the setup script and the end-to-end test are here: [github.com/jenniferevangelista-de/shield-for-data](https://github.com/jenniferevangelista-de/shield-for-data).