{"slug": "i-built-3-checks-into-my-ai-s-slack-workflow-i-ve-used-one-of-them-zero-times", "title": "I Built 3 Checks Into My AI's Slack Workflow. I've Used One of Them Zero Times.", "summary": "A developer built three safety checks into an AI's Slack workflow to ensure human-verifiable actions. An audit of the logs revealed that one check had never been used, and a trace for another check was initially misreported. The developer found that the AI's channel search tool, intended to prevent raw ID usage, failed to find the target channel in seven attempts, surfacing only an archived channel from 2019.", "body_md": "I gave an AI real permission to post to our team's Slack. Then I built three checks into how it uses that permission. When I finally audited my own logs, one of the three had never actually fired.\n\n**TL;DR: \"It worked\" and \"I can verify what it did\" are two different claims about an AI action. I built 3 checks into my workflow to close that gap. One of them turned out to have zero logged uses — and the trace I use to explain another one doesn't end the way I first described it, either.**\n\nThe AI's Slack access runs through MCP tools, and one of the first problems I hit had nothing to do with the message content — it was the channel ID. A channel ID is a machine-readable string. It's exactly the kind of identifier an AI can use correctly and a human can't sanity-check by glancing at it. So the first check I put in place was simple: before the AI uses a channel ID for anything, make it get there through `slack_search_channels`\n\n— a name search, not a raw ID lookup. That turns a step nobody can eyeball into a step a human can.\n\nHere's what that actually looked like in one real search sequence, from a session log audit on 2026-07-09. This is the tool call shape and the actual result payload for each of the 7 calls, query strings replaced with category placeholders — everything else, including field names and result formatting, is what the connector actually sent and returned:\n\n```\n// call 1\n{\"name\": \"slack_search_channels\", \"input\": {\"query\": \"[project name]\", \"channel_types\": \"public_channel,private_channel\"}}\n// result: {\"results\": \"# Search Results for: [project name]\\n\\nNo results found.\\n\"}\n\n// call 2\n{\"name\": \"slack_search_channels\", \"input\": {\"query\": \"[a number]\", \"channel_types\": \"public_channel,private_channel\"}}\n// result: {\"results\": \"# Search Results for: [a number]\\n\\nNo results found.\\n\"}\n\n// call 3\n{\"name\": \"slack_search_channels\", \"input\": {\"query\": \"[place name]\", \"channel_types\": \"public_channel,private_channel\"}}\n// result: {\"results\": \"# Search Results for: [place name]\\n\\nNo results found.\\n\"}\n\n// call 4\n{\"name\": \"slack_search_channels\", \"input\": {\"query\": \"[an abbreviation]\", \"channel_types\": \"public_channel,private_channel\", \"include_archived\": true}}\n// result: {\"results\": \"# Search Results for: [an abbreviation]\\n\\n## Channels (1 results)\\n### Result 1 of 1\\nName: #[redacted]\\nCreated: 2019-03-14\\nIs Archived: true\\nChannel Type: private_channel\\n\"}\n\n// call 5\n{\"name\": \"slack_search_channels\", \"input\": {\"query\": \"[another project-ish term]\", \"channel_types\": \"public_channel,private_channel\"}}\n// result: {\"results\": \"# Search Results for: [another project-ish term]\\n\\nNo results found.\\n\"}\n\n// call 6\n{\"name\": \"slack_search_channels\", \"input\": {\"query\": \"[a department keyword]\", \"channel_types\": \"public_channel,private_channel\"}}\n// result: {\"results\": \"# Search Results for: [a department keyword]\\n\\nNo results found.\\n\"}\n\n// call 7\n{\"name\": \"slack_search_channels\", \"input\": {\"query\": \"[a person's name]\", \"channel_types\": \"public_channel,private_channel\"}}\n// result: {\"results\": \"# Search Results for: [a person's name]\\n\\nNo results found.\\n\"}\n```\n\nSeven calls. One of them surfaced a result: an archived private channel that had been sitting untouched since 2019. It wasn't the channel anyone was looking for. The other six, including the last one, came back empty. The search never landed on the target channel in this sequence at all.\n\nThat's not the story I originally told about this trace. An earlier draft of this piece said the seventh call found a match. It didn't — I'd counted that seven calls happened without checking what each one actually returned, and I only caught the difference when someone else re-read the raw results instead of trusting my summary of them.\n\nBut sit with what the search *did* catch. An archived channel is not a channel you post a live update to. If the AI had skipped the name search and used a channel ID it had cached or guessed instead, that ID could just as easily have pointed at this same dead channel — and the message would have gone nowhere, with nothing that looked like an error. The search didn't find the target. It found and rejected a bad guess before anything got sent, and then kept looking instead of settling for it. That's the actual value of turning a machine-readable ID into a name search first: not that it always finds the right answer, but that it makes a wrong answer visible instead of silent.\n\nThat's check one. Check two is a separate MCP tool the connector exposes, `slack_send_message_draft`\n\n, sitting next to `slack_send_message`\n\nas a distinct, deliberately slower path — draft first, confirm, then send. I adopted it specifically for anything high-stakes. Check three isn't a tool call at all: after a send, don't trust \"I sent it\" as a terminal state. Reopen the actual channel and check that the count, the recipient, and the content match what was intended.\n\nThree checks. Two phases: pre-send (verify the plan before it executes) and post-send (verify the result after it executes). I'd assumed I was running all three consistently. I hadn't checked.\n\n| check | phase | what it's for | logged uses |\n|---|---|---|---|\n`slack_search_channels` before using a channel ID |\npre-send | turn a machine-readable ID into a human-checkable name, so a bad guess is visible instead of silent | 7 calls in one sequence (2026-07-09 audit); 1 result found, 0 matches to the target |\n`slack_send_message_draft` before a high-stakes send |\npre-send | gate the send behind an explicit confirm step | 0, across every project I audited |\n| re-open the channel and check count/recipient/content | post-send | never trust \"sent\" as a terminal state | not tool-logged — I can't produce a number for this one |\n\nThe zero is the number that changed my mind. The connector I use exposes `slack_send_message_draft`\n\nas a distinct tool from `slack_send_message`\n\n, and I'd adopted it specifically so high-stakes sends would go through a confirm step instead of firing directly. Then I audited my own session logs across the projects where it was available, looking for real invocations. I found none. The tool exists. It's available. It has never actually fired in anything I logged.\n\nThis isn't a hypothetical risk, either. AI agents misfiring a message to the wrong destination — a stale credential pointed at the wrong workspace, a notification channel confused with a similar one — is a real, recurring category of incident, not just a risk I'm imagining for the sake of the argument. That's exactly why \"the draft tool exists but I've never used it\" isn't a comfortable thing to find in an audit — it's a real, closable gap sitting next to a failure mode that's well known to happen.\n\nI'd been treating \"it worked\" and \"I can verify what it did\" as the same claim about an AI action. They're not, and the gap between them splits cleanly into two different problems.\n\n\"It worked\" and \"I can verify what it did\" are two different claims about the same AI action. Confirming the plan before execution closes the specification gap. Confirming the result after execution closes the verification gap. Most setups — mine included — only reliably close one of them.\n\nAn AI that runs correctly every time is not automatically an AI whose actions you can check after the fact. Those are different design axes. The pre-send half (name search before using an ID, draft before a high-stakes send) closes the specification gap — did the AI understand what I actually wanted it to do. The post-send half (reopening the channel, checking it matches) closes the verification gap — can I confirm what it actually did, independent of what it told me. Building the first without the second gets you an AI that's confidently wrong in a way nobody catches.\n\nI'd assumed I was running all of this consistently. The audit only confirmed that for the one piece I could actually count: the draft-confirm step had zero logged uses. The post-send channel check isn't a tool call, so I don't have the same kind of number for it — I don't have a clean way to prove I'm doing it every time, which is itself worth sitting with. The one gap I could measure turned out to be real. I can't yet tell you the other one isn't.\n\nThe 2-phase idea should generalize past Slack: something checkable before an action executes, and something checkable after. The specific mechanism here — a name search in front of an ID — is Slack-shaped, because Slack happens to expose both a machine ID and a human-searchable name for the same channel. Other tools won't have that exact shape. The underlying question does travel, though.\n\nWhat's the smallest post-action check you could add before trusting your AI's next \"sent successfully\" — and have you actually looked at your logs to see whether you're running it, or just assuming you are, the way I was?\n\n*Sho Naka (nomurasan). I ship, and I've learned the hard way that counting how many times a tool was called is not the same as checking what it actually returned.*\n\nThis piece draws on a Japanese essay covering the same underlying mechanisms, rewritten from scratch as an implementation log (not a direct translation) with AI assistance for structure and the cross-language rewrite. An earlier draft of this piece contained a factual error in its central trace: it described the search sequence as ending in a match, and it doesn't. That was caught by an independent adversarial review of the raw logs, not by the first read-through. The tool-call counts, the audit results, and the conclusions in this version are mine, checked against the individual results of my own session logs — not just the fact that a tool was called.", "url": "https://wpnews.pro/news/i-built-3-checks-into-my-ai-s-slack-workflow-i-ve-used-one-of-them-zero-times", "canonical_source": "https://dev.to/nomurasan/i-built-3-checks-into-my-ais-slack-workflow-ive-used-one-of-them-zero-times-5ak1", "published_at": "2026-07-15 02:16:47+00:00", "updated_at": "2026-07-15 02:27:15.742029+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "developer-tools"], "entities": ["Slack", "MCP"], "alternates": {"html": "https://wpnews.pro/news/i-built-3-checks-into-my-ai-s-slack-workflow-i-ve-used-one-of-them-zero-times", "markdown": "https://wpnews.pro/news/i-built-3-checks-into-my-ai-s-slack-workflow-i-ve-used-one-of-them-zero-times.md", "text": "https://wpnews.pro/news/i-built-3-checks-into-my-ai-s-slack-workflow-i-ve-used-one-of-them-zero-times.txt", "jsonld": "https://wpnews.pro/news/i-built-3-checks-into-my-ai-s-slack-workflow-i-ve-used-one-of-them-zero-times.jsonld"}}