{"slug": "route-ci-cd-alerts-to-an-agent-that-triages-by-email", "title": "Route CI/CD alerts to an agent that triages by email", "summary": "A developer at Nylas built a CI/CD alert triage agent that uses its own email inbox to receive, cluster, and summarize noisy pipeline notifications. The agent, created via Nylas's custom grant API, replies in the same email thread with a probable root cause, reducing on-call engineers' inbox clutter from dozens of messages to one. The system is designed for triage and summarization, not as a replacement for full observability stacks.", "body_md": "CI alert email is noisy, and most of it is ignorable. If you've ever owned a pipeline at any scale, you know the shape of it: a flaky integration test trips, GitHub Actions emails you, a retry passes, and now there are three messages in your inbox about a problem that fixed itself. Multiply that by every branch, every nightly, every deploy, and the signal you actually care about — the one real failure — is buried under forty notifications that all look identical until you open them.\n\nThe usual fix is to wire alerts into Slack and add a bot that reacts to emoji. That works right up until the channel becomes the new noisy inbox. The other usual fix is to point an LLM at a human's mailbox and let it \"summarize your morning.\" That's a demo, not a system — the moment you want the summarizer to *reply*, to *be a participant* in the alert thread that on-call is reading, you need it to own an address, not borrow yours.\n\nThat's the angle here. Give the triage agent its own inbox. Your CI system already knows how to email failures somewhere — point it at an address the agent controls, let the agent cluster and summarize the incoming alerts, and have it reply *in the same thread* with a probable root cause so whoever's on-call reads one message instead of forty.\n\nI work on the Nylas CLI, so the terminal commands below are the exact ones I reach for when I'm building this. I'll show the raw API call alongside each one, because in production this lives in a service, not a shell.\n\nAn **Agent Account** is a Nylas grant with its own email address. That's the whole trick. It's not a new product surface you have to learn — it's a `grant_id`\n\nthat happens to belong to a programmatic mailbox instead of a human's connected Google or Microsoft account. Every grant-scoped endpoint you already know works against it unchanged: `GET /v3/grants/{grant_id}/messages`\n\n, `POST .../messages/send`\n\n, threads, folders, all of it.\n\nSo the data plane is nothing new. You point your CI notifier at `ci-alerts@yourcompany.com`\n\n, the agent receives mail there like any inbox, and you read and reply with the same endpoints you'd use for a connected account.\n\nOne honest caveat up front: this is alert **triage and summarization over email**. It is not a replacement for your observability stack. The agent doesn't have your traces, your metrics, or your deploy history unless you give them to it. What it does well is take the firehose of alert *emails* — which is genuinely noisy and mostly redundant — collapse it into clusters, and hand on-call a probable cause to start from. Treat its root-cause guess as a hypothesis, not a verdict.\n\nCreate the account. The CLI wraps `POST /v3/connect/custom`\n\nwith `provider: nylas`\n\n:\n\n```\nnylas agent account create ci-alerts@yourcompany.com --name \"CI Triage Bot\"\n```\n\nThe raw call, if you're provisioning from a service:\n\n```\ncurl -X POST \"https://api.us.nylas.com/v3/connect/custom\" \\\n  -H \"Authorization: Bearer $NYLAS_API_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"provider\": \"nylas\",\n    \"name\": \"CI Triage Bot\",\n    \"settings\": { \"email\": \"ci-alerts@yourcompany.com\" }\n  }'\n```\n\nThe email has to be on a registered domain — either your own custom domain or a Nylas `*.nylas.email`\n\ntrial subdomain. There's no OAuth, no refresh token, no consent screen, because nobody's connecting an external account; you're minting one. The API auto-creates a default workspace and policy for the account, which matters later when we route noisy senders.\n\nNow go into your CI system's notification settings — GitHub Actions, Jenkins, CircleCI, whatever you run — and set the alert recipient to `ci-alerts@yourcompany.com`\n\n. From the agent's side, an alert is just inbound mail.\n\nInbound mail fires the standard `message.created`\n\nwebhook. The thing to internalize: **webhooks are application-scoped, not grant-scoped.** You subscribe once at the app level, and events for every grant in your app land at the same endpoint, each payload carrying a `grant_id`\n\nyou filter on. You don't register a webhook per Agent Account.\n\nSubscribe with the CLI:\n\n```\nnylas webhook create \\\n  --url https://triage.yourcompany.com/nylas/webhook \\\n  --triggers message.created\n```\n\nOr directly:\n\n```\ncurl -X POST \"https://api.us.nylas.com/v3/webhooks\" \\\n  -H \"Authorization: Bearer $NYLAS_API_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"trigger_types\": [\"message.created\"],\n    \"webhook_url\": \"https://triage.yourcompany.com/nylas/webhook\",\n    \"description\": \"CI alert triage\"\n  }'\n```\n\nTwo things every webhook handler needs to get right, and they bite people who skip them.\n\n**Verify the signature.** Nylas signs each delivery with an `X-Nylas-Signature`\n\nheader — a hex HMAC-SHA256 of the raw request body using your webhook secret. Compute the same HMAC over the raw bytes and compare. If you use a constant-time compare like Node's `crypto.timingSafeEqual`\n\n, guard that both buffers are the same length first, because it throws on a length mismatch. The CLI ships `nylas webhook verify`\n\nto check a payload locally while you're developing.\n\n**Deduplicate.** Nylas guarantees at-least-once delivery — the same event can arrive up to three times. Dedupe on the **top-level notification id**, which stays constant across every retry of one event. That's your delivery-dedup key. The inner\n\n`data.object.id`\n\nis the message id; you can additionally guard on it so you never act twice on the same alert, even if your CI somehow sent two near-identical emails.That dedup discipline is half the value of this whole system. CI alert noise *is* duplication. If your handler is idempotent on the notification id and fingerprints the underlying alert, you've already killed most of the noise before the LLM does anything clever.\n\nThe `message.created`\n\npayload gives you enough to route — sender, subject, thread id — but **don't rely on the webhook payload for the body.** The Nylas docs themselves are inconsistent on whether the body comes inline, and there's a real edge: when a message exceeds ~1 MB the trigger becomes `message.created.truncated`\n\nand the body is dropped. CI alerts with full stack traces and log tails get big. So the safe pattern is: branch on `message.created.truncated`\n\n, and fetch the full message by id whenever you actually need the body.\n\n```\ncurl \"https://api.us.nylas.com/v3/grants/$GRANT_ID/messages/$MESSAGE_ID\" \\\n  -H \"Authorization: Bearer $NYLAS_API_KEY\"\n```\n\nThe CLI equivalent reads the message and renders the body:\n\n```\nnylas email read $MESSAGE_ID $GRANT_ID\n```\n\nNow you have the raw alert text — the failing job name, the stage, the stack trace, the log excerpt your CI tool stuffed into the email. That's the input to the part Nylas doesn't do for you.\n\nHere's the line I want to be honest about: **Nylas delivers the alert mail. It does not analyze it.** The clustering, the summarization, and the root-cause guess are all your application code and your LLM. Nylas is the inbox and the transport, not the brain.\n\nA workable shape:\n\n`integration-tests`\n\njob on `main`\n\nfailed 6 times in 12 minutes, here are the three distinct error signatures\" — and ask for a tight summary plus a probable root cause. Keep the prompt deterministic: low temperature, a fixed output shape (summary, suspected cause, confidence), and validate it before you trust it.The model is doing exactly the kind of judgment work it's good at — reading messy human-and-machine text and proposing a cause. It is not doing routing or counting; your code does that, because code is deterministic and the model isn't. If the alert text says \"connection refused to postgres,\" the model can reasonably suggest the DB container didn't come up. That's a hypothesis on-call can confirm in seconds — which is the whole point.\n\nThis is where the Agent Account earns its keep. Because the agent owns the address, its reply threads naturally with the original alert — Nylas groups it using the `In-Reply-To`\n\nand `References`\n\nheaders, so on-call sees your summary attached to the exact alert it explains, not floating in a separate channel.\n\nThe CLI reply preserves the thread automatically; it fetches the original to set the recipient and subject:\n\n```\nnylas email reply $MESSAGE_ID $GRANT_ID \\\n  --body \"Clustered 6 failures of integration-tests on main (last 12 min) into one incident. Probable cause: postgres service container failed health check before tests ran — 'connection refused' on :5432 in 5 of 6 runs. Suspect the DB image bump in #4821. Confidence: medium.\"\n```\n\nThe raw call sends a reply by setting `reply_to_message_id`\n\n, which is what keeps it in the thread:\n\n```\ncurl -X POST \"https://api.us.nylas.com/v3/grants/$GRANT_ID/messages/send\" \\\n  -H \"Authorization: Bearer $NYLAS_API_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{\n    \"reply_to_message_id\": \"'\"$MESSAGE_ID\"'\",\n    \"to\": [{ \"email\": \"oncall@yourcompany.com\" }],\n    \"body\": \"Clustered 6 failures of integration-tests on main into one incident. Probable cause: postgres health check failed before tests ran. Suspect the DB image bump in #4821. Confidence: medium.\"\n  }'\n```\n\nSet `to`\n\nto wherever on-call actually reads — a rotation alias, the team list, whatever your PagerDuty schedule resolves to. The reply lands in the alert thread *and* in on-call's inbox, which is the behavior you want: one authoritative summary message per incident, threaded under the noise it replaces.\n\nMarking the original alert read, if you want the agent's inbox to reflect what it's already processed, is a separate operation — a GET fetches, it doesn't mark anything read:\n\n```\nnylas email mark read $MESSAGE_ID $GRANT_ID\ncurl -X PUT \"https://api.us.nylas.com/v3/grants/$GRANT_ID/messages/$MESSAGE_ID\" \\\n  -H \"Authorization: Bearer $NYLAS_API_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{ \"unread\": false }'\n```\n\nSome CI notifiers are chattier than others, and you may want to keep low-value automated mail out of the agent's inbox entirely — successful-build confirmations, for instance. You can do that at the platform level with a **Rule**, before your handler ever sees it.\n\nOne important constraint: inbound rules match on the **sender only**. They accept `from.address`\n\n, `from.domain`\n\n, and `from.tld`\n\nwith operators like `is`\n\n, `is_not`\n\n, `contains`\n\n, and `in_list`\n\n. They **cannot** match on subject or body. So \"route anything with `[SUCCESS]`\n\nin the subject\" is *not* a rule — that's your app code after the webhook (fetch, classify, then `nylas email move`\n\nthe message). But \"route everything from `noreply@ci-provider.com`\n\ninto a firehose folder\" is exactly a rule's job.\n\nCreate the folder first and capture its id — the `assign_to_folder`\n\naction takes a folder **ID**, not a folder name:\n\n```\ncurl -X POST \"https://api.us.nylas.com/v3/grants/$GRANT_ID/folders\" \\\n  -H \"Authorization: Bearer $NYLAS_API_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{ \"name\": \"ci-firehose\" }'\n# capture the returned folder id into CI_FOLDER_ID\n```\n\nThen create the rule, pointing the action at that id:\n\n```\nnylas agent rule create \\\n  --name \"Route CI firehose\" \\\n  --condition from.domain,is,ci-provider.com \\\n  --action assign_to_folder=$CI_FOLDER_ID\n```\n\nIf you create the rule through the raw `/v3/rules`\n\nAPI instead, the action carries that same folder ID as its value — `{\"type\": \"assign_to_folder\", \"value\": \"<CI_FOLDER_ID>\"}`\n\n— never the folder name.\n\nA rule does nothing until it's attached to a workspace via `rule_ids`\n\n. The CLI's `agent rule create`\n\nattaches it to the account's default workspace for you; the raw flow is create the rule, then patch the workspace:\n\n```\ncurl -X PATCH \"https://api.us.nylas.com/v3/workspaces/$WORKSPACE_ID\" \\\n  -H \"Authorization: Bearer $NYLAS_API_KEY\" \\\n  -H \"Content-Type: application/json\" \\\n  -d '{ \"rule_ids\": [\"'\"$RULE_ID\"'\"] }'\n```\n\nIf you ever need to re-attach or swap rules from the CLI, that's `nylas workspace update <workspace-id> --rules-ids rule1,rule2`\n\n. Reach for this only when a sender is reliably noise. The interesting routing — by failure type, by severity, by repo — is content-based, and that belongs in your code.\n\n`nylas email mark read`\n\n.Once the basic loop works — receive, cluster, reply — the natural extensions are all just more of the same grant-scoped endpoints you've already wired:\n\nThe mental model that makes all of this click: the agent is a participant, not an observer. It has an address, it reads its own mail, and it replies in the thread. Everything else — the clustering, the summarizing, the root-cause guess — is your code doing what your code is good at, over an inbox Nylas keeps boringly simple.", "url": "https://wpnews.pro/news/route-ci-cd-alerts-to-an-agent-that-triages-by-email", "canonical_source": "https://dev.to/mqasimca/route-cicd-alerts-to-an-agent-that-triages-by-email-3g6a", "published_at": "2026-07-01 10:49:40+00:00", "updated_at": "2026-07-01 11:19:09.806546+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence", "large-language-models"], "entities": ["Nylas", "GitHub Actions", "Jenkins", "CircleCI", "CI Triage Bot"], "alternates": {"html": "https://wpnews.pro/news/route-ci-cd-alerts-to-an-agent-that-triages-by-email", "markdown": "https://wpnews.pro/news/route-ci-cd-alerts-to-an-agent-that-triages-by-email.md", "text": "https://wpnews.pro/news/route-ci-cd-alerts-to-an-agent-that-triages-by-email.txt", "jsonld": "https://wpnews.pro/news/route-ci-cd-alerts-to-an-agent-that-triages-by-email.jsonld"}}