Route CI/CD alerts to an agent that triages by email A developer at Nylas built a CI/CD alert triage agent that uses its own email inbox to receive, cluster, and summarize noisy pipeline notifications. The agent, created via Nylas's custom grant API, replies in the same email thread with a probable root cause, reducing on-call engineers' inbox clutter from dozens of messages to one. The system is designed for triage and summarization, not as a replacement for full observability stacks. CI alert email is noisy, and most of it is ignorable. If you've ever owned a pipeline at any scale, you know the shape of it: a flaky integration test trips, GitHub Actions emails you, a retry passes, and now there are three messages in your inbox about a problem that fixed itself. Multiply that by every branch, every nightly, every deploy, and the signal you actually care about — the one real failure — is buried under forty notifications that all look identical until you open them. The usual fix is to wire alerts into Slack and add a bot that reacts to emoji. That works right up until the channel becomes the new noisy inbox. The other usual fix is to point an LLM at a human's mailbox and let it "summarize your morning." That's a demo, not a system — the moment you want the summarizer to reply , to be a participant in the alert thread that on-call is reading, you need it to own an address, not borrow yours. That's the angle here. Give the triage agent its own inbox. Your CI system already knows how to email failures somewhere — point it at an address the agent controls, let the agent cluster and summarize the incoming alerts, and have it reply in the same thread with a probable root cause so whoever's on-call reads one message instead of forty. I work on the Nylas CLI, so the terminal commands below are the exact ones I reach for when I'm building this. I'll show the raw API call alongside each one, because in production this lives in a service, not a shell. An Agent Account is a Nylas grant with its own email address. That's the whole trick. It's not a new product surface you have to learn — it's a grant id that happens to belong to a programmatic mailbox instead of a human's connected Google or Microsoft account. Every grant-scoped endpoint you already know works against it unchanged: GET /v3/grants/{grant id}/messages , POST .../messages/send , threads, folders, all of it. So the data plane is nothing new. You point your CI notifier at ci-alerts@yourcompany.com , the agent receives mail there like any inbox, and you read and reply with the same endpoints you'd use for a connected account. One honest caveat up front: this is alert triage and summarization over email . It is not a replacement for your observability stack. The agent doesn't have your traces, your metrics, or your deploy history unless you give them to it. What it does well is take the firehose of alert emails — which is genuinely noisy and mostly redundant — collapse it into clusters, and hand on-call a probable cause to start from. Treat its root-cause guess as a hypothesis, not a verdict. Create the account. The CLI wraps POST /v3/connect/custom with provider: nylas : nylas agent account create ci-alerts@yourcompany.com --name "CI Triage Bot" The raw call, if you're provisioning from a service: curl -X POST "https://api.us.nylas.com/v3/connect/custom" \ -H "Authorization: Bearer $NYLAS API KEY" \ -H "Content-Type: application/json" \ -d '{ "provider": "nylas", "name": "CI Triage Bot", "settings": { "email": "ci-alerts@yourcompany.com" } }' The email has to be on a registered domain — either your own custom domain or a Nylas .nylas.email trial subdomain. There's no OAuth, no refresh token, no consent screen, because nobody's connecting an external account; you're minting one. The API auto-creates a default workspace and policy for the account, which matters later when we route noisy senders. Now go into your CI system's notification settings — GitHub Actions, Jenkins, CircleCI, whatever you run — and set the alert recipient to ci-alerts@yourcompany.com . From the agent's side, an alert is just inbound mail. Inbound mail fires the standard message.created webhook. The thing to internalize: webhooks are application-scoped, not grant-scoped. You subscribe once at the app level, and events for every grant in your app land at the same endpoint, each payload carrying a grant id you filter on. You don't register a webhook per Agent Account. Subscribe with the CLI: nylas webhook create \ --url https://triage.yourcompany.com/nylas/webhook \ --triggers message.created Or directly: curl -X POST "https://api.us.nylas.com/v3/webhooks" \ -H "Authorization: Bearer $NYLAS API KEY" \ -H "Content-Type: application/json" \ -d '{ "trigger types": "message.created" , "webhook url": "https://triage.yourcompany.com/nylas/webhook", "description": "CI alert triage" }' Two things every webhook handler needs to get right, and they bite people who skip them. Verify the signature. Nylas signs each delivery with an X-Nylas-Signature header — a hex HMAC-SHA256 of the raw request body using your webhook secret. Compute the same HMAC over the raw bytes and compare. If you use a constant-time compare like Node's crypto.timingSafeEqual , guard that both buffers are the same length first, because it throws on a length mismatch. The CLI ships nylas webhook verify to check a payload locally while you're developing. Deduplicate. Nylas guarantees at-least-once delivery — the same event can arrive up to three times. Dedupe on the top-level notification id , which stays constant across every retry of one event. That's your delivery-dedup key. The inner data.object.id is the message id; you can additionally guard on it so you never act twice on the same alert, even if your CI somehow sent two near-identical emails.That dedup discipline is half the value of this whole system. CI alert noise is duplication. If your handler is idempotent on the notification id and fingerprints the underlying alert, you've already killed most of the noise before the LLM does anything clever. The message.created payload gives you enough to route — sender, subject, thread id — but don't rely on the webhook payload for the body. The Nylas docs themselves are inconsistent on whether the body comes inline, and there's a real edge: when a message exceeds ~1 MB the trigger becomes message.created.truncated and the body is dropped. CI alerts with full stack traces and log tails get big. So the safe pattern is: branch on message.created.truncated , and fetch the full message by id whenever you actually need the body. curl "https://api.us.nylas.com/v3/grants/$GRANT ID/messages/$MESSAGE ID" \ -H "Authorization: Bearer $NYLAS API KEY" The CLI equivalent reads the message and renders the body: nylas email read $MESSAGE ID $GRANT ID Now you have the raw alert text — the failing job name, the stage, the stack trace, the log excerpt your CI tool stuffed into the email. That's the input to the part Nylas doesn't do for you. Here's the line I want to be honest about: Nylas delivers the alert mail. It does not analyze it. The clustering, the summarization, and the root-cause guess are all your application code and your LLM. Nylas is the inbox and the transport, not the brain. A workable shape: integration-tests job on main failed 6 times in 12 minutes, here are the three distinct error signatures" — and ask for a tight summary plus a probable root cause. Keep the prompt deterministic: low temperature, a fixed output shape summary, suspected cause, confidence , and validate it before you trust it.The model is doing exactly the kind of judgment work it's good at — reading messy human-and-machine text and proposing a cause. It is not doing routing or counting; your code does that, because code is deterministic and the model isn't. If the alert text says "connection refused to postgres," the model can reasonably suggest the DB container didn't come up. That's a hypothesis on-call can confirm in seconds — which is the whole point. This is where the Agent Account earns its keep. Because the agent owns the address, its reply threads naturally with the original alert — Nylas groups it using the In-Reply-To and References headers, so on-call sees your summary attached to the exact alert it explains, not floating in a separate channel. The CLI reply preserves the thread automatically; it fetches the original to set the recipient and subject: nylas email reply $MESSAGE ID $GRANT ID \ --body "Clustered 6 failures of integration-tests on main last 12 min into one incident. Probable cause: postgres service container failed health check before tests ran — 'connection refused' on :5432 in 5 of 6 runs. Suspect the DB image bump in 4821. Confidence: medium." The raw call sends a reply by setting reply to message id , which is what keeps it in the thread: curl -X POST "https://api.us.nylas.com/v3/grants/$GRANT ID/messages/send" \ -H "Authorization: Bearer $NYLAS API KEY" \ -H "Content-Type: application/json" \ -d '{ "reply to message id": "'"$MESSAGE ID"'", "to": { "email": "oncall@yourcompany.com" } , "body": "Clustered 6 failures of integration-tests on main into one incident. Probable cause: postgres health check failed before tests ran. Suspect the DB image bump in 4821. Confidence: medium." }' Set to to wherever on-call actually reads — a rotation alias, the team list, whatever your PagerDuty schedule resolves to. The reply lands in the alert thread and in on-call's inbox, which is the behavior you want: one authoritative summary message per incident, threaded under the noise it replaces. Marking the original alert read, if you want the agent's inbox to reflect what it's already processed, is a separate operation — a GET fetches, it doesn't mark anything read: nylas email mark read $MESSAGE ID $GRANT ID curl -X PUT "https://api.us.nylas.com/v3/grants/$GRANT ID/messages/$MESSAGE ID" \ -H "Authorization: Bearer $NYLAS API KEY" \ -H "Content-Type: application/json" \ -d '{ "unread": false }' Some CI notifiers are chattier than others, and you may want to keep low-value automated mail out of the agent's inbox entirely — successful-build confirmations, for instance. You can do that at the platform level with a Rule , before your handler ever sees it. One important constraint: inbound rules match on the sender only . They accept from.address , from.domain , and from.tld with operators like is , is not , contains , and in list . They cannot match on subject or body. So "route anything with SUCCESS in the subject" is not a rule — that's your app code after the webhook fetch, classify, then nylas email move the message . But "route everything from noreply@ci-provider.com into a firehose folder" is exactly a rule's job. Create the folder first and capture its id — the assign to folder action takes a folder ID , not a folder name: curl -X POST "https://api.us.nylas.com/v3/grants/$GRANT ID/folders" \ -H "Authorization: Bearer $NYLAS API KEY" \ -H "Content-Type: application/json" \ -d '{ "name": "ci-firehose" }' capture the returned folder id into CI FOLDER ID Then create the rule, pointing the action at that id: nylas agent rule create \ --name "Route CI firehose" \ --condition from.domain,is,ci-provider.com \ --action assign to folder=$CI FOLDER ID If you create the rule through the raw /v3/rules API instead, the action carries that same folder ID as its value — {"type": "assign to folder", "value": "