Build an AI Agent Error Triage Pipeline with n8n, Gemini & Slack

A developer detailed a production-ready n8n workflow for triaging AI agent errors, which deduplicates alerts, classifies failures into five categories using Google Gemini, and posts severity-scored reports to Slack. The pipeline integrates with Sentry and generic webhooks, and includes a suppression window to reduce alert fatigue.

When LLM agents hit production traffic, manual error triage stops scaling fast. The same five failure types keep appearing under different traces, each re-diagnosed from scratch. Duplicate alerts flood Slack. Engineers start muting the channel. This post walks through a production-ready n8n workflow that reads a raw agent error, deduplicates it, classifies it against five real failure categories using Gemini, attaches a fix playbook, and posts a severity-scored triage report to Slack — without forcing a classification when the model isn't confident. Free workflow JSON at the end. LLM agent failures cluster into a small number of repeatable categories: | Category | Description | |---|---| | Tool Call Failure | External API, database, or function call failed outright | | Context Window Exhaustion | Conversation or tool history exceeded model context limit | | State Corruption | Agent memory/task state in unanticipated condition | | Retry Loop | Agent stuck repeating the same failed step without progress | | Uncertain | Error doesn't cleanly fit any named category | The problem: nothing remembers having seen these before. Same failure type, fresh diagnosis, every time. One bug generates dozens of identical alerts. Without severity signals, everything looks equally urgent — which means nothing is — and engineers tune out. ┌─────────────────────────────────────────────────────────────┐ │ n8n AI Agent Triage Workflow │ │ │ │ Webhook: POST /agent-error-triage │ │ ↓ │ │ Accepts: Sentry native payload OR { message: "..." } │ │ ↓ │ │ Extract & Dedup Code Node │ │ → Normalize payload title, culprit, stack trace │ │ → Build fingerprint hash of identifying fields │ │ → Check 30-min in-memory suppression window │ │ ↓ ↓ │ │ isDuplicate=true isDuplicate=false │ │ ↓ ↓ │ │ Skip No-Op Classify with Gemini │ │ ↓ │ │ Structured Output Parser │ │ → { category, confidence, severity, │ │ summary, reasoning } │ │ ↓ │ │ Build Triage Report Code Node │ │ → Map category → fix playbook │ │ → Assemble Slack message │ │ ↓ │ │ Post Triage Report: Slack Node │ │ → agent-alerts channel │ └─────────────────────────────────────────────────────────────┘ | Tool | Role | |---|---| | n8n self-hosted | Orchestration | | Sentry optional | Error source via webhook | | Google Gemini | AI classification | | LangChain in n8n | Chain + Structured Output Parser | | Slack Bot | Triage report delivery | Listens on POST /agent-error-triage . Accepts two payload shapes: Sentry native webhook: { "data": { "event": { "title": "ToolCallError: Timeout on /api/crm", "culprit": "agent.tool executor.call external", "exception": { "values": { "stacktrace": { "frames": ... } } } } } } Generic payload any error source : { "message": "Agent failed: context limit exceeded after 47 tool calls" } Both shapes are normalised to the same internal structure in the next node. This is the most important node in the pipeline. It does two things: Normalise the payload: js // Code Node — Extract & Dedup const input = $input.first .json; // Handle both Sentry and generic payloads let title, culprit, stackTrace; if input.data?.event { // Sentry native format const event = input.data.event; title = event.title || 'Unknown Error'; culprit = event.culprit || ''; const frames = event.exception?.values?. 0 ?.stacktrace?.frames || ; stackTrace = frames.map f = ${f.filename}:${f.lineno} in ${f.function} .join '\n' ; } else { // Generic format title = input.message || 'Unknown Error'; culprit = input.culprit || ''; stackTrace = input.stack trace || ''; } // Build fingerprint — hash of identifying fields not timestamps/IDs const fingerprintSource = ${title}::${culprit}::${stackTrace.substring 0, 200 } ; const fingerprint = fingerprintSource.split '' .reduce hash, char = { return hash << 5 - hash + char.charCodeAt 0 ; }, 0 .toString 36 ; // Check 30-minute suppression window using n8n static data const staticData = $getWorkflowStaticData 'global' ; const seen = staticData.seenFingerprints || {}; const now = Date.now ; const windowMs = 30 60 1000; // 30 minutes // Clean expired entries Object.keys seen .forEach fp = { if now - seen fp windowMs delete seen fp ; } ; const isDuplicate = seen fingerprint ; if isDuplicate { seen fingerprint = now; } staticData.seenFingerprints = seen; return { json: { title, culprit, stackTrace, fingerprint, isDuplicate } } ; Why non-cryptographic hashing: MD5/SHA isn't available in n8n's sandboxed JS environment by default. The djb2-style hash above is sufficient for fingerprinting — collision probability across a 30-minute window is negligible for error triage purposes. isDuplicate == true → Skip No-Op node, logs fingerprint isDuplicate == false → Continue to classification This is what prevents one bug from generating forty identical Slack reports. You are an AI agent error classifier. Classify the error into exactly one of these categories: 1. tool call failure — A call to an external API, database, function, or tool failed outright. Look for timeout errors, HTTP 4xx/5xx codes, connection refused, or authentication failures in tool calls. 2. context window exhaustion — The model's context limit was reached. Look for "context length exceeded", "max tokens", or errors after long chains of tool calls with no summarisation. 3. state corruption — The agent's internal memory or task state is in an unexpected condition. Look for key errors on expected state fields, type mismatches in state objects, or missing required state values. 4. retry loop — The agent is stuck repeating the same failed action without progress. Look for repeated identical tool calls, loop detection errors, or max retry exceeded messages. 5. uncertain — The error doesn't clearly fit any category above. Use this when evidence is ambiguous or the error message lacks enough context to classify confidently. Rules: - Choose uncertain rather than force a bad fit - Base classification only on evidence in the error, not assumptions - Confidence should reflect actual evidence strength, not optimism Return ONLY valid JSON: { "category": "tool call failure|context window exhaustion|state corruption|retry loop|uncertain", "confidence": 0.0-1.0, "severity": "low|medium|high|critical", "summary": "One sentence plain-language description of what happened", "reasoning": "Why you chose this category based on specific evidence in the error" } Key settings: 0.1 — keeps labels consistent across identical errors regardless of time-of-day gemini-1.5-flash or gemini-1.5-pro — confirm availability against your live credential before deploying 512 — sufficient for the JSON response The LangChain Structured Output Parser sub-node enforces the JSON schema. If parsing fails model added markdown fences, malformed JSON, etc. , route to the error branch rather than letting bad output propagate downstream. Maps category to fix playbook and assembles the Slack message: // Code Node — Build Triage Report const { category, confidence, severity, summary, reasoning } = $input.first .json; const { title, culprit, stackTrace } = $ 'Extract & Dedup' .first .json; const playbooks = { tool call failure: '1. Check external service status page for outages', '2. Review tool call logs for HTTP status codes and timeouts', '3. Verify API credentials and rate limit headers', '4. Add retry with exponential backoff if not already present' , context window exhaustion: '1. Add conversation summarisation step before context limit is reached', '2. Review tool call chain length — reduce unnecessary calls', '3. Implement sliding window for conversation history', '4. Consider chunking long tool outputs before adding to context' , state corruption: '1. Add state validation checks at each agent step', '2. Review state schema for missing required fields', '3. Check for race conditions in concurrent agent runs', '4. Add state recovery logic or restart from last valid checkpoint' , retry loop: '1. Implement loop detection — track repeated tool call signatures', '2. Set max retry limit with circuit breaker', '3. Add fallback action when retry limit is reached', '4. Review condition for loop exit — is the success criteria reachable?' , uncertain: '1. Review full stack trace manually — insufficient context for classification', '2. Improve agent error logging to include more diagnostic detail', '3. Check recent deployments for changes that could explain the failure', '4. Escalate to senior engineer if pattern recurs' }; const severityEmoji = { critical: '🔴', high: '🟠', medium: '🟡', low: '🟢' }; const confidenceBar = confidence = 0.8 ? 'High' : confidence = 0.5 ? 'Medium' : 'Low'; const playbook = playbooks category || playbooks.uncertain; // Build Slack Block Kit message const blocks = { type: 'header', text: { type: 'plain text', text: ${severityEmoji severity || '⚪'} AI Agent Error Triage Report } }, { type: 'section', fields: { type: 'mrkdwn', text: Category: \n${category.replace / /g, ' ' .toUpperCase } }, { type: 'mrkdwn', text: Severity: \n${severity.toUpperCase } }, { type: 'mrkdwn', text: Confidence: \n${confidenceBar} ${ confidence 100 .toFixed 0 }% }, { type: 'mrkdwn', text: Culprit: \n${culprit || 'Unknown'} } }, { type: 'section', text: { type: 'mrkdwn', text: Error: \n${title} } }, { type: 'section', text: { type: 'mrkdwn', text: Summary: \n${summary} } }, { type: 'section', text: { type: 'mrkdwn', text: AI Reasoning: \n${reasoning} } }, { type: 'section', text: { type: 'mrkdwn', text: Suggested Fix Steps: \n${playbook.join '\n' } } }, { type: 'divider' } ; return { json: { blocks, category, severity, confidence } } ; Configure the Slack node with: agent-alerts channel ID blocks array from previous node Slack bot scoping — important: The bot token should be limited to chat:write in the specific channel. Do not use a token with broad workspace permissions for this integration. Low confidence or JSON parse failures should not silently drop. Build an error branch: js // Error branch — uncertain fallback const errorContext = { timestamp: new Date .toISOString , stage: $input.first .json.failedNode || 'classification', rawError: $input.first .json.error?.message || 'Unknown', originalTitle: $ 'Extract & Dedup' .first .json?.title || 'Unknown' }; // Override with uncertain classification return { json: { category: 'uncertain', confidence: 0, severity: 'medium', summary: 'Classification failed — manual review required', reasoning: Workflow error at ${errorContext.stage}: ${errorContext.rawError} , ...errorContext } } ; Route this into the Build Triage Report node so the uncertain playbook still goes to Slack — a failed classification is still an incident that needs visibility. Gemini isn't a hard requirement. Replacing it means updating the credential and model node — the Structured Output Parser is model-agnostic as long as the JSON schema is valid. Claude and GPT-4o both work well for structured classification tasks. Note on Gemini rate limits: Free tier limits vary by model version and can change. For production traffic, confirm current limits before relying on the free tier — hitting a rate limit mid-triage means classification requests silently fail. The 30-minute window is a starting point. If the same bug tends to resurface after fixes and rollbacks, a shorter window makes sense. If your agents produce very infrequent errors, a longer window reduces noise further. js const windowMs = 30 60 1000; // Adjust this value Currently everything posts to Slack automatically. To add a human gate before high-severity classifications trigger downstream automation: Build Triage Report ↓ IF severity == critical ↓ Send approval email: Proceed / Escalate ↓ n8n Wait Node ↓ Webhook: receives decision ↓ Route to downstream automation Currently reports live in Slack only. For trend tracking which failure category appears most this month, which service generates the most errors : Post to Slack ↓ Jira: Create Issue OR Linear: Create Issue Fields: title=error title, label=category, priority=severity Sentry is one input, not the only one. Any system that can POST a JSON body to the webhook URL — a custom logging pipeline, a different error tracker, CloudWatch alerts, a Datadog webhook — can feed this workflow. Classification quality depends on logging quality. A stack trace with a real culprit and clear exception type gives the model solid evidence. A log entry that says Exception: failed gives it nothing — the workflow correctly labels that uncertain rather than guessing. Better logging matters more than classifier tuning for sparse errors. No persistent storage for trend analysis. The dedup fingerprint store uses n8n workflow static data, which is in-memory and resets on workflow restart. For cross-session dedup or trend reporting, connect to a database or sheet instead. The uncertain category is a feature. If you find everything is coming back uncertain, the problem is upstream logging quality, not the classifier threshold. IT Path Solutions published the complete n8n workflow — webhook trigger, Extract & Dedup node, Gemini classification chain, Structured Output Parser, fix playbook mapping, Slack Block Kit reporting, and error handling — all pre-connected. Import into any n8n instance, add credentials, send a test error through the webhook. 👉 Download the free n8n AI agent monitoring workflow https://www.itpathsolutions.com/ai-agent-monitoring-workflow Setup guide covers: Gemini credential config, Slack bot token scoping, Sentry webhook setup, dedup window tuning, and how to test end-to-end before pointing production traffic at it. The pipeline works because it does the repetitive diagnostic work — normalisation, dedup, classification, playbook lookup — before a human sees the alert. The on-call engineer opens Slack and sees a structured triage report instead of a raw stack trace to interpret. The honest uncertain fallback is what makes it trustworthy: it admits when it doesn't know rather than sending an engineer down the wrong path with a confident wrong answer. Run one test error through the webhook to confirm the chain works before enabling for production traffic. Full guide and JSON: itpathsolutions.com/ai-agent-monitoring-workflow https://www.itpathsolutions.com/ai-agent-monitoring-workflow Running something similar in production? What failure category shows up most? Drop it in the comments.