Editorial analysis: For security teams and ML practitioners, the most relevant takeaway is the operational pattern: decompose a complex detection problem into narrowly scoped agentic components, combine deterministic rules with model-derived signals, and close the loop with human feedback to maintain signal quality. This approach addresses volume, reduces analyst load, and preserves traceable decision logic, while introducing tradeoffs around model maintenance and orchestration complexity.
What happened
Red Canary's engineering blog reports the company built an AI triage agent that uses a graph of orchestrated subagents to handle reported phishing emails, and the blog attributes 94% accuracy to that workflow. The blog describes discrete subagents for parsing and enrichment, feature extraction that mixes traditional boolean checks and AI-guided true/false feature outputs, a rules engine for deterministic outcomes, and a classification subagent that consumes those features. Red Canary's product page advertises performance metrics that include 99% noise reduction, 3 minutes for AI agent investigations, and a 60% reduction in triage and notification time. The blog cites the Anti-Phishing Working Group for volume context, pointing to more than 1.1 million phishing emails in Q2 2025.
Technical details reported
- • Parsing and enrichment: the first subagent normalizes raw email into a data object and enriches metadata with domain reputation and historical indicators, per the blog. - • Feature extraction: Red Canary describes a hybrid approach where classic code checks produce boolean features and AI-powered checks use prompting to return true/false judgments plus reasoning, which the company says captures sentiment, intent, and other NLP-derived signals. The blog frames these as distilled features for downstream deterministic logic. - • Rules engine and classification: the workflow applies deterministic rules before classification to ensure non-ML overrides, then runs classification and escalation logic, according to the blog.
Editorial analysis - technical context: The architecture described mirrors an emerging SecOps pattern where agentic orchestration coordinates small, testable components instead of relying on one large model. That pattern improves modular testing and auditability, because rules and boolean features produce explainable decision points. At the same time, incorporating LLM-driven checks introduces recurring engineering needs: prompt engineering, drift monitoring, cost and latency budgeting, and evaluation pipelines for feature correctness.
What to watch
observers should track independent evaluations of the claimed 94% end-to-end accuracy and the product-page performance metrics; look for published false-positive and false-negative rates, latency under production load, and documentation of the feedback loop used for retraining or retuning AI checks. Also monitor how vendors reconcile deterministic rules with model outputs when they conflict, and whether privacy or data-retention controls are documented for enriched metadata.
Key Points #
- 1Modular agentic pipelines pair deterministic rules with LLM-derived features to scale triage while retaining explainability for analysts.
- 2Hybrid feature extraction converts complex NLP signals into boolean features, simplifying downstream rule logic and audits.
- 3Scaling LLM-based checks raises operational needs: prompt engineering, drift monitoring, latency and cost controls, and human-in-the-loop feedback.
Scoring Rationale #
This report describes a practical, deployable pattern for automating phishing triage that matters to SecOps and ML engineers, but it is an incremental operational model rather than a frontier research breakthrough. Claims are currently vendor-reported and need independent validation.
Practice interview problems based on real data
1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.