Detecting Anomalies in CI/CD Pipelines with ML

A developer built PipelineSentinel, a deployable layer that scores each CI/CD run against its own pipeline's history to flag anomalies before a human has to triage them. The tool pulls run metadata from the GitHub Actions REST API — duration, pass/fail outcome, retry count, triggering event, and branch — and uses a scikit-learn IsolationForest over duration, failure status, and attempt count, chosen because it is unsupervised and adapts to each pipeline's own distribution rather than a fixed global threshold.

Your CI run turns red. You open the logs, scroll through a wall of output, and fifteen minutes later find the answer: it's that flaky test again — the one everyone half-recognizes but nobody's fixed. You retry the job, it goes green, you move on. Multiply that by every engineer on the team, every week, and it adds up to real hours spent on triage that a five-second glance shouldn't require. The question I wanted to answer: could the pipeline tell you this run looks unusual before a human has to dig in? A single pass/fail bit isn't enough to build on. Pipelines fail for structurally different reasons — a flaky test, an infra hiccup, a dependency break, resource exhaustion — and "unusual" is relative to that pipeline's own history, not a universal threshold. A 10-minute run might be completely normal for one workflow and a five-alarm anomaly for another that usually finishes in 30 seconds. Fixed thresholds and simple failure-rate alerts miss this: they treat every pipeline the same and only catch what you already thought to watch for. PipelineSentinel is a small, deployable layer that sits on top of existing CI tooling and scores each run against its own pipeline's history. Data. It pulls run metadata straight from the GitHub Actions REST API — run duration, pass/fail outcome, retry/attempt count, triggering event, and branch. No log-parsing required for a first pass. Model. The baseline model is an IsolationForest scikit-learn over three signals: run duration, whether the run failed, and how many attempts it took. IsolationForest is a good first choice here because it's unsupervised — it doesn't need a hand-labeled set of "here's what an anomaly looks like," which you don't have on day one — and it adapts to each pipeline's own distribution instead of a fixed global cutoff. python python from sklearn.ensemble import IsolationForest FEATURE COLUMNS = "duration seconds", "failed", "run attempt" model = IsolationForest contamination=contamination, random state=42 df "is anomaly" = model.fit predict df FEATURE COLUMNS == -1