# Building explain-ci: The AI Was the Smallest Part

> Source: <https://dev.to/gopalcnepal/building-explain-ci-the-ai-was-the-smallest-part-1n2h>
> Published: 2026-09-10 15:04:23+00:00

CI goes red. Before I can think about the actual fix, I'm expanding the failing job and scrolling through three hundred lines of dependency resolution to find the one line that matters. Then the next person on the repo does the same thing, from scratch.

So I built [explain-ci](https://github.com/gopalcnepal/explain-ci): a GitHub Action that reads the failed job's log and posts a plain-English explanation as a PR comment. By the time anyone opens the PR, it's already there.

That's a real run. Here's what CI printed:

```
E       AssertionError: assert [] == ['washers']
E         Right contains one more item: 'washers'
```

And what the comment said: the function used `<` where it needed `<=`, so items sitting exactly on the threshold were skipped. The traceback tells you the assertion failed. It never tells you why. The model worked that out from the log alone — it never saw the source file.

I expected this to be the interesting bit. It isn't.

GitHub's job logs carry structure markers — `##[group]`, `##[endgroup]`, `##[error]`. Find the first error, walk backwards to see which step was running, slice the log into three sections. That's plain string handling, no model involved. Then those sections go to the provider with a short prompt asking for two things: root cause and suggested fix.

That's the whole feature. One API call and some parsing.

The one decision worth anything is what you *don't* send. Not the whole log — each section is capped, and the cap keeps the **tail**, because errors cluster at the end while the top is setup noise. That holds whether the job printed 200 lines or 200,000.

Everything that took real time was about making the thing well-behaved enough that you'd leave it installed.

**It must never fail your pipeline.** An action that explains failures has no business causing one. If the GitHub API hiccups or the provider returns a 500, it emits a warning and exits 0. Your build status should reflect your build, not my action's bad day.

**It edits its own comment.** Re-run a failing job four times and you don't want four bot comments. Each one carries a hidden marker, so a re-run finds the previous comment and updates it in place.

**Only the newest run comments.** Scoped per workflow — a repo with five workflows shouldn't get five explanations.

**Credentials get scrubbed first.** Log text goes to a third party. GitHub masks the secrets it registered, but tokens minted or echoed *during* a job — an OIDC session key, a git remote carrying a password, a verbose `curl` dumping its auth header — reach the log unmasked. So they're stripped before anything leaves the runner.

None of that is AI work. All of it is the difference between a demo and something you'd actually install.

The stale-run check was supposed to stop old runs commenting on new commits. It queried the endpoint that returns runs from *every* workflow in the repo, so if any other workflow had run more recently, the current run decided it was stale and said nothing. In a single-workflow repo it looked fine. In a real one it silently did nothing.

The second is more interesting, because I caused it while fixing something else. My first pass at secret redaction flagged any long run of characters as a possible key. Deep file paths match that trivially:

``` php
/home/runner/work/myorg/myproject/src/main/java/com/example/Service.java
  ->  /[REDACTED].java
```

It over-redacted rather than leaked, so nothing was exposed — but it removed the failing file path, which is exactly the context the model needs. My tests missed it because every path I'd invented happened to contain dots and underscores, which break the match. A real path didn't.

Both bugs have the same shape: the code was tested against what I imagined the input looked like, not against a real one.

```
explain-failure:
  needs: test
  if: always()
  runs-on: ubuntu-latest
  steps:
    - uses: gopalcnepal/explain-ci@v1
      with:
        api_key: ${{ secrets.OPENAI_API_KEY }}
```

It has to be its own job — the failed job's log isn't queryable through the API until that job has finished.

Bring your own key. It speaks the OpenAI-compatible API, so OpenAI, Gemini, Claude, Groq, Mistral and friends all work, and `base_url` points it at a self-hosted Ollama if your logs can't leave your network.

It's free and MIT. If you try it and it gets something wrong, I'd like to hear about it.