Asking my CI "why did that fail?": GitHub Actions cognee

I wanted to ask my CI a plain question: "which workflows failed recently, and why?" So I built a connector that turns GitHub Actions runs into memory an LLM can answer from.

When a build goes red, finding out why means a lot of clicking. You open the Actions tab, pick a run, expand the failed job, then scroll a raw log full of timestamps and ANSI codes until you find the ##[error] line.

That works for one run. It breaks down when you want the pattern: is this the same test that failed last week? Which workflow keeps needing approval? What changed on main lately? The answers are spread over dozens of runs, and none of them are searchable together.

I ran into this myself while contributing to cognee-community from a fork. In its Actions tab, my runs sat alongside many others marked action_required, because workflows on fork PRs wait for a maintainer to approve them. Real failures were mixed in with those waiting runs, and the only way to tell them apart was to open each one.

I contributed a GitHub Actions data-source connector to cognee-community (PR #383). cognee is an open-source memory layer for AI apps: you add data, it builds a knowledge graph plus embeddings, and you query it in natural language.

The connector is a single function, github_actions_source(). You give it one or more owner/repo names, and it returns a dlt source that cognee can ingest directly. Each completed workflow run becomes one readable document. Failed runs can include a short summary of their log instead of the whole thing.

The connector does the tedious part: it reads the API, trims the logs and writes each run up as markdown. cognee does the rest.

Here is the real document for the failed bump run from the demo below. Its whole failure log comes down to two error lines:

Repository: topoteretes/cognee-community
Workflow: automation | Bump cognee in the database adapters (.github/workflows/bump_cognee.yml)
Run: #22, attempt 1
Conclusion: failure
Event: schedule
Branch: main
Commit: 5a595a6 Merge pull request #349 from AbdulWasih05/feat/asana-connector
Started: 2026-10-28T06:33:36Z
Finished: 2026-10-28T06:33:45Z

## Jobs
- bump: failure
  - failed step: Check the PR token is configured

## Failure log: bump
Errors:
- Set the COGNEE_BUMP_PR_TOKEN secret (see the header of this workflow).
- Process completed with exit code 1.

A few kilobytes of noisy log become a dozen lines an LLM can reason over. Each document also keeps the run's URL, so an answer can always be traced back to GitHub.

I pointed the connector at the 15 latest completed runs of topoteretes/cognee-community on 28 October 2026. The LLM was an Azure OpenAI deployment, and embeddings came from text-embedding-3-large. Everything below is real output.

1. Fetch the runs. The connector pulled 15 runs, their jobs, and the logs of the failed jobs in 8.2 s. Only 3 had actually failed. 10 were waiting for approval.

2. Build the memory. One call, cognee.remember(), stored the 15 documents and built the knowledge graph and embeddings in 105.3 s.

await cognee.remember(
    github_actions_source(
        repos=["topoteretes/cognee-community"],
        max_runs_per_repo=15,
        include_failed_logs=True,
    ),
    dataset_name="github-actions-blog",
)

3. Ask questions. Here are cognee's answers, unedited:

Q: Which workflows failed recently, and why?

On 2026-10-28, three workflows failed:

Clay connector tests (#1): Python 3.11 and 3.12 both failed test_check_active_cancellation; extraction raised ResourceExtractionError: Sync cancelled by host. Ruff Lint and Format (#1098): Clay code had a nested-if lint issue, unescaped regex patterns in tests, and a formatting issue. Scheduled "Bump cognee in the database adapters" (#22): failed because the COGNEE_BUMP_PR_TOKEN secret was not configured. The Klaviyo connector runs were marked action_required, not failed.

Q: How do I fix the failing bump workflow?

Add the COGNEE_BUMP_PR_TOKEN secret to the repository's GitHub Actions secrets (see the workflow header for details), then rerun the scheduled workflow. It failed because that required secret wasn't configured.

Q: Which workflows are waiting for approval?

The workflows marked action_required (waiting for approval) are:

• Klaviyo connector tests
• Vercel connector tests
• Prometheus connector
• Ruff Lint and Format (runs #1094, #1095, #1101, and #1102)

The first two answers are what I wanted. Each gives the cause and the fix, down to the failing test name and the missing secret, with no log open. The first even points out that the Klaviyo runs were waiting, not failing.

The third answer is close but not complete. It lists 7 of the 10 waiting runs and misses Ruff runs #1096, #1097 and #1100. Graph completion answers from the context it retrieves, so it isn't an exhaustive query. For an exact list, the table from step 1 is still the right tool.

It helps most when you need the story across many runs, not the details of one.

Because it's a plain cognee dataset, CI history can sit next to your other data, such as docs, issues or code. One question can then draw on all of them.

You need Python 3.11 to 3.13, a GitHub token, and an LLM key for cognee. For a public repo, a token with no extra scopes is enough. For a private repo it needs repo or actions:read.

git clone https://github.com/topoteretes/cognee-community
cd cognee-community/packages/connector/github-actions
uv sync
export GITHUB_TOKEN=$(gh auth token)
export LLM_API_KEY=...   # or put cognee's settings in a .env file
python
import asyncio
import cognee
from cognee_community_connector_github_actions import github_actions_source

async def main():
    source = github_actions_source(
        repos=["owner/repo"],
        max_runs_per_repo=50,
        include_failed_logs=True,
    )
    await cognee.remember(source, dataset_name="github-actions")
    print(await cognee.search(
        query_text="Which workflows failed recently, and why?",
        query_type=cognee.SearchType.GRAPH_COMPLETION,
        datasets=["github-actions"],
    ))

asyncio.run(main())

Useful options:

workflow and branch narrow the runs to one workflow file or one branch.created_after limits how far back to go.log_tail_lines controls how much of each failed step's output is kept (default 50). Run it again later to sync. Runs that have dropped out of the window are removed from memory, so the dataset tracks your current CI.

If the PR is still open when you read this, use the branch from PR #383.

##[error] lines and the output just before them made the documents small and the answers sharper. If you try it, I'd like to hear what you ask your CI. Feedback and issues on the PR are welcome.

Links: cognee · cognee-community · PR #383