Asking my CI "why did that fail?": GitHub Actions cognee

A developer contributed a GitHub Actions data-source connector, github_actions_source(), to the open-source cognee-community project, converting completed workflow runs into markdown documents that cognee ingests into a knowledge graph and embeddings for natural-language querying. In a demo against the 15 latest completed runs of topoteretes/cognee-community, the connector fetched runs, jobs and failed-job logs in 8.2 seconds, and cognee.remember() built the memory in 105.3 seconds; the LLM then correctly identified three failures, including a scheduled bump workflow that failed on a missing COGNEE_BUMP_PR_TOKEN secret.

I wanted to ask my CI a plain question: "which workflows failed recently, and why?" So I built a connector that turns GitHub Actions runs into memory an LLM can answer from. When a build goes red, finding out why means a lot of clicking. You open the Actions tab, pick a run, expand the failed job, then scroll a raw log full of timestamps and ANSI codes until you find the error line. That works for one run. It breaks down when you want the pattern: is this the same test that failed last week? Which workflow keeps needing approval? What changed on main lately? The answers are spread over dozens of runs, and none of them are searchable together. I ran into this myself while contributing to cognee-community from a fork. In its Actions tab, my runs sat alongside many others marked action required , because workflows on fork PRs wait for a maintainer to approve them. Real failures were mixed in with those waiting runs, and the only way to tell them apart was to open each one. I contributed a GitHub Actions data-source connector to cognee-community https://github.com/topoteretes/cognee-community PR 383 https://github.com/topoteretes/cognee-community/pull/383 . cognee https://github.com/topoteretes/cognee is an open-source memory layer for AI apps: you add data, it builds a knowledge graph plus embeddings, and you query it in natural language. The connector is a single function, github actions source . You give it one or more owner/repo names, and it returns a dlt https://dlthub.com source that cognee can ingest directly. Each completed workflow run becomes one readable document. Failed runs can include a short summary of their log instead of the whole thing. The connector does the tedious part: it reads the API, trims the logs and writes each run up as markdown. cognee does the rest. Here is the real document for the failed bump run from the demo below. Its whole failure log comes down to two error lines: Repository: topoteretes/cognee-community Workflow: automation | Bump cognee in the database adapters .github/workflows/bump cognee.yml Run: 22, attempt 1 Conclusion: failure Event: schedule Branch: main Commit: 5a595a6 Merge pull request 349 from AbdulWasih05/feat/asana-connector Started: 2026-10-28T06:33:36Z Finished: 2026-10-28T06:33:45Z Jobs - bump: failure - failed step: Check the PR token is configured Failure log: bump Errors: - Set the COGNEE BUMP PR TOKEN secret see the header of this workflow . - Process completed with exit code 1. A few kilobytes of noisy log become a dozen lines an LLM can reason over. Each document also keeps the run's URL, so an answer can always be traced back to GitHub. I pointed the connector at the 15 latest completed runs of topoteretes/cognee-community on 28 October 2026. The LLM was an Azure OpenAI deployment, and embeddings came from text-embedding-3-large . Everything below is real output. 1. Fetch the runs. The connector pulled 15 runs, their jobs, and the logs of the failed jobs in 8.2 s . Only 3 had actually failed. 10 were waiting for approval. 2. Build the memory. One call, cognee.remember , stored the 15 documents and built the knowledge graph and embeddings in 105.3 s . await cognee.remember github actions source repos= "topoteretes/cognee-community" , max runs per repo=15, include failed logs=True, , dataset name="github-actions-blog", 3. Ask questions. Here are cognee's answers, unedited: Q: Which workflows failed recently, and why? On 2026-10-28, three workflows failed: Clay connector tests 1 : Python 3.11 and 3.12 both failed test check active cancellation ; extraction raised ResourceExtractionError: Sync cancelled by host . Ruff Lint and Format 1098 : Clay code had a nested-if lint issue, unescaped regex patterns in tests, and a formatting issue. Scheduled "Bump cognee in the database adapters" 22 : failed because the COGNEE BUMP PR TOKEN secret was not configured. The Klaviyo connector runs were marked action required , not failed. Q: How do I fix the failing bump workflow? Add the COGNEE BUMP PR TOKEN secret to the repository's GitHub Actions secrets see the workflow header for details , then rerun the scheduled workflow. It failed because that required secret wasn't configured. Q: Which workflows are waiting for approval? The workflows marked action required waiting for approval are: - Klaviyo connector tests - Vercel connector tests - Prometheus connector - Ruff Lint and Format runs 1094, 1095, 1101, and 1102 The first two answers are what I wanted. Each gives the cause and the fix, down to the failing test name and the missing secret, with no log open. The first even points out that the Klaviyo runs were waiting, not failing. The third answer is close but not complete. It lists 7 of the 10 waiting runs and misses Ruff runs 1096, 1097 and 1100. Graph completion answers from the context it retrieves, so it isn't an exhaustive query. For an exact list, the table from step 1 is still the right tool. It helps most when you need the story across many runs, not the details of one. | Who | Question they can ask | What used to happen | |---|---|---| | Maintainer | "Which PRs are stuck waiting for workflow approval?" | Scan the Actions tab run by run | | Contributor | "Why did the tests fail on my branch?" | Open the job, expand the step, scroll the raw log | | On-call / release owner | "Is this the same error that broke main last week?" | Rely on memory, or search Slack | | Team lead | "Summarize recent CI activity in this repo" | Build a dashboard, or just not know | | AI coding agent | Reads the failure summary before proposing a fix | Fed a whole log, or nothing | Because it's a plain cognee dataset, CI history can sit next to your other data, such as docs, issues or code. One question can then draw on all of them. You need Python 3.11 to 3.13, a GitHub token, and an LLM key for cognee. For a public repo, a token with no extra scopes is enough. For a private repo it needs repo or actions:read . git clone https://github.com/topoteretes/cognee-community cd cognee-community/packages/connector/github-actions uv sync export GITHUB TOKEN=$ gh auth token export LLM API KEY=... or put cognee's settings in a .env file python import asyncio import cognee from cognee community connector github actions import github actions source async def main : source = github actions source repos= "owner/repo" , max runs per repo=50, include failed logs=True, await cognee.remember source, dataset name="github-actions" print await cognee.search query text="Which workflows failed recently, and why?", query type=cognee.SearchType.GRAPH COMPLETION, datasets= "github-actions" , asyncio.run main Useful options: workflow and branch narrow the runs to one workflow file or one branch. created after limits how far back to go. log tail lines controls how much of each failed step's output is kept default 50 . Run it again later to sync. Runs that have dropped out of the window are removed from memory, so the dataset tracks your current CI. If the PR is still open when you read this, use the branch from PR 383 https://github.com/topoteretes/cognee-community/pull/383 . error lines and the output just before them made the documents small and the answers sharper. If you try it, I'd like to hear what you ask your CI. Feedback and issues on the PR are welcome. Links: cognee https://github.com/topoteretes/cognee · cognee-community https://github.com/topoteretes/cognee-community · PR 383 https://github.com/topoteretes/cognee-community/pull/383