{"slug": "asking-my-ci-why-did-that-fail-github-actions-cognee", "title": "Asking my CI \"why did that fail?\": GitHub Actions cognee", "summary": "A developer contributed a GitHub Actions data-source connector, github_actions_source(), to the open-source cognee-community project, converting completed workflow runs into markdown documents that cognee ingests into a knowledge graph and embeddings for natural-language querying. In a demo against the 15 latest completed runs of topoteretes/cognee-community, the connector fetched runs, jobs and failed-job logs in 8.2 seconds, and cognee.remember() built the memory in 105.3 seconds; the LLM then correctly identified three failures, including a scheduled bump workflow that failed on a missing COGNEE_BUMP_PR_TOKEN secret.", "body_md": "I wanted to ask my CI a plain question: \"which workflows failed recently, and why?\" So I built a connector that turns GitHub Actions runs into memory an LLM can answer from.\n\nWhen a build goes red, finding out why means a lot of clicking. You open the Actions tab, pick a run, expand the failed job, then scroll a raw log full of timestamps and ANSI codes until you find the `##[error]` line.\n\nThat works for one run. It breaks down when you want the pattern: is this the same test that failed last week? Which workflow keeps needing approval? What changed on `main` lately? The answers are spread over dozens of runs, and none of them are searchable together.\n\nI ran into this myself while contributing to cognee-community from a fork. In its Actions tab, my runs sat alongside many others marked `action_required`, because workflows on fork PRs wait for a maintainer to approve them. Real failures were mixed in with those waiting runs, and the only way to tell them apart was to open each one.\n\nI contributed a **GitHub Actions data-source connector** to [cognee-community](https://github.com/topoteretes/cognee-community) ([PR #383](https://github.com/topoteretes/cognee-community/pull/383)). [cognee](https://github.com/topoteretes/cognee) is an open-source memory layer for AI apps: you add data, it builds a knowledge graph plus embeddings, and you query it in natural language.\n\nThe connector is a single function, `github_actions_source()`. You give it one or more `owner/repo` names, and it returns a [dlt](https://dlthub.com) source that cognee can ingest directly. Each completed workflow run becomes one readable document. Failed runs can include a short summary of their log instead of the whole thing.\n\nThe connector does the tedious part: it reads the API, trims the logs and writes each run up as markdown. cognee does the rest.\n\nHere is the real document for the failed bump run from the demo below. Its whole failure log comes down to two error lines:\n\n```\nRepository: topoteretes/cognee-community\nWorkflow: automation | Bump cognee in the database adapters (.github/workflows/bump_cognee.yml)\nRun: #22, attempt 1\nConclusion: failure\nEvent: schedule\nBranch: main\nCommit: 5a595a6 Merge pull request #349 from AbdulWasih05/feat/asana-connector\nStarted: 2026-10-28T06:33:36Z\nFinished: 2026-10-28T06:33:45Z\n\n## Jobs\n- bump: failure\n  - failed step: Check the PR token is configured\n\n## Failure log: bump\nErrors:\n- Set the COGNEE_BUMP_PR_TOKEN secret (see the header of this workflow).\n- Process completed with exit code 1.\n```\n\nA few kilobytes of noisy log become a dozen lines an LLM can reason over. Each document also keeps the run's URL, so an answer can always be traced back to GitHub.\n\nI pointed the connector at the 15 latest completed runs of `topoteretes/cognee-community` on 28 October 2026. The LLM was an Azure OpenAI deployment, and embeddings came from `text-embedding-3-large`. Everything below is real output.\n\n**1. Fetch the runs.** The connector pulled 15 runs, their jobs, and the logs of the failed jobs in **8.2 s**. Only 3 had actually failed. 10 were waiting for approval.\n\n**2. Build the memory.** One call, `cognee.remember()`, stored the 15 documents and built the knowledge graph and embeddings in **105.3 s**.\n\n```\nawait cognee.remember(\n    github_actions_source(\n        repos=[\"topoteretes/cognee-community\"],\n        max_runs_per_repo=15,\n        include_failed_logs=True,\n    ),\n    dataset_name=\"github-actions-blog\",\n)\n```\n\n**3. Ask questions.** Here are cognee's answers, unedited:\n\n**Q: Which workflows failed recently, and why?**\n\nOn 2026-10-28, three workflows failed:\n\n**Clay connector tests (#1):** Python 3.11 and 3.12 both failed `test_check_active_cancellation`; extraction raised `ResourceExtractionError: Sync cancelled by host`.\n**Ruff Lint and Format (#1098):** Clay code had a nested-if lint issue, unescaped regex patterns in tests, and a formatting issue.\n**Scheduled \"Bump cognee in the database adapters\" (#22):** failed because the `COGNEE_BUMP_PR_TOKEN` secret was not configured.\nThe Klaviyo connector runs were marked `action_required`, not failed.\n\n**Q: How do I fix the failing bump workflow?**\n\nAdd the `COGNEE_BUMP_PR_TOKEN` secret to the repository's GitHub Actions secrets (see the workflow header for details), then rerun the scheduled workflow. It failed because that required secret wasn't configured.\n\n**Q: Which workflows are waiting for approval?**\n\nThe workflows marked `action_required` (waiting for approval) are:\n\n- Klaviyo connector tests\n- Vercel connector tests\n- Prometheus connector\n- Ruff Lint and Format (runs #1094, #1095, #1101, and #1102)\n\nThe first two answers are what I wanted. Each gives the cause and the fix, down to the failing test name and the missing secret, with no log open. The first even points out that the Klaviyo runs were waiting, not failing.\n\nThe third answer is close but not complete. It lists 7 of the 10 waiting runs and misses Ruff runs #1096, #1097 and #1100. Graph completion answers from the context it retrieves, so it isn't an exhaustive query. For an exact list, the table from step 1 is still the right tool.\n\nIt helps most when you need the story across many runs, not the details of one.\n\n| Who | Question they can ask | What used to happen | \n|---|---|---|\n| Maintainer | \"Which PRs are stuck waiting for workflow approval?\" | Scan the Actions tab run by run | \n| Contributor | \"Why did the tests fail on my branch?\" | Open the job, expand the step, scroll the raw log | \n| On-call / release owner | \"Is this the same error that broke `main` last week?\" | Rely on memory, or search Slack | \n| Team lead | \"Summarize recent CI activity in this repo\" | Build a dashboard, or just not know | \n| AI coding agent | Reads the failure summary before proposing a fix | Fed a whole log, or nothing | \n\nBecause it's a plain cognee dataset, CI history can sit next to your other data, such as docs, issues or code. One question can then draw on all of them.\n\nYou need Python 3.11 to 3.13, a GitHub token, and an LLM key for cognee. For a public repo, a token with no extra scopes is enough. For a private repo it needs `repo` or `actions:read`.\n\n```\ngit clone https://github.com/topoteretes/cognee-community\ncd cognee-community/packages/connector/github-actions\nuv sync\nexport GITHUB_TOKEN=$(gh auth token)\nexport LLM_API_KEY=...   # or put cognee's settings in a .env file\npython\nimport asyncio\nimport cognee\nfrom cognee_community_connector_github_actions import github_actions_source\n\nasync def main():\n    source = github_actions_source(\n        repos=[\"owner/repo\"],\n        max_runs_per_repo=50,\n        include_failed_logs=True,\n    )\n    await cognee.remember(source, dataset_name=\"github-actions\")\n    print(await cognee.search(\n        query_text=\"Which workflows failed recently, and why?\",\n        query_type=cognee.SearchType.GRAPH_COMPLETION,\n        datasets=[\"github-actions\"],\n    ))\n\nasyncio.run(main())\n```\n\nUseful options:\n\n`workflow` and `branch` narrow the runs to one workflow file or one branch.`created_after` limits how far back to go.`log_tail_lines` controls how much of each failed step's output is kept (default 50).\nRun it again later to sync. Runs that have dropped out of the window are removed from memory, so the dataset tracks your current CI.\n\nIf the PR is still open when you read this, use the branch from [PR #383](https://github.com/topoteretes/cognee-community/pull/383).\n\n`##[error]` lines and the output just before them made the documents small and the answers sharper.\nIf you try it, I'd like to hear what you ask your CI. Feedback and issues on the PR are welcome.\n\n**Links:** [cognee](https://github.com/topoteretes/cognee) · [cognee-community](https://github.com/topoteretes/cognee-community) · [PR #383](https://github.com/topoteretes/cognee-community/pull/383)", "url": "https://wpnews.pro/news/asking-my-ci-why-did-that-fail-github-actions-cognee", "canonical_source": "https://dev.to/blews/asking-my-ci-why-did-that-fail-github-actions-cognee-2b9a", "published_at": "2026-10-28 13:03:46+00:00", "updated_at": "2026-10-28 13:11:31.036972+00:00", "lang": "en", "topics": ["ai-agents", "ai-tools", "developer-tools", "large-language-models", "ai-infrastructure"], "entities": ["cognee", "cognee-community", "GitHub Actions", "topoteretes", "dlt", "Azure OpenAI", "text-embedding-3-large"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/asking-my-ci-why-did-that-fail-github-actions-cognee", "markdown": "https://wpnews.pro/news/asking-my-ci-why-did-that-fail-github-actions-cognee.md", "text": "https://wpnews.pro/news/asking-my-ci-why-did-that-fail-github-actions-cognee.txt", "jsonld": "https://wpnews.pro/news/asking-my-ci-why-did-that-fail-github-actions-cognee.jsonld"}}