Teaching Cognee to Remember Your Background Jobs: Building the Trigger.dev Connector

A developer built a Trigger.dev connector for Cognee, the open-source memory layer for AI agents, that syncs background job history — schedules, errors, failed runs and deployments — into searchable memory via a triggerdev_source(...) function returning a dlt resource passed to cognee.remember(). The connector deliberately omits run payloads, outputs, metadata, cost data and Git remote URLs to limit sensitive data exposure, excludes volatile fields so unchanged records produce identical documents, and only deletes a schedule after a confirmed HTTP 404 rather than treating missing listing data as deletion. It also stops cleanly on HTTP 429s or revoked API keys instead of retrying, and retains historical records within a configurable lookback_days window defaulting to seven days.

Pull Request: topoteretes/cognee-community 464 https://github.com/topoteretes/cognee-community/pull/464 Related Issue: topoteretes/cognee 6197 https://github.com/topoteretes/cognee/issues/6197 When a background job fails at 3 a.m., finding the reason is rarely straightforward. The answer is usually scattered across schedule configurations, error logs, stack traces, and the deployment that introduced the bug. I wanted an AI agent to see all of that information in one place. That led me to build a Trigger.dev connector for Cognee https://github.com/topoteretes/cognee , a system that turns background job history into searchable memory. The goal was simple: help AI agents understand what happened, identify patterns in recurring failures, and connect those failures to recent deployments. The connector exposes a triggerdev source ... function that returns a dlt resource, which can then be passed to cognee.remember . With a single integration, it syncs a Trigger.dev environment into Cognee's memory, collecting four types of records: Each record becomes a document associated with the node set triggerdev:task:<task id . This allows an AI agent to answer questions such as: Why does the nightly invoice job keep failing, and what changed in the latest deployment? Instead of searching through multiple dashboards and logs, the agent can retrieve relevant information while staying within the context of a specific task. Building the connector was not just about retrieving API data. I also needed to make synchronization reliable, protect sensitive information, and ensure that failures would not accidentally destroy existing memory. Background job data can contain sensitive customer information, so the connector deliberately avoids storing run payloads, outputs, or metadata. It also excludes cost information and the deployment's Git remoteUrl , which could contain a token. The API key is never included in logs or error messages, and the api url must be an HTTPS origin. The principle was straightforward: store the information needed for debugging without unnecessarily storing sensitive data. Repeated synchronization should not create unnecessary updates. To achieve this, I excluded volatile fields such as next-run timestamps, changing error counts, and the duration of an active run. As a result, an unchanged record produces an identical document. When the connector runs again without any meaningful changes, Cognee does not need to process the same information repeatedly. During my live test, the connector correctly skipped unchanged records. One of the most important requirements was making sure that missing data would not automatically be interpreted as deleted data. Schedules are retrieved through a complete listing. If a schedule is missing from that listing, the connector checks it individually by ID. Only a confirmed HTTP 404 results in removing that schedule from memory. If the listing fails or returns an incomplete result, the synchronization fails instead of deleting existing records. This prevents temporary API failures from turning into permanent data loss. Errors, failed runs, and deployments represent historical events. They should not disappear simply because they are no longer recent. The connector retrieves records within a configurable lookback days window, which defaults to seven days. Older historical records remain in memory rather than being deleted during synchronization. This helps preserve the context an AI agent may need to identify recurring problems over time. The connector also needs to handle API limitations and interruptions without making things worse. When it encounters HTTP 429 responses, exhausts its request budget, or discovers that an API key has been revoked during execution, it stops cleanly. It does not sleep and repeatedly retry after hitting a rate limit. Instead, the next synchronization can resume the work. This makes the connector more predictable and avoids wasting requests. My first implementation was wrong, and the reason it failed taught me an important lesson. It called API endpoints that did not exist. Even worse, it treated a failed listing as an empty listing, which could have removed every stored task and schedule from memory. The problem was not immediately obvious because my tests replaced the entire HTTP client with a fake implementation. The tests passed, but they were testing my assumptions about the API rather than verifying that the implementation actually worked with it. I had effectively built a system that confirmed its own assumptions. I rebuilt the connector around the documented Trigger.dev management API and changed the testing strategy. The updated tests now: I also tested the integration through cognee.add to verify the behavior beyond the API client itself. The result was 83 passing tests , with both ruff check and ruff format --check completing successfully. After rebuilding the implementation and strengthening the tests, I tested it against a real Trigger.dev Cloud development environment running SDK version 4.8.1. The environment contained two schedules and three deliberately failing runs. The live test confirmed that the listing, filtering, and pagination parameters worked as expected. I also verified schedule deletion: after a schedule was deleted, the next synchronization rechecked it, received a 404 response, and removed it from memory. However, the live test also uncovered a subtle issue that was not apparent from the documentation. The endpoint GET /api/v3/runs/{id} returns error information at the top level of the response alongside attemptCount . The attempts field, meanwhile, can be empty. My original implementation relied on attempts to retrieve error information. Consequently, failed-run documents were being created without their errors. The fix was to read the top-level error first and fall back to the attempts data when necessary. This was a good example of why testing against the real API matters. A response can look perfectly reasonable in a mocked test while behaving differently in production. While addressing the error-handling issue, I also made two improvements. First, very short run durations are now displayed in milliseconds rather than being rounded to 0.0s . Second, stack traces are limited to a maximum of five frames from the task's own code, with shortened file paths. The original stack traces contained mostly runtime frames and exposed my local directory paths. Filtering them made the stored error information more useful and reduced unnecessary path disclosure. Deployments were not tested against the live development environment because it did not contain any deployments. That part of the connector is currently covered by the HTTP-level fake tests, rather than a live deployment test. I wanted to be transparent about this limitation instead of treating the entire integration as fully validated against a live environment. I built the connector with Claude Code, and the commits include a Co-Authored-By trailer. I reviewed the implementation and ran the live tests myself. The biggest lesson for me was that a passing test suite is only as reliable as the assumptions behind it . Mocking is useful, but a mock that reproduces your own incorrect understanding of an API can make a broken implementation look correct. Testing against the real service early helped me discover issues that would have otherwise remained hidden, especially around response structures, deletion behavior, and error handling. For integrations involving external APIs, I now see realistic HTTP-level tests and live validation as complementary parts of the development process, not optional extras. To run the connector's tests, navigate to the connector directory and execute: cd packages/connector/triggerdev uv run --with pytest --with pytest-asyncio pytest tests/ The pull request is currently open for review: View the Trigger.dev Connector PR — cognee-community 464 https://github.com/topoteretes/cognee-community/pull/464 The related issue is available here: topoteretes/cognee 6197 https://github.com/topoteretes/cognee/issues/6197 . This project started with a practical question: what if an AI agent could understand the history behind a failed background job instead of looking at an isolated error message? The connector brings schedules, failures, run details, and deployment information into a shared memory layer, making it possible to connect events that would otherwise remain scattered across different parts of a system. Building it also reinforced an important engineering lesson for me: correctness is not just about writing code that passes tests. It is about verifying assumptions, handling failure safely, protecting sensitive data, and checking that the implementation behaves as expected against the real service. That is what made this project valuable to me—not just connecting Trigger.dev to Cognee, but learning how to build an integration that can be trusted with real operational data.