# A cross-repository coding agent using the GitHits API

> Source: <https://githits.com/blog/cross-repo-agent-githits-api/>
> Published: 2026-10-22 00:00:00+00:00

[Back to blog](https://githits.com/blog/)

October 22, 2026 · 9 min read

# A cross-repository coding agent using the GitHits API

Cloning every related repo was the baseline on 42 tasks. GitHits matched that accuracy (79–94%) with none of the per-sandbox setup cost.

We ran four models three times on 42 tasks that cross repository boundaries. With no access to the other repositories, the models passed 9–53% of tasks on average. With read access through GitHits, they passed 79–94%. With every related repository cloned into the sandbox, they also passed 79–94%.

Pooled over the four models, both setups passed 90% of tasks. For each model, the GitHits and cloned results were within the spread between repeats. DeepSeek V4.1 Flash with GitHits passed 92% of tasks at a mean model cost of $2.09 per 1,000 tasks. Codex with GitHits passed 94%, at an estimated $23.02.

## The task

A common setup for a coding agent working on a pull request is a sandbox with one repository checked out. The code it changes is used by other repositories, and the code it calls lives in them. With no access to them, the agent has to answer questions about the other repositories from memory. As a baseline, we also ran every task with all the related repositories cloned into the sandbox.

The GitHits API reads source from a public GitHub repository at any ref: a release tag, a branch, or a full commit SHA, including commits that are not in any release yet. An agent can list directories, search with a regular expression, read files and diff two refs, across many repositories, without a checkout. For example, one task used pytest’s unreleased main branch, pinned by commit.

We gave the agent three kinds of tasks:

- **Update code for a change in another repository** (10 tasks). Each one is a real fix commit. For example, a team bumps its botocore pin from 1.43.98 to 1.43.101, and aiobotocore needs the matching change in its S3 region-redirect code. The agent has to find what botocore changed between those versions and write the same change in aiobotocore.
- **Will my change break other repositories?** (16 tasks). For example, a developer is about to rename`aiobotocore.session.AioSession` to`AsyncSession` , with no alias, and asks whether that breaks any other repository at its pinned version, and where. The answer is yes:`s3fs/core.py` constructs`AioSession` . Two of the 16 changes break nothing.
- **Which files in other repositories use this symbol?** (16 tasks). For example, which production files use`run_sync` from`jupyter_core.utils` . Some tasks contain traps:`jupyter_server` also calls a`run_sync` , but it imports it from anyio, not from jupyter_core.

## How the agent works

The agent runs a loop of up to 10 turns. In each turn, the model asks for up to 4 tool calls or gives its final answer. It has three tools for its own repository: list, search and read. It has four tools for the other repositories:

1. **List** a directory, with`/v1/list` .
2. **Search** with a regular expression across one or more repositories at their pinned refs, with`/v1/grep` .
3. **Read** lines of a file, with`/v1/read` .
4. **Diff** a repository between two refs, with`/v1/code/diff` . This is how the agent sees what changed upstream between two releases.

Each repository is addressed as a target such as `github:boto/botocore@1.43.101`. The agent could also call GitHits semantic search; nearly all of its calls to other repositories were searches and reads.

In the cloned setup, the same four tools run `git ls-tree`, `git grep`, `git show` and `git diff` on local clones. The prompts, the turn limits and the tools for the agent’s own repository are identical across setups.

## How we tested it

We used 29 public Python repositories in five groups of related projects: pytest, pluggy and eight pytest plugins; botocore, boto3, s3transfer, aiobotocore and s3fs; httpcore, httpx, starlette, FastAPI, respx and uvicorn; five Jupyter repositories; and kombu, celery and py-amqp. Each repository is pinned to a commit or release tag.

For the update tasks, the agent starts at the parent of the real fix commit. We graded each edit with checks that the real fix passes and its parent fails, and the edited file must still parse. These checks match patterns in the code. They are not test runs. For the other two task types, we built the answer key with `git grep` on the pinned snapshots and checked it by hand. An impact answer passes if the verdict is right and its list of affected files scores at least 0.6 on F1, a measure that combines missed files and wrongly listed files. A usage answer passes if its file list reaches the same score.

We ran Gemma 4 31B at its default settings, DeepSeek V4.1 Flash and Kimi K2.6 with reasoning off, and Codex `gpt-6.1-sol` at low reasoning effort. Each model ran every task three times in each setup, 1,512 runs in total. All three setups ran in the same afternoon, with runs in random order within each repeat, so they shared the same API and model-provider conditions.

## Tasks passed

Share of the 42 tasks passed, as the mean of three runs, with the lowest and highest run in brackets:

| Model | No access to other repos | GitHits API | All repos cloned | 
|---|---|---|---|
| Gemma 4 31B | 9% (5–12) | 79% (74–83) | 79% (71–83) | 
| DeepSeek V4.1 Flash | 16% (14–19) | 92% (90–93) | 94% (93–95) | 
| Kimi K2.6 | 19% (17–21) | 94% (93–95) | 94% (93–95) | 
| Codex gpt-6.1-sol | 53% (48–60) | 94% (93–95) | 94% (93–95) | 

One task is about 2.4 points of a single run. The largest gap between GitHits and clones was DeepSeek’s, at one task on average, in favor of clones. Gemma’s results varied the most between repeats: in individual repeats, GitHits was 5 tasks ahead in one and 4 tasks behind in another. With three runs per task, the data does not show either setup ahead.

Without access to the other repositories, Codex passed 53%, well ahead of the open models. These are popular public projects, and Codex answered many tasks from what it remembered.

## By task type

The impact and usage questions are mostly a matter of finding every place a symbol is used. With read access, through GitHits or clones, models answered 95–97% of them correctly. With no access, they answered 23–29% correctly.

The update tasks were harder in every setup: 68% through GitHits, 72% with clones and 19% with no access. This is the one task type where clones came out ahead for every model, by one or two of each model’s 30 runs. With 10 tasks, the pooled results of the three repeats overlap (62–70% through GitHits, 62–78% with clones), so we can’t separate that difference from run-to-run variation. One task asked the agent to adapt pytest-xdist to a change on pytest’s unreleased main branch, pinned by commit. It passed in 9 of 12 runs through GitHits and 9 of 12 with clones.

## Cost and setup

Mean model cost per 1,000 tasks, for the two setups with access to the other repositories:

| Model | GitHits API | All repos cloned | 
|---|---|---|
| Gemma 4 31B | $2.07 | $2.05 | 
| DeepSeek V4.1 Flash | $2.09 | $1.83 | 
| Kimi K2.6 | $16.11 | $13.43 | 
| Codex gpt-6.1-sol | $23.02 | $23.19 | 

The costs are model spend reported by OpenRouter. GitHits API fees are not included. Codex ran through a ChatGPT subscription, so its figures are estimates at OpenRouter list price, based on our own transcript sizes and excluding Codex’s own harness prompt. Model cost was about the same for Gemma and Codex, and higher with GitHits for DeepSeek and Kimi.

We cloned every related repository into the sandbox only as a baseline, to measure the difference. The local clones of the 29 repositories took about 435 MB of disk. That setup is repeated in every sandbox that needs those repositories, so it grows with the number of sandboxes and repositories. GitHits read access reached the same task accuracy, within the spread between repeats, with none of that per-sandbox setup: no clones of the other repositories, and no disk space for them.

## Use the GitHits API

This agent uses four endpoints: `/v1/list`, `/v1/grep` and `/v1/read` to browse, search and read other repositories at a pinned ref, and `/v1/code/diff` to compare two refs of a repository. The REST API is available for evaluation, with the base URL `https://api.githits.dev/v1`; the v1 contract is not yet finalized.

Before renaming `aiobotocore.session.AioSession`, an agent can check the other repositories at their pinned commits in one request:

```
curl -s https://api.githits.dev/v1/grep \
  -H "Authorization: Bearer $GITHITS_API_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{
    "pattern": "AioSession\\(",
    "targets": [
      {"target": "github:fsspec/s3fs@17a38ae9dc6ff4df64a338d69e26712c152a36ff",
       "corpus": "source", "path_selectors": [{"kind": "glob", "value": "s3fs/*.py"}]},
      {"target": "github:boto/boto3@65340debb6393c434f864f86245c46aad08ab3fe",
       "corpus": "source", "path_selectors": [{"kind": "glob", "value": "boto3/**/*.py"}]}
    ]
  }'
{
  "hits": [
    {"target_index": 0, "file_path": "s3fs/core.py", "line": 325,
     "line_content": "For example: aiobotocore.session.AioSession(profile='test_user')"},
    {"target_index": 0, "file_path": "s3fs/core.py", "line": 665,
     "line_content": "self.session = aiobotocore.session.AioSession(**self.kwargs)"}
  ],
  "targets": [
    {"target_index": 0, "target": "github:fsspec/s3fs@17a38ae9dc6ff4df64a338d69e26712c152a36ff", "traversal": "complete"},
    {"target_index": 1, "target": "github:boto/boto3@65340debb6393c434f864f86245c46aad08ab3fe", "traversal": "complete"}
  ],
  "traversal": "complete",
  "total_matches": 2
}
```

Both targets were searched completely. Line 665 of `s3fs/core.py` is the constructor call the rename would break; line 325 is the same name in a docstring. boto3 has no matches. Each hit also carries a `read` action to pass to `/v1/read` for the surrounding lines.

- **Get access:** create an API token in the token settings of your[GitHits account](https://app.githits.com/) and send it in an`Authorization: Bearer` header. The[authentication guide](https://docs.githits.com/api/authentication) covers token storage for CI and runtime use.
- **First request:** the[API quickstart](https://docs.githits.com/api/quickstart) shows how to call an endpoint.
- **Reference:** the[API overview](https://docs.githits.com/api/overview) links to the v1 reference for every endpoint, and the[API playground](https://api.githits.dev/) lets you send test requests with your token.

If you’re a software factory interested in using our API, we’d love for you to [book a call with us](https://calendly.com/d/dvxq-7dn-msq/meeting-with-githits-select-15-or-30-minutes) to discuss what it would look like.