A faster dependency check using the GitHits API

A dependency-check agent built on the GitHits API by an unnamed team finished checks in a median of 4.6 seconds at a mean model cost of $0.00015 per check in an October 14, 2026 session, with accuracy comparable to Codex's. The roughly 200-line Python agent fetches latest package versions from GitHits, sends one batched upgrade-review request, filters GitHub severity labels from OSV.dev for high and critical advisories, merges alias-linked records, and uses a single Gemma 4 31B model call to write the report. The test compared Codex alone against the GitHits agent across 35 pinned packages in four sets (two npm, one PyPI, one mixed) with 20 runs per setup and 80 runs total.

Back to blog https://githits.com/blog/ October 13, 2026 · 9 min read A faster dependency check using the GitHits API We compared Codex with a dependency check that retrieves package data through the GitHits API and uses one model call to write the report. We built a dependency-check agent using the GitHits API. Code pulls the facts from the index, and one model call writes the report. In one session on October 14, 2026, the GitHits agent with Gemma 4 31B finished in a median of 4.6 seconds, at a mean model cost of $0.00015 per check. Its accuracy scores were comparable to Codex’s. A software factory changes dependencies all the time. An agent adds a package to finish a feature, and another bumps a version to fix a build. Each change raises the same questions. Is the upgrade a major version jump? Which known high and critical vulnerabilities does it fix? Does it introduce new ones? You can ask a coding agent to do this. It looks up the latest versions, finds the advisories, matches them to version ranges, and checks their severity. Repeating that investigation after every dependency change takes time and uses tokens. We wanted to know whether a narrower agent could do the same check fast and cheaply enough to run every time. The question Every run in this experiment answered the same question for a set of pinned dependencies: Our app is upgrading these dependencies to their latest versions: … For each: is it a major/breaking upgrade, how many high/critical advisories affect the current version, how many are fixed by the target, and does the target introduce any new high/critical advisories? Answer as a table. We score “breaking” as a major version change. Checking whether an upgrade breaks the application’s code is a separate task. How the agent works The agent is about 200 lines of Python. It runs four steps before calling the model: 1. Fetch the latest published version of each package from GitHits. 2. Send one batched upgrade-review request comparing each current version with its target. The response includes security advisories for both versions. 3. Read GitHub’s reviewed severity labels from OSV.dev and select the high and critical advisories. 4. Group advisory records linked by aliases, then count which affect the current version, which the upgrade fixes, and which affect the target but not the current version. The model then writes the report from the question and the computed results, with no tools available. Here is a condensed version of the code: python def depcheck pkgs, model : 1. latest version of each package, from the GitHits index latest = get f"/v1/packages/{reg}/{name}?fields=package" "package" "latest version" for reg, name, in pkgs 2. one batched upgrade review: advisories for current and latest batch = {"registry": reg, "name": name, "current version": cur, "target version": tgt} for reg, name, cur , tgt in zip pkgs, latest reviews = post "/v1/package-upgrade-reviews?fields=security.direct", {"packages": batch} 3 + 4. GitHub severity labels from OSV.dev, aliases merged, counted rows = count high critical pkgs, latest, reviews "reviews" , github label 5. the only model call: turn the computed table into the report return model prompt question pkgs , rows If security data is unavailable, the agent retries once and then marks the row as unknown rather than returning a count of zero. How we tested it We compared Codex on its own with our GitHits agent, using two models for each. We tested 35 packages pinned to older versions, divided into four sets: - Two sets of ten npm packages. - One set of ten PyPI packages. - One mixed set: moment, node-fetch, and socket.io from npm, plus aiohttp and tornado from PyPI. Each of the four setups ran every set five times in shuffled order. That gave us 20 runs per setup and 80 runs in the October 14 session. The Codex-only setups had the GitHits MCP server disabled and no GitHits API key in their environment. We prepared the answer key from npm, PyPI, and OSV.dev before the runs. A package’s answer counted as correct when all four fields matched: major version change, advisories affecting the current version, advisories fixed, and advisories new in the target. Each setup had 175 answers to score: 35 packages checked five times. Time, cost, and token use We first compared both approaches using gpt-6.1-sol. Codex looked up versions, advisories, affected version ranges, and severity labels during the run. The GitHits agent fetched and processed that data in code before calling the model. Median time fell from 61.2 to 12.8 seconds, about 4.8 times faster. Mean model cost fell from $0.133 to $0.0093, about 14 times lower. We also tested Codex with gpt-6-astra at high reasoning effort and with web search enabled. For the fourth setup, we replaced gpt-6.1-sol in the GitHits agent with Gemma 4 31B through OpenRouter. Times are medians; costs and token counts are means per run. | Setup | Median time | Mean model cost | Total tokens | Input of which cached | Output | |---|---|---|---|---|---| | Codex alone, gpt-6.1-sol | 61.2 s | $0.133 | 312.1k | 309.6k 268.8k | 2.5k | | Codex alone, gpt-6-astra, high effort, web search enabled | 100.2 s | $0.897 | 306.7k | 303.0k 257.7k | 3.7k | | GitHits agent + Codex gpt-6.1-sol | 12.8 s | $0.0093 | 15.0k | 14.7k 12.3k | 0.33k | | GitHits agent + Gemma 4 31B | 4.6 s | $0.00015 | 0.71k | 0.53k 0.13k | 0.18k | The Gemma setup completed the check in a median of 4.6 seconds, about 22 times faster than Codex with gpt-6-astra. The Codex runs used a ChatGPT subscription, so we calculated model costs from logged token usage and list prices. Gemma’s cost came from OpenRouter. Although web search was enabled for gpt-6-astra, its logs recorded no web calls. It used shell tools instead, averaging 7.8 calls per run. Why we scored the answers three ways Two advisory records can describe the same vulnerability or disagree about which versions it affects. Whether we combine those records, and which version ranges we use, changes the expected answer. We scored the same answers against three rules: - Group linked records “merged” . Combine records linked by aliases and count them once. A GitHub Security Advisory identifier GHSA and a CVE identifier can refer to the same vulnerability. - Count advisory IDs separately “strict” . Count distinct advisory IDs separately, even when they share aliases. For lodash, two GHSA IDs counted as two under this rule and one under the merged rule. - Use GitHub’s affected versions “GitHub ranges” . Use GitHub’s reviewed version ranges to decide whether an advisory affects a package version. Those ranges can differ from the matches returned by OSV. The scores below show correct package answers out of 175 under each rule. | Setup | Group linked records | Count IDs separately | Use GitHub’s affected versions | |---|---|---|---| | Codex alone, gpt-6.1-sol | 158/175 | 163/175 | 168/175 | | Codex alone, gpt-6-astra, high effort, web search enabled | 160/175 | 165/175 | 170/175 | | GitHits agent + Codex gpt-6.1-sol | 175/175 | 170/175 | 165/175 | | GitHits agent + Gemma 4 31B | 175/175 | 170/175 | 165/175 | Both GitHits setups scored 175/175 under the merged rule, which is also the rule their code uses. They led under the merged and separate-ID rules; Codex with gpt-6-astra led under GitHub’s affected version ranges. For cryptography and pillow, OSV matched an extra record that GitHub’s reviewed version ranges did not include. Along with lodash, these cases explain most of the score differences. Codex alone also made two isolated errors, one involving mongoose and one involving nuxt. Index for knowledge, inference for reasoning We want models to spend their time on decisions that require reasoning. Code can look up versions, fetch advisories, and count them. Moving those operations out of the agent loop reduced the number of model calls and tokens needed for this check. Here, the model only had to write the report, and Gemma was sufficient. Deciding which upgrades to make and how to adapt the application takes more reasoning. That is where we want the coding agent to focus. GitHits gives us version-aware access to the code, documentation, and package data needed for that work. We can let a coding agent choose its queries through MCP or the CLI, or write a fixed sequence of API calls for a recurring task like this one. Where this fits in a factory We designed this check to run after an agent changes a manifest or lockfile. The 4.6-second median makes it practical to pass the advisory report to the next agent step, so it can address a flagged dependency while working on the change. Build it with the GitHits API The upgrade-review endpoint accepts multiple packages in one request. Here is a comparison of two pinned dependencies and their target versions: curl -X POST https://api.githits.dev/v1/package-upgrade-reviews \ -H "Authorization: Bearer $GITHITS API TOKEN" \ -H "Content-Type: application/json" \ -d '{ "packages": { "registry": "npm", "name": "lodash", "current version": "4.17.20", "target version": "4.18.1" }, { "registry": "pypi", "name": "urllib3", "current version": "1.26.4", "target version": "2.8.0" } }' See the API documentation https://docs.githits.com/api/overview for the full set of endpoints. If you’re a software factory interested in using our API, we’d love for you to book a call with us https://calendly.com/d/dvxq-7dn-msq/meeting-with-githits-select-15-or-30-minutes to discuss what it would look like.