# We benchmarked a security detector against 500k lines of Go. It found a real bug.

> Source: <https://dev.to/stopthatslop/we-benchmarked-a-security-detector-against-500k-lines-of-go-it-found-a-real-bug-3pj2>
> Published: 2026-08-24 19:05:48+00:00

We're building deslop — rule packs that stop coding agents from

re-introducing bugs that were already fixed. One half of that is

detection: if a rule claims agents keep writing a certain kind of bug,

there should be a detector that catches it, and that detector should be

measured before it's allowed to fail anyone's build.

So we benchmarked one of our security detectors against roughly 503,000

lines of Go across eight well-known open-source projects.

It produced exactly one finding. The finding was real.

The rule is simple to state: secrets and tokens must be compared in

constant time. A plain string equality check (`==`

) against a secret leaks

information through timing — on shared infrastructure, an attacker who can

measure how long a comparison takes can, in principle, recover the secret

piece by piece. The standard fix is `crypto/subtle.ConstantTimeCompare`

.

Writing a detector that fires on every `==`

next to a variable named

`token`

is easy and useless. Real code compares all kinds of things to all

kinds of things, and most of them are fine. Our detector only fires when

an operand has a **traceable credential source** — a value that came from

`r.Header.Get(...)`

, `os.Getenv(...)`

, or an assignment chained from one

of those. It skips test files and vendored code. It resolves named helper

functions one level deep, because `CheckOrigin: isValidOrigin`

and

`CheckOrigin: func(...) { return true }`

are the same bug wearing

different clothes.

Eight repositories, pinned commits, ~503k lines of Go (counted as

non-blank lines, tests and vendor excluded):

| Repo | What it is | LOC | Findings |
|---|---|---|---|
| caddy | web server / reverse proxy | 104,660 | 0 |
| lazygit | TUI git client | 141,754 | 0 |
| restic | backup / storage infra | 88,525 | 0 |
| goreleaser | release automation | 92,357 | 1 |
| fzf | CLI fuzzy finder | 33,306 | 0 |
| task | task runner | 23,162 | 0 |
| chi | HTTP router | 12,082 | 0 |
| viper | configuration | 7,194 | 0 |

Seven clean repos matter as much as the hit. A detector that cries wolf on

`if key == "border-native"`

(fzf compares lexer tokens, not credentials)

or `if setting.Key == "vcs.modified"`

(task dispatches on config keys)

is a detector nobody will leave enabled. Our first version produced

exactly those false positives — twenty-two of them. We hardened the

operand analysis (require the credential source, not the name) and

re-ran. One finding.

`goreleaser`

, `internal/client/gitlab.go`

, in `checkUseJobToken`

:

```
ciToken := os.Getenv("CI_JOB_TOKEN")
if ciToken == "" {
    return false
}
// ...
if ctx.Config.GitLabURLs.UseJobToken {
    return token == ciToken
}
```

`token`

is the user-configured GitLab API token. `ciToken`

is the CI job

token from the environment. The comparison decides which API client to

use — and it's done with plain string equality.

To be clear about severity: this is **hardening, not a zero-day**.

Exploiting a timing side channel over a network on a comparison like this

is largely theoretical, and `ConstantTimeCompare`

still leaks length

(though `CI_JOB_TOKEN`

lengths are fixed per environment). We said exactly

that in the PR. The maintainers' response treated it as a reasonable

correctness improvement — which is the right way to think about it.

```
import "crypto/subtle"
// ...
return subtle.ConstantTimeCompare([]byte(token), []byte(ciToken)) == 1
```

Two hunks, one line changed. The PR is here:

[goreleaser#6813](https://github.com/goreleaser/goreleaser/pull/6813).

`token == "giteatoken"`

in
goreleaser (a placeholder comparison) looks identical to a credential
check if you match on identifiers. Trace where the value came from
instead.deslop is open source: [github.com/Amaresh/deslop](https://github.com/Amaresh/deslop).

The detector that found this, and the benchmark that measured it, are part

of the repo.

*Found a false positive in your own Go codebase? That's data — the issue
tracker is open.*