# I pulled 480 real AGENTS.md/CLAUDE.md files from 272 repos — and a regex bug taught me not to trust my own heuristics

> Source: <https://dev.to/sattva2020/i-pulled-480-real-agentsmdclaudemd-files-from-272-repos-and-a-regex-bug-taught-me-not-to-trust-1ocb>
> Published: 2026-08-12 19:14:23+00:00

A corpus of real AGENTS.md / CLAUDE.md files, the instruction files coding agents like Claude Code, Copilot, and friends read before touching a repo, pulled verbatim from 272 production repos (React, Grafana, LangChain, Home Assistant, and 268 others). Not a curated list of links: the actual files, stored in full wherever the license permits it, greppable in one tree.

Repo: [https://github.com/sattva2020/agents-md-in-the-wild](https://github.com/sattva2020/agents-md-in-the-wild)

One analysis pass tries to detect literal directory-tree diagrams, the kind with box-drawing characters or ASCII branch markers. First version was naive: look for tree-shaped characters on a line, count matches. Ran it across the corpus and got 69% of files "containing a directory map." That felt way too high, a directory tree in an instructions file is a specific, deliberate thing, not something 7 in 10 projects bother writing.

Turned out the regex was matching markdown tables. A row like `| src/ | entry point |`

has a pipe and something path-shaped on the left, and my "line looks like a tree branch" check didn't care what came after the first slash. Fix: require a run of 3+ consecutive tree-shaped lines, and explicitly reject lines that parse as table rows before counting them. Dropped the number from 69% to 17%, which matches manual spot-checks.

Same story with the "mentions secrets" heuristic. First cut matched bare "token," which turned out to catch files talking about LLM context-window tokens, not API tokens. Had to tighten the pattern to require "api," "access," or similar nearby.

Lesson, twice over: a keyword or shape that looks specific to you is rarely as specific as it looks once you run it against real-world text at scale. 480 files was enough to expose both bugs, I doubt either would've shown up against the 5-10 examples I originally tested with.

Storage is tiered by SPDX license: files under a curated allowlist of about 26 redistributable licenses (MIT, Apache-2.0, BSD variants, GPL/LGPL/AGPL, CC0, etc.) are stored verbatim with provenance. Everything else, including repos with no declared license, gets metadata-only storage: headings and structure, no body text. Default is "all rights reserved" unless the license says otherwise; unlicensed files still get counted in the structural analysis, they just don't get their text redistributed.

It's meant to refresh weekly via a GitHub Action, full disclosure, that's currently stuck behind a billing issue on my GitHub account, so treat "weekly" as aspirational until I sort that out.

Genuinely curious what's missing, or whether there's a pattern signal worth adding. Repo again: [https://github.com/sattva2020/agents-md-in-the-wild](https://github.com/sattva2020/agents-md-in-the-wild)
