16 repos. About 1.46 million GitHub stars between them. Zero A grades.
AGENTS.md
has quietly become the cross-tool standard for repo-level agent instructions — a single file that Cursor, Codex, Copilot, Claude Code and a growing list of others read on startup. So I went looking for how the AI-agent ecosystem itself uses the convention it created. I swept 36 of the best-known AI coding-agent and agent-framework repositories, found 16 with at least 20k stars shipping a root AGENTS.md
, and scored every file with a deterministic engine — no LLM judge, same file same score on every machine, reproducible against the exact commit SHAs at the bottom of this post.
The result: mean 70.0, median grade C, not a single A. The repo whose team popularized the convention ranks 13th of 16. And the best AGENTS.md
I could find anywhere belongs to a library with 170 stars.
This is the same scorer I used to grade my own merged MCP contribution and the one behind the case for static instruction-file security. It is open source, MIT, stdlib-only: the rubric is a dict you can read, not a hidden prompt.
The ranking #
The set is every well-known AI coding-agent or agent-framework project at or above 20k stars that ships a root AGENTS.md
, each scored at its HEAD on 2026-07-07 with released schliff==8.5.0
. The AGENTS.md rubric is three dimensions: structure (weight 0.4), operational coverage (0.4), and efficiency (0.2).
| # | Repo | Stars | Score | Grade | structure | op. coverage | efficiency |
|---|---|---|---|---|---|---|---|
| 1 | block/goose | 51k | 84.2 | B | 95 | 75 | 81 |
| 2 | langchain-ai/langchain | 141k | 81.0 | B | 85 | 85 | 65 |
| 3 | ComposioHQ/composio | 29k | 79.6 | B | 85 | 75 | 78 |
| 4 | sst/opencode | 183k | 77.8 | B | 95 | 65 | 69 |
| 5 | QwenLM/qwen-code | 26k | 77.8 | B | 85 | 75 | 69 |
| 6 | n8n-io/n8n | 195k | 77.8 | B | 85 | 75 | 69 |
| 7 | langgenius/dify | 148k | 75.2 | B | 90 | 55 | 86 |
| 8 | Significant-Gravitas/AutoGPT | 185k | 74.2 | C | 95 | 45 | 91 |
| 9 | All-Hands-AI/OpenHands | 80k | 72.4 | C | 80 | 75 | 52 |
| 10 | Kilo-Org/kilocode | 26k | 71.6 | C | 85 | 75 | 38 |
| 11 | zed-industries/zed † | 87k | 70.0 | C | 95 | 45 | 70 |
| 12 | browser-use/browser-use | 103k | 69.4 | C | 75 | 75 | 47 |
| 13 | openai/codex | 96k | 66.4 | C | 85 | 55 | 52 |
| 14 | crewAIInc/crewAI | 55k | 55.2 | D | 80 | 15 | 86 |
| 15 | huggingface/smolagents | 28k | 50.0 | D | 75 | 0 | 100 |
| 16 | RooCodeInc/Roo-Code | 24k | 37.0 | E | 75 | 0 | 35 |
Seven B, six C, two D, one E. No A, no S. † zed ships AGENTS.md
as a symlink to its .rules
file; the score is for the resolved content, and the symlink itself is a finding — a tool that fetches the raw AGENTS.md
gets the six-character string .rules
, not instructions.
One repo was excluded as a non-independent data point: OpenInterpreter’s AGENTS.md
is byte-identical to openai/codex’s — the same git blob, since the project is a codex derivative for open models. Counting it twice would have been dishonest.
The standard-bearer paradox #
openai/codex
— the repo whose team did the most to popularize AGENTS.md
— scores 66.4, a C, 13th of 16. The file is not lazy: 22.5 KB across 28 headings, genuinely rich in review conventions and TUI style rules. But the operational-coverage dimension asks a narrow question: does this file equip an agent to operate the repo? Test and lint commands are there (just test
, just fix
). A build command is not. It appears nowhere in the file — for a Rust and Bazel monorepo. There are no commit or PR conventions either. I filed the gaps as an issue with an offer to fix.
The pattern generalizes across all sixteen. Structure is uniformly strong — 75 to 95, because headings are easy. What separates the top of the table from the bottom is operational coverage: the real setup, build, and test commands, and the gotchas an agent cannot cheaply rediscover at runtime.
Famous frameworks, empty operations #
Three files earn a D or E, and each survived me reading the raw Markdown by hand — these are not scorer artifacts:
crewAI (55.2, D): 1 KB of contribution platitudes — “follow best practices”, DRY, YAGNI. The only concrete commands are for the docs site (mintlify dev
). Nothing tells an agent how to set up, build, or test the actual Python framework. Operational coverage: 15 of 100.smolagents (50.0, D): the entire file is 157 bytes — four bullet points of generic advice (“Be Pythonic”, “follow OOP principles”). Operational coverage: 0.Roo-Code (37.0, E): 445 bytes — one genuinely useful UI-state gotcha and nothing else. No setup, no build, no test. Operational coverage: 0.
A high structure score with zero operational coverage is the signature of a file written for a human skim, not for an agent that has to run the project.
The best file in the sample has 170 stars #
Not one of the sixteen giants reaches an A. The best AGENTS.md
in my entire sample — the corpus below plus these sixteen — belongs to maxcountryman/underway, a Postgres-backed job queue with 170 stars: 91.0, grade A. It is 1,728 tokens covering setup, build, test, lint, and project structure as copy-pasteable commands.
block/goose
(84.2, B) shows the same shape among the big repos: 4.8 KB, almost five times smaller than codex’s file, covering Setup, Commands (Build / Test / Lint), Structure, and a Development Loop. Operational beats encyclopedic. An A-grade instruction file costs discipline, not headcount.
(For calibration, and fully disclosed: the scorer’s own repo file scores 91.6. That is the ceiling I know how to reach — and a 170-star library reached it independently, which is the more interesting fact.)
The wild is worse #
Before the sixteen, I scored a 30-file corpus of public AGENTS.md
files pulled from GitHub code search (collected 2026-04, re-scored on 8.5.0): mean 61.5, median 61.4. Seventeen of the thirty land at D or below. In aggregate, the median public AGENTS.md
is a D-plus — prose that reads fine and operates nothing.
”Can’t you just game a scorer like this?” #
You could, and an earlier version of this scorer fell for exactly the obvious attack: well-formed headings with junk commands stuffed into fenced code blocks. echo
, ls
, pwd
in a fenced bash block scored 92.5, an A, before the operational-coverage dimension existed — a failure I wrote into the spec rather than quietly patch. Today that same file scores 56.0, a D, with operational coverage 0: read-only verbs earn nothing, and full credit requires command diversity.
One honest limit remains, documented and pinned by a test: a plausible fabrication — invented but realistic-looking commands in well-formed sections — is indistinguishable from a real minimal file by any static scorer. The guarantee is scoped precisely: worthless text cannot outrank operational text. That is a smaller claim than “this measures quality”, and it is deliberately smaller.
What this measures, and what it does not #
The rubric is opinionated, and its weights are version one. The 0.4 / 0.4 / 0.2 headline was calibrated against a 30-file corpus without ground-truth labels. The validity evidence today is case-study-level — documented before-and-afters where the score moved with real fixes — not benchmark-level. If you disagree with a weight, the rubric is a readable dict and you can point at the line.Structure, not truth. The scorer cannot verify that a documented command actually runs. It reads form, not behavior.One snapshot. Every score is for the exact HEAD commit listed below, on engine 8.5.0. Files change; re-run before quoting.A low score can be a deliberate choice. A minimal file is a legitimate decision — it simply will not equip an agent, which is the only thing the number claims. In a separate audit, a 14.6k-token skill file kept its efficiency dimension at 47 because inlining was load-bearing. That was an informed decline, not a defect.The sweep is broad, not exhaustive. I checked 36 candidates. If I missed a repo above 20k stars with a rootAGENTS.md
, tell me and I will score it.
Reproduce it #
pip install schliff==8.5.0
curl -sO https://raw.githubusercontent.com/openai/codex/main/AGENTS.md
schliff score AGENTS.md
Or paste any file into the playground, or drop the live badge into your own repo. The whole point of using a deterministic score instead of a judge model is that this table is checkable — every number above comes out of a rule engine you can read, pin, and argue with.
Appendix: the sweep #
Scored (16): the table above. Excluded (1): OpenInterpreter/open-interpreter — AGENTS.md
byte-identical to openai/codex’s. Checked, no root AGENTS.md (19): cline, Aider-AI/aider, continuedev/continue, google-gemini/gemini-cli, anthropics/claude-code, microsoft/vscode-copilot-chat, microsoft/autogen, geekan/MetaGPT, microsoft/semantic-kernel, TabbyML/tabby, AntonOsika/gpt-engineer, SWE-agent/SWE-agent, stackblitz/bolt.new, Pythagora-io/gpt-pilot, reworkd/AgentGPT, TransformerOptimus/SuperAGI, getcursor/cursor, stitionai/devika, plandex-ai/plandex — itself a finding: more than half of the best-known agent repos I checked do not ship the ecosystem’s own standard file at the repo root.
HEAD commits, 2026-07-07: goose f96f62d9
· langchain 2d8100c4
· composio 040ffd49
· opencode 1c25b2f2
· qwen-code 40340ef5
· n8n 66ad8b93
· dify 6edce14e
· AutoGPT e2711b17
· OpenHands cc80397e
· kilocode b0348cbc
· zed fc827a21
· browser-use 052787f9
· codex cca16a10
· crewAI 799ab0f5
· smolagents 526069c1
· Roo-Code b867ec91
· underway 89e9bf9f
.