There's a module for that

Marcus Mailbox (the author) released agent-module-documentation, agent-first documentation for every Drupal 11 module, and the Agent Module Knowledge MCP, a DDEV add-on that lets coding agents search that documentation locally, following an October 9 talk at the Drupal AI Learners Club. The author's evals found an agent forced to read module code spent on average 47,600 more tokens, 24 more seconds and 10 more cents per task than one given the docs, which the author estimates at roughly 476 million tokens and $376,000 a year if 10,000 Drupal developers did it once a day. The projects target coding agents that write custom Drupal modules instead of searching drupal.org, bypassing Security Team coverage that contrib modules receive.

Drupal AI There's a module for that Coding agents write a custom module before they look on drupal.org. So I had an agent document every Drupal 11 module, and put the result behind a local MCP server, so your agent can find, judge and use what is already there. On 9 October I gave a talk at the Drupal AI Learners Club https://www.drupal.org/docs/develop/development-tools/ai-coding-tools-for-drupal-development/drupal-ai-learners-club-sessions about two projects I have been playing around with for about two months: agent-module-documentation https://github.com/Drupal-AI/agent-module-documentation , agent-first documentation for every Drupal 11 module, and the Agent Module Knowledge MCP https://github.com/Drupal-AI/ddev-drupal-agent-module-knowledge-mcp , a DDEV add-on that lets your coding agent search it. Thanks to Amber and Angie for having me. This is the written version. The recording is below, and the slides are online https://contrib-knowledgebase-presentation.marcusmailbox.com/ if you want to click through them yourself. "Build me a Facebook login" Ask a coding agent for a Facebook login on your Drupal site and it happily builds one. No question asked, no module searched for. You get a brand new OAuth flow written from scratch, enabled on your site and maintained by nobody. I have tried this with Claude, and it just starts creating a custom module, even though there are contrib modules for this that are covered by the Security Team. That is the first of six problems agents have with Drupal right now. Problem 2 is not only an agent problem. Search on drupal.org works when you already use the maintainer's words. Shy One-Time https://www.drupal.org/project/shy one time fixes the "one-time login link has already been used" error that Outlook's link scanner causes, but search for "password reset link already used outlook" and it isn't in the top 12 checked on 8 October . Problem 4 costs money. In the evals further down, an agent that had to read the code spent on average 47,600 more tokens, 24 more seconds and 10 more cents per task than one that had the docs. If 10,000 Drupal developers did that once a day, that is about 476 million tokens a day, or $376,000 a year, to learn again what one documentation run already wrote down. That is a back-of-the-envelope number, not a measurement. And problem 5 is where Drupal should win. The Security Team, security advisories for covered contrib and Update Status put a whole security process behind a covered module. A custom OAuth module written by an agent has none of that, and a non-developer can't spot a missing state check on the callback. Meanwhile, in JavaScript land I worked a lot with Express.js before, and bootstrapping an Express.js app with an agent just works better. npm has a public search API and a CLI, the README ships in the package, and the model has seen a massive amount of training data about it. Drupal's long tail of modules is barely in the training data, and if you don't ask for a framework, the agent is likely to suggest Python or Node.js. When agents pick the stack, "it just worked" wins, and Drupal gets left behind. Attempt 1: a hosted module finder The first attempt was the Drupal Module Finder https://module-finder.marcusmailbox.com/ , which still exists. It is a semantic search index over every Drupal 11 contrib module, plus a skill so agents can call it: npx skills add ivanboring/drupal-module-finder It works. But it is another server to host, scale and pay for. I gave about 50 people access to it, and that is fine for me to host. If every Drupal developer used it, it would start costing money. The other option is asking drupal.org's infrastructure to run something we don't even know gives better results than traditional search. Attempt 2: run it on your own machine So the second attempt asks a different question: what if every developer ran it locally? DDEV is already the default local stack for Drupal developers, so the search became an MCP server shipped as a DDEV add-on. One command, a restart, and your agent knows the Drupal ecosystem: ddev add-on get Drupal-AI/ddev-drupal-agent-module-knowledge-mcp ddev restart Register it with Claude Code on the host : claude mcp add --transport http agent-module-docs http://<project .ddev.site:9130/mcp The agent gets five tools: search modules , get module , list docs , read doc and list categories . It uses plain HTTP on port 9130 on purpose: DDEV's HTTPS uses a local mkcert certificate, and Node-based clients like Claude Code don't trust it. The traffic never leaves your machine. In the talk I showed it live, with two Drupal sites set up with the One Line Installer https://www.drupal.org/project/one line installer , which runs the agents in containers and can add this MCP server through its agent-ready Drupal build kit. Both agents got the same prompt about Outlook breaking one-time login links, asking them to fix it by writing some code. The one with the MCP server searched the local index and told me to install Shy One-Time. The vanilla one wrote its own code instead. agent-module-documentation Every Drupal 11 module, read by an agent 8,794 projects installed, read and documented, so your agent doesn't have to. The MCP server is the small part. The part that is worth something is the dataset behind it. In popularity order, an agent installed every Drupal 11 compatible project on a real Drupal 11 site in DDEV, read its code and configuration, and wrote compact, agent-first documentation for it. It took about 1,000 commits over 51 days, and it stays current: scanners watch the drupal.org release feeds and document new stable minor versions again. You can browse the catalog https://drupal-ai.github.io/agent-module-documentation/ without an agent. Every module version gets three kinds of documentation, one for each of the problems above. For finding modules usage.md has a short summary, a long summary and then 15 to 30 use cases written in the user's words, like "Fix 'link already used' complaints", and data.json adds categories and keywords. These are the sentences people actually type, so the search from problem 2 now has something to match. It works very well on small and medium-sized modules. Where it misses is modules that can do everything: it suggests ECA https://www.drupal.org/project/eca less often than it should, because you can't write down every use case ECA covers. For judging modules data.json holds the core version requirement, security advisory coverage, maintenance status, contributor count, dependencies, and whether the module ships Drush commands, tool plugins or agent skills. The long summary is an honest assessment with trade-offs and caveats, not marketing. For using modules agent/start.md is a cheap index with the key facts, always under 500 tokens. A bigger module links to solution docs for configuration, APIs, plugins, Drush commands and hooks. The rule: every agent doc must be shorter than reading the source. For Social Auth Facebook, start.md says that its settings config doesn't exist until the form is saved and which OAuth callback to register in the Meta app: exactly the things an agent would otherwise dig out of the source, one file at a time. And a security read If we say "Drupal is more secure", we have to check it. The agent reading the source also looks for insecure defaults, SSRF, injection, access bypass, CSRF and credentials stored in plain text. It is not a formal audit. AI is very good at finding security issues, and just as good at finding false positives, so people from the AI initiative do a first human review before anything goes to the Security Team https://www.drupal.org/security-team/report-issue or a maintainer. We don't want to spam them. The notes stay out of the public repository, and real issues go through coordinated disclosure. The Security Team are the true heroes here; we try to make their work easier, not add to it. Agent docs are the source, human docs are a render Why write docs for agents and not for humans? Because the agent docs can be the source. They are dense, structured and grounded in the real code, so you can render them into whatever a person needs. From the 109 lines of one module's agent docs I generated a narrated video https://contrib-knowledgebase-presentation.marcusmailbox.com/shy.mp4 , a PDF guide https://contrib-knowledgebase-presentation.marcusmailbox.com/derived/pdf/shy-one-time-guide.pdf , a game https://contrib-knowledgebase-presentation.marcusmailbox.com/derived/game/index.html and a podcast https://contrib-knowledgebase-presentation.marcusmailbox.com/derived/podcast/shy-one-time-podcast.mp3 . The podcast is the one I use myself: I go for long walks, and when I want to understand a concept, I have three hosts talk about it. Write the knowledge once, and render it for every audience. Does it actually help? Whenever you write a skill, you have to write evals for it. It is very easy to create a skill and think it works so well, but vanilla Claude Code is also very good, and you might just be feeding it context it doesn't need. We measure three things: is it more correct, and if it is just as correct, is it faster or does it use fewer tokens? If none of that is true, you don't need it. The evals have three arms: a vanilla agent, an agent with a skill that tells it where the docs are, and an agent with the right docs already in its context. The tasks come in three levels. Easy is answering a question, medium is inspecting a live site, and hard is building something, checked against the live site's state. | Long-tail modules | | | | |---|---|---|---| | Measure | Vanilla | Skill | Docs in context | |---|---|---|---| | Correct, Claude Haiku 4.5 | 80% | 86% | 100% best | | Time per task, Claude Opus 4.8 | 33.3 s | 21.2 s | 9.2 s best | | Tokens per task, Claude Opus 4.8 | 71.6k | 85.1k | 24.0k best | | Cost per task, Claude Opus 4.8 | $0.19 | $0.15 | $0.09 best | Precision goes up on long-tail modules, most of all for smaller models. Time drops by about a third with the skill and by almost three quarters with the docs in context. To be honest about the tokens: the skill arm uses more of them than vanilla, because the agent spends them learning where the docs are. Handing over the right doc directly is what cuts tokens to a third, and that is the MCP server's job. The evals did not test the MCP server itself; it sits somewhere between those two arms. And the extra tokens are a first-question cost: when you ask a second question about the same module, the token use goes down again. Then the plot twist. With a frontier model and the top 100 modules Pathauto, Token, Webform, Metatag and friends , the model already knows. Claude Opus 4.8 got 97% right without the docs and 98% with the skill, which added overhead: 94.6k instead of 82.6k tokens and $0.169 instead of $0.159 per task. The only gain was some speed, 36 seconds down to 31. Smaller models still benefit even there: Claude Haiku on hard top-100 builds went from 33% to 78%, but that is three cases, so treat it as a hint. That frontier result is the good news. The fact that the docs help at all means the model doesn't know the module. If every module were as well known as Pathauto, you could ask about any of them and get the right answer, and that is what we want for the rest of Drupal. Training data for the next model Why does Opus know Pathauto? Because the internet does. The top 100 modules have years of blog posts, Stack Exchange answers, documentation and issues, and all of it is in the training data. The long tail has a README, and maybe a project page. I didn't think of this side effect myself. Scott Falconer https://www.drupal.org/u/scott-falconer pointed it out after seeing the project: the 24,642 agent docs are published openly on GitHub. Today they are context: agents read them through the MCP server. Tomorrow they could be knowledge, because public, structured docs grounded in real code are exactly what a training run can pick up. They are synthetic, yes, but generated from installing and reading the actual module, which is still better Drupal training data than nothing. Whether any lab will train on them, I can't promise. It is a hope with a good reason behind it. I started this in my free time, and it has now moved into the Drupal AI initiative. It is MIT licensed the docs for each module keep that module's license, almost always GPL-2.0-or-later , it will keep being updated, and you can use it for whatever you want. If you need a SKILL.md for your own module, you can start from what is there. How the MCP server works The idea is that GitHub takes the hit, and drupal.org gets no load at all. A GitHub Action in the add-on's repository pulls the documentation, builds one SQLite file, docs.db , and a container image, and publishes them as a GitHub Release about 200 MB gzipped and on GHCR. Your DDEV project downloads them, and search runs on your laptop. There is no new infrastructure to approve or scale. Search is hybrid, because each kind misses something: - Lexical BM25 in SQLite FTS5 matches the words you typed. Great for exact names and config keys like graph version or pathauto , but it misses "mail scanner eats my reset link". - Semantic KNN with sqlite-vec over embeddings matches meaning, so "reset link already used in Outlook" lands near "Stop a mail scanner consuming a reset link". It is fuzzy on exact identifiers. - Fused with Reciprocal Rank Fusion: each result scores the sum of 1 / 60 + its rank in both lists. I won't pretend I know the theory behind it; it is how the search people say you should merge the two, and the results are very good for such a small database. The embeddings come from Model2Vec, static embeddings where every token has a precomputed vector, so embedding a text is mostly a lookup. The model, minishlab/potion-retrieval-32M , is baked into the image and runs fully offline. It indexes the roughly 10,000 documents in about 20 seconds, with no PyTorch and no GPU. It is also locked down: a non-root container, a read-only database, doc bodies served from the database with no filesystem access, and the endpoint is only exposed through the DDEV router. One ask for maintainers Ship agent skills inside your module. The AI module https://www.drupal.org/project/ai 1.5.x has eight of them in .agents/skills/ , from creating an AI provider to a guardrail plugin. Ask an agent to "add an AI provider for X" and it follows the maintainers' own recipe, and gets it right almost every time. A skill in the module is versioned with the code and written by the people who know the module best. When I gave the talk, only 11 of the 8,794 projects shipped skills. Add .agents/skills/<name /SKILL.md to yours; the knowledge base already detects shipped skills and lists them per module. Skills make your module easy to extend and easy to set up, and they teach future models how to use it. Now you do something better The dataset is free. It is open on GitHub under the MIT license: use it, fork it, index it your own way. That is the part that took time, about 1,000 commits over 51 days, and it is the part worth something. The MCP server is just one way to put it in front of an agent, and it doesn't have to be this one. It was vibe coded in about four hours, mostly by searching and putting common Python libraries together, and it is about 600 lines of Python. Replace it freely. It could be hosted search, a drupal.org integration, Project Browser, skills with scripts, or your own agent. And the DDEV idea might be a bad one. It is one container, model and database per project, so memory and disk for every project you run, and about 200 MB to download for every install. It uses about 180 MB of memory even when it does nothing. There are newer Gemma-based embedding models from Google that perform better and run in about 2 GB of RAM, so the search could improve too. A shared global service or a hosted endpoint may be the better long-term shape. What came up in the questions The Q&A ran past the hour, and most of it was about documentation. These are the parts I want to keep. How do the evals work? Each module with evals has typical questions with the outcome you want. Some can be checked with code: for the Address https://www.drupal.org/project/address module, a correct answer mentions getCountryCode . Others need an AI to judge how well the agent did. A wrapper runs Claude Code, Codex or OpenCode headless, and their JSONL logs show how they got to the answer. For evals of Drupal modules and skills in general, look at AI Evals https://www.drupal.org/project/ai evals . Why GitHub and not drupal.org? Amber asked the big one. Documentation is often the last thing a maintainer does, and sometimes it never happens: some modules don't even have a README, or worse, a project page that doesn't say what the module does. Well-thought-out human documentation is a lot better than what AI writes, but AI documentation is a lot better than nothing. We also thought about opening a merge request with skills on every module, but not everyone wants AI in their module, and not everyone has time to review it. This dataset is the middle ground, until the knowledge lives inside the modules. Angie suggested the front door: drupal.org is the source agents already trust, so ideally the information comes from there, and an agent could read the maintainer's own skills first and fall back to this dataset when there are none. We will continue that conversation. Why an MCP server and not just the files? I tried giving the agent the documentation as skills with scripts, and it worked about as well. But Vercel's skills tool flags skills as malicious when they tell the agent to download things; our video tutorial skill gets flagged because it installs FFmpeg. And in the Drupal community, more people know DDEV. Does it work offline? Yes. It recommends a module and knows how to develop for it without internet access. Only the Composer commands it gives you need the internet. Can the docs help humans too? Around 1,000 modules in the repository also have human documentation, written for MkDocs, the documentation system in the Drupal Association's GitLab template. Doing it for every module would take three or four months, but for a small or medium-sized module it takes about ten minutes. So the skill is what we should give maintainers: if you don't have time to write documentation, run this, because it is better than none. It could also be an opt-in, where a maintainer asks for documentation and gets a merge request. Christopher Weber said that MkDocs documentation on drupal.org has been discussed with the Drupal Association before and stalled on money. One argument for it: the industry is already paying for the missing information in token costs. Try it - ddev-drupal-agent-module-knowledge-mcp https://github.com/Drupal-AI/ddev-drupal-agent-module-knowledge-mcp : the DDEV add-on with the MCP server. - agent-module-documentation https://github.com/Drupal-AI/agent-module-documentation : the dataset, the method and the evals. - The catalog https://drupal-ai.github.io/agent-module-documentation/ : browse the docs in a browser. - Drupal Module Finder https://module-finder.marcusmailbox.com/ : the hosted semantic search from attempt 1. - The slides https://contrib-knowledgebase-presentation.marcusmailbox.com/ from the talk. I also used the MCP server to build this site; How I vibe-configed this website in under 10 hours https://workflows-of-ai.com/blog/how-i-vibe-configed-this-website has the details. Chapters 16 min read Other ways to consume this Listen to this article 20:47 Articles read out podcast feed https://workflows-of-ai.com/articles-read-out.xml Podcast: There's a module for that Podcast episode 5:30 Workflows of AI Podcast Listen in your podcast app Transcript - AI disclosure - This podcast episode is created and spoken by AI based on articles written on workflows-of-ai.com. - Kwame Kwilson - Today's post is called "There's a module for that". It's the write-up of a talk Marcus gave at the Drupal AI Learners Club. Jane, what's it really about? - Jane Olson - You ask a coding agent for a Facebook login on your Drupal site, and it doesn't look for a module. It just starts writing one. Brand new OAuth flow, maintained by nobody. And there are contrib modules for that, covered by the Security Team. - Kwame Kwilson - And that costs something. - Erin Milleu - It does. In Marcus's evals, an agent that had to read the code spent on average forty-seven thousand six hundred more tokens per task than one that had the docs. Scale that to ten thousand developers and it's about three hundred and seventy-six thousand dollars a year. Back of the envelope, he says. - Kwame Kwilson - So whose fault is it? The agent's? - Erin Milleu - Largely, yes. I'd push back on blaming Drupal. An agent that writes security code before it has searched for an existing solution is behaving badly, full stop. That's a model problem. - Jane Olson - Hang on. The agent can't search drupal dot org. It's an HTML app behind a bot challenge. And even humans struggle. There's a module called Shy One-Time that fixes the Outlook "link already used" error, and if you search for "password reset link already used outlook", it's not in the top twelve. - Erin Milleu - Then the agent should say "I couldn't search", not quietly write a module. - Jane Olson - Sure, but you can't fix that in the model by Thursday. You can fix the docs. - Kwame Kwilson - Let's leave that open. What did he build? - Jane Olson - The dataset. An agent installed every Drupal eleven compatible project on a real site, read the code, and wrote docs for agents. - Erin Milleu - Eight thousand seven hundred and ninety-four projects. Twenty-four thousand six hundred and forty-two agent docs. And seventy-two percent of the projects are covered by security advisories, which is recorded per module, so an agent can filter on it. - Jane Olson - And I love the use cases. Fifteen to thirty per module, written the way users actually talk. "Fix 'link already used' complaints." That's what people type, not the maintainer's clever module name. - Erin Milleu - The detail I liked is the security read. The agent looks for things like injection and access bypass while it reads the code, but people from the AI initiative review it before anything reaches the Security Team. AI is good at finding issues and equally good at finding false positives. - Jane Olson - Nobody wants their inbox full of robot panic. - Kwame Kwilson - Then the evals. Does it actually help? - Erin Milleu - On the long tail, yes. On the famous modules, not really. With the top hundred, Pathauto, Webform and so on, Claude Opus got ninety-seven percent right on its own and ninety-eight percent with the skill. And the skill added tokens and cost. - Jane Olson - Which is the good news It means the model already knows the popular modules. The long tail is where it matters. Haiku went from eighty percent to a hundred with the right docs in context. - Erin Milleu - On easy questions, mostly one run each. And the hard top-hundred builds that went from thirty-three to seventy-eight percent are three cases. I'd want a lot more runs before I called that a result. - Jane Olson - He literally calls it a hint. - Erin Milleu - Then I'll accept it as a hint. - Kwame Kwilson - Here's a number that surprised me. When Marcus gave the talk, eleven of the eight thousand seven hundred and ninety-four projects shipped their own agent skills. Eleven. - Jane Olson - And the AI module alone has eight of them. Ask an agent to add an AI provider, and it follows the maintainers' own recipe. - Erin Milleu - That's the right place for it. A skill inside the module is versioned with the code, so it matches the release you installed. - Jane Olson - And it's written by the people who know the module best. - Kwame Kwilson - Last topic. The MCP server itself. - Jane Olson - Four hours. That's how long Marcus says he spent vibe coding it. About six hundred lines of Python. He basically tells people to replace it. The dataset is free, MIT licensed, and the server is just one way to put it in front of an agent. - Erin Milleu - I appreciate that honesty. Most people present their four-hour prototype as an architecture. - Jane Olson - Erin, you'd have spent four hours writing the evaluation plan for the four-hour prototype. - Erin Milleu - And it would have been a very good plan. - Jane Olson - There's also the training data angle. Scott Falconer pointed out that the docs are public on GitHub, so the next models might simply learn the long tail. - Erin Milleu - Might. Nobody can promise a lab will train on them. And he admits the DDEV setup might be a bad idea too. A container, a model and a database per project, about a hundred and eighty megabytes of memory doing nothing. - Jane Olson - A hundred and eighty megabytes is less than my browser uses to show me one cookie banner. - Kwame Kwilson - On that note. The dataset is free, the server is replaceable, and the long tail finally has something to read. Thanks, both. The post in three sentences Coding agents write their own Drupal modules before they look on drupal.org, and they can't search, judge or use the modules that are there without reading all the code. agent-module-documentation has agent-first docs for all 8,794 Drupal 11 compatible projects, and a DDEV add-on gives your agent hybrid search over them through a local MCP server. On long-tail modules the docs made agents more precise and up to three times faster; on the top 100, frontier models already know.