Make AI Agents See Your Website

RevoGrid, a JavaScript data grid library, has adopted llms.txt and Agent Skills to make its documentation accessible to AI coding agents like Codex, Cursor, and Claude Code. The llms.txt file provides a text-first entry point for agents, while the SKILL.md structure offers progressive disclosure of documentation. This approach ensures agents can find the same installation guides, API references, and migration notes that a human developer would read.

AI coding agents are now part of the developer workflow. Whether we like that shift or hate it, users ask Codex, Cursor, Claude Code, GitHub Copilot, and other tools to install packages, wire examples, migrate code, and explain APIs. For JavaScript data grids https://rv-grid.com/blog/datagrid , that often means asking an agent to build columns, editors, filters, Pivot views, or Gantt timelines. We ran into the obvious problem: the docs were written for people. A person can use the sidebar, search, breadcrumbs, examples, and product context. An agent often starts with a URL and a vague instruction. If it cannot find the right entry point quickly, it guesses. There are several ways to make a site easier for agents to use: an MCP server for live retrieval, skill bundles for structured on-demand context, and llms.txt as a public discovery layer. That is where llms.txt helps. It is a public, text-first entry point that tells AI agents where the important documentation lives: installation guides, API reference, examples, migration notes, troubleshooting, full text exports, and any richer machine-readable bundles. The goal is not to “optimize for bots” at the expense of people. The goal is to make sure that when a user asks an agent to build with RevoGrid, the agent can find the same source material a careful developer would read first. llms.txt Actually Used? llms.txt began as a proposal https://llmstxt.org/ , published by Answer.AI co-founder Jeremy Howard in September 2024. There is meaningful adoption among major developer platforms: llms.txt index llms-full.txt export. llms.txt index llms.txt and llms-full.txt files. llms.txt and llms-full.txt llms.txt Is For Use it to answer these questions quickly: Example shape: Product Documentation Primary Docs - Installation https://rv-grid.com/guide/installation : Install and configure the package. - API Reference https://rv-grid.com/guide/api/revoGrid : Public TypeScript API. - Migration Guide https://rv-grid.com/guide/migration : Upgrade notes. llms.txt to a Documentation Site https://rv-grid.com . /llms.txt . .skill bundles or MCP instructions.For most docs sites, llms.txt can be generated from the same content collection, sidebar, or route manifest that powers the website. That is important. If the AI-facing docs drift from the real docs, agents will confidently produce stale answers. llms.txt If your product has paid, enterprise, or advanced modules, say so directly. Agents need to know whether a feature is core, Pro, Enterprise, experimental, or deprecated. Ambiguity here turns into broken code later. For large docs, a split skill is usually better than asking an agent to read one giant file. The RevoGrid skill gives agents a SKILL.md index and one reference file per docs page. This structure is no longer limited to one agent. Anthropic introduced Agent Skills https://www.anthropic.com/engineering/equipping-agents-for-the-real-world-with-agent-skills as folders containing instructions, scripts, and resources, and later released the format as an open standard. OpenAI's Codex documentation https://developers.openai.com/codex/skills also recommends a SKILL.md file with optional references/ , scripts/ , and assets/ directories. Both describe progressive disclosure: the agent sees a small description first and loads the full skill or individual references only when the task needs them. The format and current ecosystem are documented at Agent Skills https://agentskills.io/ . This is the best format when: For coding agents, the better structure is closer to how a human reads docs: product-docs/ ├── SKILL.md └── references/ ├── getting-started/ │ └── installation.md ├── api-reference/ │ └── grid.md └── guides/ └── migration.md SKILL.md is the index and operating guide. It explains when to use the docs and links to focused reference files. This mirrors the progressive-disclosure model documented by both OpenAI https://developers.openai.com/codex/skills and the Agent Skills specification https://agentskills.io/specification : advertise a small amount of metadata, then load detailed instructions and references only when relevant. Each file under references/ should represent one real documentation page. The agent can read the index, choose two or three relevant files, and avoid loading a massive corpus into context. This improves answer quality because the model sees complete local context for the page it selected: title, URL, description, source path, examples, and surrounding explanation. It also makes failures easier to debug because you can see which reference file the agent used. The implementation should be part of your documentation build pipeline. That keeps the work boring, repeatable, and less likely to rot. At build time or request time: llms.txt as a short index. llms-full.txt as the full fallback corpus. SKILL.md plus references/<section /<page .md . SKILL.md folder format; if your target client supports installable .skill archives, also package the directory as a zip with a .skill extension.The output should be deterministic. If the same docs produce different filenames on every build, agents and caches cannot rely on links. Each split reference file should be self-contained: Page Title URL: https://rv-grid.com/guide/page/ Source: src/content/docs/guide/page.mdx Description: Short page summary. Page content starts here... Keep the original code examples. Remove interactive-only wrappers that do not help in text form. If your docs use MDX components, unwrap or remove them carefully so the remaining Markdown still reads naturally. This is not only for bots. Developers are already trying to understand how their docs should work with agents, and there is not much practical writing from teams that have actually wired this into a docs pipeline. That is the reason to write about it. Not because another SEO page needs to exist, but because someone else will hit the same problem: they publish a full text export, point an agent at it, and still get an answer that misses the important part. The useful searches are real: The page should answer those questions directly. If it does that, search visibility is a side effect of being useful, not the whole point. llms.txt Too Large If llms.txt becomes another full corpus, it stops being useful as an index. Keep it short and link out. llms-full.txt A full export is helpful, but it is not enough for large docs. Agents often grep it and miss important context. Reference files need a good SKILL.md or manifest. Without an index, the agent still has to guess which file to open. Do not write a separate AI manual by hand unless you can maintain it. Generate from the same docs source whenever possible. llms.txt as robots.txt llms.txt provides content and navigation; it does not grant or deny crawler access. Keep using robots.txt and the documented controls for each crawler when access policy matters. There is no reliable evidence that llms.txt alone improves AI citations or search visibility. The Ahrefs study https://ahrefs.com/blog/what-is-llms-txt/ , critical field reports https://medium.com/@kaispriestersbach/the-llms-txt-is-dead-more-precisely-a-dud-ab7bee4f469c , and practitioner discussion on Reddit https://www.reddit.com/r/SEO/comments/1pqi6uv/is llmtxt useful and beneficial for better ai/ all show why the claim remains disputed. Measure requests in server logs and evaluate agent answers against real tasks instead of assuming an effect. llms.txt short and navigational. llms-full.txt as a full fallback. llms.txt . robots.txt , not llms.txt . llms.txt proposal llms.txt implementation with Markdown page links and a full export. llms.txt SKILL.md format and validation rules. llms.txt llms.txt llms.txt llms.txt , and Should You Care About It? llms.txt is dead. More precisely: a dud.”The goal is not to feed the model more text. The goal is to help the agent choose the right text. For a small project, llms.txt plus clean Markdown pages may be enough. For larger libraries, frameworks, and enterprise components, add levels: a short index, a full fallback corpus, split reference files, and MCP for live retrieval. That is the model we use for RevoGrid: make the docs easy for humans to browse and easy for AI agents to discover, select, and read. The fact that major platforms publish llms.txt and formally support Skills gives this approach credibility; honest measurement and multiple retrieval paths keep it grounded.