AI assistants now answer "who does X in Y" questions directly, and they build those answers from what they can read about you. For a small team, the cheapest thing you control is your own site: what crawlers are allowed to read, and how clearly your facts are stated.
On veilvex.com (an Astro static site in English, Russian, Azerbaijani and German) we treat this as part of the build, not a one-off file. Here is the setup and the rules we ended up with.
Many sites block AI crawlers by accident: a template, a CDN toggle, a security plugin. We generate robots.txt at build time and list the crawlers by name, even though User-agent: * already allows everything:
User-agent: *
Allow: /
Content-Signal: search=yes, ai-input=yes, ai-train=yes
User-agent: GPTBot
Allow: /
User-agent: OAI-SearchBot
Allow: /
User-agent: ClaudeBot
Allow: /
User-agent: PerplexityBot
Allow: /
User-agent: Google-Extended
Allow: /
Why list them? Because the next person who adds a "block bots" rule will see the intent, and because a test can assert that every agent in the list stays allowed. The Content-Signal line follows contentsignals.org and states how the content may be used.
Also check your CDN: on Cloudflare, "Block AI bots" and bot-fight settings can override a perfectly good robots.txt. We verified with real requests that GPTBot, ClaudeBot and PerplexityBot get a 200 and the full HTML.
llms.txt is a proposed convention: a short Markdown file at the site root that tells language models what the site is and where the important pages are. It is not a ranking factor and no engine promises to use it. We still ship it because it costs nothing and removes ambiguity.
The important decision: never write it by hand. Ours is built by a script from the same i18n content files that render the pages. So the price on the services page and the price in llms.txt cannot drift apart.
What goes in (English, compact):
Organization.description in JSON-LD and external profiles);
Next to it there is llms-full.txt with the full text of every page in all four languages (about 400 KB for us), for tools that want everything in one fetch.
This is the part that keeps it honest. The script throws, and the deploy stops, when:
llms.txt grows over prices.ts);[[...]] or {{...}}.
A simplified version of the price check:
for (const s of services) {
const expected = formatFrom(PRICES[s.id]);
if (!s.line.includes(expected)) {
throw new Error(`llms.txt: ${s.id} shows a price that is not ${expected}`);
}
}
Models repeat what they read. If your site says "guaranteed first place" in one corner, that is what an assistant may quote back to a customer. A build-time check is cheaper than finding it later.
Each service page carries Service + Offer and FAQPage JSON-LD built from the same content object as the visible text, with stable @id s across languages. The organization node uses the same brand line as llms.txt. One source, three outputs (HTML, JSON-LD, llms.txt), so a crawler sees the same facts whichever it reads.
<lastmod> is the date of the last git commit that touched the page's source files, not the build time. A build date on every URL tells crawlers nothing. After a deploy, a script compares the new sitemap with the live one and sends only new or changed URLs to IndexNow (Bing, Yandex and others).
None of this makes an assistant recommend you. It removes the reasons not to: blocked crawlers, conflicting prices, vague descriptions. Being named still depends on being mentioned in places models trust: directories, reviews, articles, discussions. We check monthly by asking ChatGPT, Perplexity and Gemini the same set of queries and logging who they name.
robots.txt and not blocked by your CDNlastmod dates, IndexNow after deploy
You can see the live files at veilvex.com/llms.txt and veilvex.com/robots.txt. Questions about the setup are welcome in the comments.
We are Veilvex, a remote team working on GEO (visibility in AI answers), SEO, websites and AI automation.