# How to check llms.txt on every site you run

> Source: <https://lopebase.com/blog/check-llms-txt-on-every-site>
> Published: 2026-11-04 12:00:00+00:00

[Blog](https://lopebase.com/blog)

# How to check llms.txt on every site you run

The first product gets an llms.txt the week you read about it. You write it by hand, list the docs and the pricing page, and move on. By the fourth product, one file still links to pages you renamed in the spring, one site answers `/llms.txt` with its home page, and the newest product never got one at all. Nothing in any of the apps reports an error, because nothing in the apps reads the file.

Checking one llms.txt takes a minute. Checking every product's, every week, needs a routine. Here is what to check, how to check it the way an AI tool fetches the file, and how to keep the files from going stale in the first place.

## What a valid llms.txt has

llms.txt was [proposed in September 2024](https://llmstxt.org/) as a short markdown map of a site for AI tools to read. There is no official schema, so "valid" means the file has the parts the proposal describes:

- **A text file at the root** , at`https://yourproduct.com/llms.txt` , answering with a 200 status.
- **One `#` title** on the first line, naming the product.
- **A `>` summary** , one sentence on what the product is, before any section.
- **`##` sections of links** , one markdown link per line, with a short note after each:`- [Pricing](https://yourproduct.com/pricing): Plans and limits` .
- **An optional `## Optional` section** for pages a tool can skip when it is short on room.

The [llms.txt checker](https://lopebase.com/tools/llms-txt-checker) covers the format in more detail, with an example file. The rest of this post is about the part that goes wrong with several products: files that were right once and are not any more.

## Check it the way a tool fetches it

Opening `/llms.txt` in a browser is not a real test. The browser shows whatever comes back, and an HTML page that happens to contain your text looks fine. Ask for the file the way a script would and read the status and type first:

```
# Status, content type, and any redirect
curl -sI https://yourproduct.com/llms.txt
# The first lines of the file itself
curl -sL https://yourproduct.com/llms.txt | head -n 5
```

You want a 200 with `content-type: text/plain` or `text/markdown`, and a first line that starts with `#`. Three failures account for most broken files:

- **The app answers with its HTML.** Single-page apps and many hosts send back`index.html` with a 200 for any path they do not know, so a missing file looks like a working page. If the first line is`<!doctype html>` , there is no llms.txt, whatever the status says.
- **A redirect to somewhere else.** A move from`www` to the bare domain, or a marketing site that moved to a new host, can send`/llms.txt` to a page that does not have it. Follow the redirect and check where it lands.
- **Bot protection in the way.** A challenge page meant for scrapers can answer requests that do not come from a browser. If the file loads in your browser but`curl` gets a challenge, AI tools probably get one too.

## Check the links inside, not just the file

A file can pass every format check and still send readers to dead pages. Links go stale when you rename a docs section, retire a plan, or move the blog. The format checks will not catch it, because a broken link is still a well-formed link. Check each one:

```
# Print the status of every link in the file
curl -s https://yourproduct.com/llms.txt \
  | grep -o '](https\?://[^)]*' | cut -c3- \
  | while read url; do
      echo "$(curl -s -o /dev/null -w '%{http_code}' "$url") $url"
    done
```

Anything other than a 200 needs a fix: a 404 is a page that moved, and a 301 is a link to update so tools do not have to follow it. While you are there, check that the links point at the same domain the file is on. A file copied from one product to start the next is how a link to the wrong product's pricing page ends up in it.

## Generate it from the build, not by hand

The lasting fix for stale files is to stop writing them by hand. If a product's site is built from a list of pages, the build can write llms.txt from the same list: a short hand-written intro with the title and summary, then a link and description for each page that matters. A new page shows up in the file the day it ships, and a removed page leaves with it.

That is how we do it on lopebase.com: the build writes llms.txt from a short file of facts and the list of every page, so nobody has to remember it. If a product's site is a hosted builder with no build step, keep the file in the repository next to the app and add the link check above to whatever runs before a deploy.

## A weekly check across every product

With several products, the useful view is one row per domain, so a broken file stands out from the rest:

| What to record for each product's llms.txt |  |  | 
|---|---|---|
| Check | Pass | Common failure | 
|---|---|---|
| Served | 200, text/plain or text/markdown | HTML home page with a 200 | 
| Title and summary | A `#` line, then a`>` line | Summary missing or placed after a section | 
| Links | Every link answers 200 | Renamed docs pages, retired plans | 
| Right product | Links on the file's own domain | Copied from another product | 
| Size | The main pages only | A copy of the whole site | 

It takes a few minutes per product, so it fits in a [weekly review](https://lopebase.com/blog/weekly-review-for-saas-founders) next to uptime and search. A product you just launched or redesigned deserves a check the same day, since that is when links move.

## What llms.txt does not do

It is worth keeping the file in proportion. llms.txt is a proposal that some AI tools and coding agents read, not a standard every tool follows. Google has not said it uses llms.txt for search, and the file does not change how you rank. It also does not let anything in: if your robots.txt blocks AI crawlers, a perfect llms.txt will not be read by them. Check that separately with the [AI crawler checker](https://lopebase.com/tools/ai-crawler-checker).

So treat it as cheap upkeep: a few lines per product, generated where possible, checked on the same schedule as everything else. A wrong file is worse than none, because it sends tools to pages that are gone.

## How LopeBase does it

We run several products ourselves, and each one has its own llms.txt. The free [llms.txt checker](https://lopebase.com/tools/llms-txt-checker) reads any domain's file with no account: whether it is served as text, its title, summary, sections, and link count, with a flag for each missing part. It does not follow the links inside the file, so keep the link check above for that.

In LopeBase, the same check is part of each property's [SEO audit](https://lopebase.com/docs/seo#site-checks), so every product's llms.txt shows on one screen next to robots.txt, sitemaps, and structured data, and you can run it again whenever you like. When a site has no file, the audit gives you a prompt to write one from your pages. The AI part of the site health score counts whether the file exists alongside which AI crawlers robots.txt lets in. What LopeBase does not do: write or host the file for you, or tell you which AI tools have read it.

**Every product's llms.txt, on one screen.** Free for one property. No card.

[Start free](https://app.lopebase.com/signup)