Can ChatGPT and Claude read your website? A 5-minute checklist

A developer published a five-minute checklist for making websites readable by AI assistants such as ChatGPT, Claude and Perplexity, covering robots.txt user agents (GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot), JavaScript-rendered content that AI crawlers cannot execute, an /llms.txt file, JSON-LD structured data, and SPF/DKIM/DMARC email authentication. The author wrapped the checks into pay-per-result tools on Apify that score a domain from 0 to 100 and return a concrete fix for each issue, priced at 0.005 USD per result.

Search is changing: more people ask ChatGPT, Claude or Perplexity instead of opening Google. If those assistants cannot read your site, you will not be quoted. Here is a five-minute checklist. Open yoursite.com/robots.txt and look for these user agents: GPTBot , ClaudeBot , PerplexityBot , Google-Extended , CCBot . A blanket Disallow: / under User-agent: blocks all of them. Many CMS security plugins add it silently. Run curl -s https://yoursite.com | grep -i "your headline" . If the headline is missing, your content is rendered by JavaScript, and most AI crawlers do not run JavaScript. They see an empty page. A plain markdown file at /llms.txt that lists your key pages and describes the site. It is not a standard everyone honors yet, but it costs ten minutes and some assistants read it. JSON-LD Organization , Product , Article , FAQPage tells a crawler what a page is without guessing. Validate it: a missing required property usually means the markup is ignored. If your domain sends mail, check SPF, DKIM and DMARC. Without them a lot of your mail lands in spam, and Google and Yahoo now require them for bulk senders. I got tired of doing this by hand, so I wrapped the checks into small tools on Apify. Each takes a domain and returns a 0-100 score with a concrete fix for every issue: They are pay-per-result 0.005 USD per result , so scanning a handful of domains costs cents. Feedback and feature requests are welcome in the comments.