Search is changing: more people ask ChatGPT, Claude or Perplexity instead of opening Google. If those assistants cannot read your site, you will not be quoted. Here is a five-minute checklist.
Open yoursite.com/robots.txt and look for these user agents: GPTBot, ClaudeBot, PerplexityBot, Google-Extended, CCBot. A blanket Disallow: / under User-agent: * blocks all of them. Many CMS security plugins add it silently.
Run curl -s https://yoursite.com | grep -i "your headline". If the headline is missing, your content is rendered by JavaScript, and most AI crawlers do not run JavaScript. They see an empty page.
A plain markdown file at /llms.txt that lists your key pages and describes the site. It is not a standard everyone honors yet, but it costs ten minutes and some assistants read it.
JSON-LD (Organization, Product, Article, FAQPage) tells a crawler what a page is without guessing. Validate it: a missing required property usually means the markup is ignored.
If your domain sends mail, check SPF, DKIM and DMARC. Without them a lot of your mail lands in spam, and Google and Yahoo now require them for bulk senders. I got tired of doing this by hand, so I wrapped the checks into small tools on Apify. Each takes a domain and returns a 0-100 score with a concrete fix for every issue:
They are pay-per-result (0.005 USD per result), so scanning a handful of domains costs cents. Feedback and feature requests are welcome in the comments.