rankprobe: an SEO and AI-crawler audit that fails your PR before a stray noindex ships

I cross-post my dev.to articles to a small Jekyll site on GitHub Pages. Last week I pointed a new audit tool at that site and it told me 15 out of 15 pages had a meta description Google will cut off, 15 out of 15 titles were too long, no page had an og:image, and the origin served no robots.txt, no sitemap and no llms.txt. None of that was visible from the browser. Every page looked fine.

That tool is rankprobe, and this post shows what it does on a real site and how to wire it into CI.

Repo: github.com/bayuh85/rankprobe (MIT, Python 3.10+, standard library only, 80 offline tests)

SEO breakage is silent. A noindex left over from staging, a canonical pointing at the wrong host, a robots.txt that blocks GPTBot and ClaudeBot because someone copied a "block AI" snippet, JSON-LD with a trailing comma. The page renders, the PR gets approved, and you find out weeks later from Search Console, or from the fact that no AI search engine ever cites you.

The usual options are a paid SEO suite (slow, per-seat, not in your CI) or handing the HTML to a chat model and asking "what's wrong with my SEO?", which spends tokens rediscovering things a parser settles in milliseconds, and sometimes invents issues.

I wanted the boring part to be deterministic and the model part to be optional and small.

rankprobe audit <url-or-file> fetches the page (or reads a local HTML file) and runs 36 checks: status and redirect chain, meta robots and X-Robots-Tag, canonical, hreflang, title and description length, one h1, heading order, missing alt, lang, viewport, page weight, Open Graph and Twitter tags, JSON-LD syntax and required fields, plus robots.txt, sitemap and llms.txt at the origin. Every finding carries the evidence it found.rankprobe robots <url> prints which AI crawlers (GPTBot, OAI-SearchBot, ChatGPT-User, ClaudeBot, Claude-User, PerplexityBot, Google-Extended, CCBot, Applebot-Extended) robots.txt allows or blocks. It uses its own matcher: longest match wins, Allow wins ties, * and $ work.rankprobe plan <url> runs the audit and sends --max-pages N crawls same-origin links and flags duplicate titles and descriptions. --fail-under N exits 1 for CI. --sarif writes SARIF 2.1.0 for GitHub code scanning. --json is there for scripts.

pip install git+https://github.com/bayuh85/rankprobe
rankprobe audit https://your-site.dev

Here is the crawl I ran on the blog, 15 pages:

rankprobe audit https://bayuh85.github.io/atmorouter-blog/ --max-pages 15

Grouped by check id, the findings were:

Each finding shows the actual value, so you don't have to go and look:

On-page
  warn   meta-description        Meta description is 197 characters; expected 50-160.
         url: https://bayuh85.github.io/atmorouter-blog/posts/applyfit/
         evidence: {"description": "A Python CLI and MCP server that keeps your profile and every job posting as plain files, scores fit offline from 0 to 100, ...", "length": 197}

Two of these deserve a comment, because a good audit tool should tell you things you then decide to ignore:

canonical-target`` robots-txt / sitemap / llms-txt`` bayuh85.github.io/robots.txt), because that's the only place crawlers look. A project site under a subpath can't fix that without a user-site repo. Good to know before you blame the generator. The real fixes are all in one Jekyll layout: shorten the <title> template, cap the description, add a default og:image. One PR, 15 pages fixed.

For comparison, the same command on example.com returns score: 49/100 with no canonical, no h1, no description and no JSON-LD. Running rankprobe robots https://www.nytimes.com shows all nine AI crawlers blocked, which is a deliberate choice on their side. You probably want to know whether your site makes the same choice by accident.

Then I asked for a plan on one post:

rankprobe plan https://bayuh85.github.io/atmorouter-blog/posts/vetskill/
sent 5774 chars (~1444 tokens, estimated as chars/4) to atmo/deepseek-v4.1-flash at https://atmorouter.dev/v1


## 1. robots-txt — create /robots.txt (also fixes sitemap discovery and locks in AI crawler access)
...
## 3. canonical-target — point the canonical at the URL you want indexed
... If the dev.to post should stay canonical (you're treating GitHub Pages as a mirror), leave it ...

## 4. title-length — shorten the title to 15–65 chars
Current title is 170 characters ...
<title>vetskill: scan a Claude Code skill before install</title>

The prompt tells the model to work only from the findings it was given, and the first line tells you exactly how much was sent. 5.7 KB of JSON instead of the 26 KB HTML page. The plan took about 10 seconds.

The repo ships examples/github-actions-audit.yml. The important line is:

- run: rankprobe audit "$TARGET" --fail-under 80 --sarif rankprobe.sarif

Point TARGET at your preview deploy URL, or at a built HTML file in the repo. A PR that adds a noindex or breaks the JSON-LD now fails the check, and the SARIF upload puts the findings in the code scanning tab. The audit only fetches your page plus robots.txt, the sitemap and llms.txt from its origin, and never calls a model.

plan is one short chat call, so it should cost close to nothing. rankprobe defaults to AtmoRouter, a cheap AI API gateway and OpenRouter alternative: 181 models behind one OpenAI-compatible and Anthropic-compatible key, pay per token, averaging 79.8% below official prices when I checked today. The default model, atmo/deepseek-v4.1-flash, is free there, and that's the model in the run above.

It isn't tied to it. Any OpenAI-compatible endpoint works:

RANKPROBE_BASE_URL=http://127.0.0.1:11434/v1 RANKPROBE_MODEL=llama3.2 OPENAI_API_KEY=local \
  rankprobe plan https://your-site.dev

rankprobe reads the HTML the server sends. It doesn't run JavaScript, so a client-rendered SPA will look empty to it, and to a lot of crawlers too. It doesn't measure Core Web Vitals, check backlinks or know anything about rankings. The multi-page score adds penalties across pages, so a 15-page crawl bottoms out fast. Read the per-check counts, not the number. It's the pre-merge lint for SEO, not the whole SEO job.

plan: Checks live in checks.py, each with an id, a severity and a message, so adding one is a small PR. If it catches a stray noindex before your launch, a star helps other people find it.