# SEtting up AI search rank tracking easily > Source: > Published: 2026-08-14 12:35:44+00:00 Track whether AI search engines (ChatGPT, Claude, Perplexity, Gemini) mention or cite **your** site when users ask buying questions. Self-hosted, scheduled, no servers - a fork of this repo and a couple of API keys is the whole setup. This is the DIY version of what commercial "AI visibility" tools sell for $90+/month: scheduled prompt sampling against web-grounded AI APIs, scored for mentions and citations, trended over time. You cannot see real user conversations with AI assistants. Nobody can - not you, not the $90/month tools. What you CAN do is **sample**: define the buying questions your customers would plausibly ask, put the same questions to the AI engines on a schedule, and measure how often your site appears in the answers. **It is a poll, not a census.** Run it weekly and you get a trend line for "do I rank on AI?" - which is the thing you actually need to manage. Every answer is scored for two distinct outcomes, never conflated: **Mention**- the answer's text names your brand ("...Acme Coffee Gear is a solid choice for beginners..."). The user*saw*you.**Citation**- your domain appears in the answer's cited sources. Your content*fed*the answer, even if the text never named you. Both matter. A mention without a citation means the model knows you from training data. A citation without a mention means your content is doing work for someone else's answer. Absent means neither - and that is the number you are trying to move. - Buyers increasingly ask AI assistants where to buy, what to buy, and whom to trust. Those answers name specific stores and cite specific sites. - You cannot manage what you cannot measure. If AI answers are a channel, you need a number for it, the same way you have one for Google rankings. - Commercial AI-visibility tools exist and are fine - but the core loop (ask, score, trend) is simple enough to self-host for the price of the API calls. This repo is that loop, readable in an afternoon. Four steps, four small files. [ prompts.json](/razz1000/ai-search-rank-tracking-example-repo/blob/main/prompts.json) holds the buying questions. The shipped ones are for a fictional specialty coffee equipment store - **replace them with your own**(see [Writing good prompts](#writing-good-prompts)). ``` { "id": "where-buy-grinder", "prompt": "Where should I buy a good specialty coffee grinder online? I want a shop with real expertise, not just a marketplace.", "tags": ["store-intent", "broad"] } ``` Each engine module takes a prompt and returns the same shape: the answer text plus its cited sources. Web grounding is the point - without it the models answer from training data and cite nothing. ``` js // src/engines/perplexity.ts (the simplest of the three) const res = await fetch("https://api.perplexity.ai/chat/completions", { method: "POST", headers: { authorization: `Bearer ${process.env.PERPLEXITY_API_KEY}`, ... }, body: JSON.stringify({ model: "sonar", messages: [{ role: "user", content: prompt }] }), }); // -> { text: string, sources: string[] } ``` **OpenAI**- Responses API with the built-in`web_search` tool ()`src/engines/openai.ts` **Anthropic**- Claude Messages API with the server-side web search tool; cited sources come back as citations on the answer text ()`src/engines/anthropic.ts` **Perplexity**-`sonar` , search-grounded by default ()`src/engines/perplexity.ts` **Gemini**- Google Search grounding tool ()`src/engines/gemini.ts` Every engine is optional: if its API key is not set, it is skipped with a console note. Any subset works. Model names rot. Each engine file has a single `MODEL` constant at the top with a comment - that is the only thing to update when a provider rotates model versions. The names in this repo were current when it was built. [ src/score.ts](/razz1000/ai-search-rank-tracking-example-repo/blob/main/src/score.ts) checks each answer against the brands in [: a case-insensitive alias match in the answer](/razz1000/ai-search-rank-tracking-example-repo/blob/main/config.json) `config.json` *text*is a mention (the matching snippet is stored so you can read how you were mentioned); a domain match in the *sources*is a citation. ``` { "runs": 3, "brands": [ { "name": "Acme Coffee Gear", "aliases": ["Acme Coffee Gear", "Acme Coffee"], "domains": ["acmecoffeegear.com"] }, { "name": "Prima Coffee (competitor example)", "aliases": ["Prima Coffee"], "domains": ["prima-coffee.com"] } ] } ``` Because answers vary between identical asks, each prompt is asked `runs` times (default 3) per engine, and results are always reported as X out of N runs - never as a binary "you rank / you don't". The example config tracks a fictional brand plus one real retailer as a competitor example, so your very first run produces non-zero data. Replace both with your brand and your actual competitors. Each run appends one JSONL file to [ data/](/razz1000/ai-search-rank-tracking-example-repo/blob/main/data) - one line per (prompt, engine, run) with the full answer text, sources, and scores - and regenerates [. The workflow commits both, so](/razz1000/ai-search-rank-tracking-example-repo/blob/main/REPORT.md) `REPORT.md` **the git history is the time series**. No database. ``` npm install cp .env.example .env # add at least one API key npm run track # sample all engines, write data/ + REPORT.md npm run report # regenerate REPORT.md from existing data (no API calls) ``` The repo ships a GitHub Actions workflow ([ .github/workflows/track.yml](/razz1000/ai-search-rank-tracking-example-repo/blob/main/.github/workflows/track.yml)) that runs weekly and commits the results back. Setup: **Fork this repo****Add secrets**: repo Settings → Secrets and variables → Actions → add`OPENAI_API_KEY` /`ANTHROPIC_API_KEY` /`PERPLEXITY_API_KEY` /`GEMINI_API_KEY` (any subset works)**Edit** for your siteand`prompts.json` `config.json` Every Monday the Action runs the tracker and commits a fresh `REPORT.md` plus the raw data. No servers. You can also trigger it manually from the Actions tab (workflow_dispatch) - do that once after setup to check everything works. The tracker is only as good as the questions you feed it. Rules of thumb: **Real buying intent.**"Where should I buy X" and "what is the best X for Y" - the questions that end in a purchase, not "what is a coffee grinder".**Phrased like a customer.** Write the way a person talks to a chatbot: first person, context, constraints ("for a beginner", "under $800"). Not keyword strings.**Mix broad and long-tail.** Broad prompts ("best home espresso machine") tell you about the big race you are probably losing; long-tail prompts ("which stores sell the Comandante C40 in the US") are where a specialty site realistically appears first.**5 to 15 prompts is plenty.** More prompts means more cost and more report to read. Start small; add prompts when you have a hypothesis to test.**Keep prompt** The trend line is per prompt id - renaming an id starts its history over.`id` s stable. [ REPORT.md](/razz1000/ai-search-rank-tracking-example-repo/blob/main/REPORT.md) has three parts: **Per-brand summary table** (prompt x engine) for the latest session, with a trend arrow against the previous session: | Prompt | openai | perplexity | gemini | |---|---|---|---| | where-buy-grinder | 0/3 M - 1/3 C ↑ | 1/3 M - 2/3 C = | 0/3 M - 0/3 C ↓ | *(Illustrative numbers, not real output.)* `1/3 M` = mentioned in 1 of 3 runs; `2/3 C` = cited in 2 of 3. Expect variance - that is why runs exist. **Trend over time** - the same rates per session, so you can see whether a content push or a new competitor moved anything. **Sources that fed the answers** - every domain the engines cited this session, most-cited first. This is the competitive intel: these sites are being read to answer *your* buying questions. If your domain is not on the list, the list tells you exactly who is there instead of you - and what kind of content (reviews, comparisons, guides) is winning the citations. Mention snippets are stored too, so you can read *how* you were mentioned - "great for beginners" and "had shipping problems" are both mentions. Rough math so nobody is surprised by an API bill. Per tracking session with the default config (5 prompts x 3 runs = 15 calls per engine): **OpenAI**(`gpt-5-mini` + web search tool): the search tool call dominates, roughly $0.01 to $0.02 per call → about**$0.15 to $0.30****Anthropic**(`claude-haiku-4-5` + web search tool): a per-search fee plus tokens - and note that search results are billed as*input tokens*, which is where the money goes on this engine. On Haiku, roughly $0.02 to $0.05 per call → about**$0.30 to $0.75**. A measured warning from running this in anger: on`claude-opus-5` the exact same session cost ~20x more (search-result input tokens at Opus prices), so if you swap the`MODEL` constant into a bigger model, check your first bill.`src/engines/anthropic.ts` **Perplexity**(`sonar` ): request fees plus tokens, roughly**$0.10 to $0.15****Gemini**(`gemini-3.6-flash` + Search grounding): grounded prompts have a free daily allowance at the time of writing; on paid billing roughly**$0.50** So a full four-engine weekly run lands **around $1 to $2**, or a few dollars per month - and under $1 without the Anthropic engine or with a cheaper Claude model. Costs scale linearly: prompts x engines x runs x price per grounded call. Double the prompts, double the bill. Prices change - check each provider's current pricing page before scaling up. Same rules as the rest of this series - no pretending: **This samples the questions YOU define. It cannot see what real users ask or what they were told.** It is a controlled poll that approximates a channel you cannot observe directly.**API answers are a proxy for the consumer apps, not a screenshot of them.** The APIs use the same underlying engines and web grounding as ChatGPT, Perplexity and Gemini, but the consumer apps add their own layers (memory, location, A/B tests). Treat the trend as the signal, not any single answer.**Mentions and citations are different things** and the report never merges them.**Answers are nondeterministic.** The same prompt asked twice gives different answers - that is why every number is X out of N runs, and why the weekly trend matters more than any single session. **Why do results differ between runs?** The models are nondeterministic and the live web results behind them shift constantly. That is not a bug in the tracker; it is the nature of the channel. It is exactly why the tracker asks each prompt several times and reports X/N instead of a checkmark. **Why not scrape the ChatGPT website instead?** Scraping the consumer apps breaks their terms of service and breaks *technically* every few weeks. The APIs are the legitimate, stable path, and they use the same engines and grounding. A proxy you can run every week beats a perfect measurement you can run never. **Can I track competitors?** Yes - that is why `config.json` takes an array of brands. Add each competitor with their aliases and domains and the report shows all brands side by side. Watching a competitor's citation rate climb on your key prompts is usually the earliest signal you will get. **How is this different from AI crawler logs?** Crawler logs (GPTBot, PerplexityBot etc. in your server logs) tell you your site is being *read*. This repo tells you whether your site is being *surfaced* in answers. Being read is necessary but not sufficient - the two measurements are complementary halves of the same funnel. 📺 **YouTube walkthrough:** [Track How You Rank on ChatGPT & Claude - for free](https://youtu.be/dk2LoGmvARU) - how the tracker works, full setup with "Use this template", and real results (Prometora vs Sharetribe on AI search). This repo is the "answer side" of AI visibility: are you surfaced when people ask? The other half is the "crawl side": are AI engines reading your pages at all? [ Prometora](https://www.prometora.com) - a marketplace builder with AI Visibility built in - tracks the crawl side server-side, per listing: which AI crawlers fetched which pages, trended over time. See [their AI crawler research](https://www.prometora.com/learn/ai-crawler-data)for what that data looks like at scale. The crawl side is built into Prometora today; this repo is the answer side, self-hosted. MIT - see [LICENSE](/razz1000/ai-search-rank-tracking-example-repo/blob/main/LICENSE). Questions, ideas, corrections: [rasmus@prometora.com](mailto:rasmus@prometora.com)