{"slug": "build-your-own-ai-visibility-tracker-without-a-subscription", "title": "Build Your Own AI Visibility Tracker (Without a Subscription)", "summary": "An engineer at ATI Lab built a DIY AI visibility tracker in about 200 lines of Python that polls Google AI Overviews for a fixed prompt set, records cited domains and organic positions, and writes dated JSON. The 23-prompt run cost $0.092 in API credits, compared to commercial tools starting at $99–$165 per month. The tracker captures three key fields—citations, organic position, and trigger rate—and the author emphasizes the importance of a well-designed prompt set and using the load_async_ai_overview parameter to avoid false low trigger rates.", "body_md": "*Originally published on the ATI Lab blog. Full context, including our 23-prompt AI Overview study, is there.*\n\nYou can build a working AI visibility tracker in about 200 lines of Python. It polls Google AI Overviews for a fixed prompt set, records which domains get cited, captures your own organic position on the same query, and writes dated JSON so months compare cleanly. Our 23-prompt run cost **$0.092** in API credit. The commercial tools that rank for this term start at $99–$165 a month. Here is the build, and an honest account of what it cannot do.\n\nThis guide is written from our own tracker, which we run against atilab.io. The code below is the code we actually execute — not pseudocode. Research and drafting were AI-assisted; every number is from our own run files or a source linked in-line, and a human checked each one.\n\nThree things, and most dashboards report only the first:\n\nEverything else — sentiment, share-of-voice indices, a single blended visibility score — is a presentation layer over those three fields. You can add it later. You cannot add the third field later, because it has to be captured in the same request, on the same day, against the same SERP.\n\nOur numbers, from the run files. The Google signal uses DataForSEO's live advanced SERP endpoint, which returned an API-reported cost of **$0.0040 per query** on every call in our run. Twenty-three prompts is $0.092. Run it monthly and the year costs about $1.10.\n\nFor comparison, published pricing on the tools currently ranking for \"ai visibility tracker\" (checked 12 August 2026): [Rankscale](https://rankscale.ai/pricing) lists Pro at $99/month for 1,200 credits, Growth at $385/month, Enterprise at $780/month. [Semrush](https://www.semrush.com/pricing/) bundles AI visibility into its main plans — Starter at $165.17/month billed annually, with 50 prompts tracked daily, rising to 200 daily prompts on Advanced at $455.67/month.\n\nThose are not equivalent products and we are not going to pretend they are. The subscriptions sample answers from ChatGPT, Gemini, Perplexity, Claude and others, schedule the runs, store history, and produce reports somebody else maintains. The DIY build covers two surfaces and produces a JSON file. What the cost comparison establishes is narrower and still useful: *the underlying measurement is cheap*. Price the subscription against convenience and coverage, not against access to data you could not otherwise get.\n\nThe single biggest determinant of whether your tracker is useful is the prompt set, and it is the part no tool can do for you. Ours is 23 prompts: 4 branded, 19 non-branded, tagged by funnel stage and by the page each one should ideally send traffic to. Each record carries two forms of the same intent:\n\n```\n{\"id\": 6, \"type\": \"non-branded\", \"funnel\": \"MOFU\", \"pri\": \"P1\",\n \"target\": \"/ai-agent-cost\",\n \"prompt\": \"What does it cost to build and run AI agents for a business?\",\n \"query\": \"how much does it cost to build ai agents for business\"}\n```\n\nThe `prompt`\n\nis the conversational form a person types into an assistant. The `query`\n\nis the search-shaped form you send to Google. They are different strings on purpose, and collapsing them into one field is the most common way these projects produce uninterpretable data.\n\nTwo rules that matter more than the prompt wording:\n\n`target`\n\nfield tells you instantly which page has failed, rather than starting a research project.One request per query. The parameter that matters is `load_async_ai_overview`\n\n— without it the AI Overview block is frequently missing from the response even when it fires on the live SERP, and you will conclude your trigger rate is low when it is not.\n\n```\nres = dfs.call(\"serp/google/organic/live/advanced\", [{\n    \"keyword\": p[\"query\"],\n    \"location_code\": 2840,          # US\n    \"language_code\": \"en\",\n    \"depth\": 20,\n    \"load_async_ai_overview\": True,\n}])\n```\n\nThen walk the items once, pulling both signals out of the same response:\n\n```\nfor item in res[0].get(\"items\") or []:\n    itype = item.get(\"type\") or \"\"\n    if itype == \"ai_overview\" and not out[\"aio_present\"]:\n        out[\"aio_present\"] = True\n        refs = item.get(\"references\") or []\n        out[\"aio_refs\"] = [host_of(r.get(\"url\") or \"\") or (r.get(\"domain\") or \"\")\n                           for r in refs]\n        out[\"aio_cites_brand\"] = any(BRAND in d for d in out[\"aio_refs\"])\n    elif itype == \"organic\" and out[\"organic_rank\"] is None:\n        if BRAND in (item.get(\"domain\") or \"\"):\n            out[\"organic_rank\"] = item.get(\"rank_absolute\")\n```\n\nNote the fallback on the reference: some entries carry a `url`\n\n, some carry only a `domain`\n\n. Read one field and you will silently drop citations.\n\nBecause citation is mostly downstream of ranking. In our study we took six non-branded queries where an AI Overview fired, captured all 31 citations, and checked each cited domain against the classic results for the identical query: 74% also ranked in the organic top 20, 48% in the top 10, and only 25% were cited without ranking on the first two pages. Small sample, and we would not present it as a universal law — but the direction is clear enough to plan against. Google is largely drawing its citations from a pool it has already decided to rank. A tracker that reports citations without reporting your position on the same query has hidden the causal variable, and its output is a number you cannot act on.\n\nWith both fields, every row falls into one of four cells, and each cell has a different instruction attached:\n\nThat distribution is the reason we are unsentimental about AEO tactics. Nineteen of twenty-three prompts told us the same thing: rank first. We published the full findings in our [measurement of 23 AI Overviews in our category](https://www.atilab.io/blog/we-measured-23-ai-overviews), including the queries where Google ignored vendor sites entirely.\n\nThe second surface worth polling is the embeddings layer that many AI applications query before generating an answer. We use Exa: send the conversational `prompt`\n\nform, and record whether your domain comes back and at what rank.\n\n``` python\ndef check_exa(p, api_key):\n    out = {\"exa_rank\": None, \"exa_url\": None, \"exa_top\": []}\n    res = exa_search(p[\"prompt\"], api_key)\n    for i, r in enumerate(res.get(\"results\") or [], 1):\n        if BRAND in host_of(r.get(\"url\")):\n            out[\"exa_rank\"] = i\n            break\n    return out\n```\n\nThis signal behaves very differently from the Google one and that is the point of having it. We surfaced in retrieval on 5 of 23 prompts, at ranks 1, 1, 1, 1 and 6 — four first places on a set where Google cited us three times, all branded. Retrieval visibility and citation visibility are not the same thing, and a tracker with one surface will tell you a confident, incomplete story.\n\nA small decision with a large effect on the competitor map. One AI Overview citing five pages of the same domain is still one prompt's worth of visibility. Count it once:\n\n```\nfreq = {}\nfor r in rows:\n    for d in set(r.get(\"aio_refs\") or []):   # set() — one prompt, one vote\n        freq[d] = freq.get(d, 0) + 1\n```\n\nSkip the `set()`\n\nand any domain with a habit of multi-page citation looks like it owns your category. With it, our map reads cleanly: YouTube appeared in 11 of the 21 triggered AI Overviews, Reddit in 8, LinkedIn in 4. Counted as raw references those three domains are 37 of 174 citations, 21%. That is a meaningful minority, not the domination the common advice implies — the remaining 79% went to ordinary vendor and agency content, spread across 116 distinct domains. The citation pool is not closed, which is the encouraging half of the result.\n\nThe whole value of this thing is the month-over-month diff, so the output directory is the date and nothing overwrites anything:\n\n```\noutdir = os.path.join(OUTDIR, datetime.date.today().isoformat())\nos.makedirs(outdir, exist_ok=True)\njson.dump({\"date\": date, \"location_code\": loc, \"language_code\": lang,\n           \"brand\": BRAND, \"results\": rows},\n          open(os.path.join(outdir, \"results.json\"), \"w\"), indent=1)\n```\n\nWrite a flat `summary.csv`\n\nalongside it. JSON for diffing, CSV for the person who wants to sort it in a spreadsheet.\n\nFour things, all worth knowing before you spend an afternoon debugging them:\n\n`None`\n\nrather than crashing the run — losing one query is fine, losing the other 22 is not.`url`\n\nfirst and fall back to `domain`\n\n.\n\n``` python\ndef call_dfs(path, payload, attempts=3):\n    for n in range(attempts):\n        try:\n            return dfs.call(path, payload)\n        except Exception as exc:\n            if n == attempts - 1:\n                return None\n            time.sleep(2 * (n + 1))\n```\n\nBe straight about this, because the vendors ranking above this page are not always. **No public API exposes what ChatGPT, Claude, Gemini or Perplexity actually said to a real user in a real session.** Every tool reporting \"your ChatGPT visibility\" is running its own prompts through an API and treating the output as a sample. That is a legitimate proxy and often a useful one. It is a sample of what a model tends to say, not a record of what your buyers were told.\n\nYou can build that proxy yourself too — send your prompt list to an assistant API, check whether your domain appears in the answer, repeat n times per prompt because the outputs vary. Just cost it honestly: a meaningful sample means several runs per prompt per platform, and that is where the DIY approach stops being nine cents and starts approaching a subscription. This is the point at which buying is a reasonable decision.\n\nMonthly, on the same day, with an unchanged prompt set. More frequent runs mostly measure SERP volatility. Re-run early only after a specific content or technical change you want to attribute.\n\nThen read the report as a work queue, not a scoreboard. Rows in the bottom-right go to ordinary ranking work. Rows in the top-right — you rank, you are not cited — go to a formatting pass on an existing page, which is the cheapest work on the list. Rows in the top-left get a diary note and a re-check. That is the whole method, and it is the same discipline we apply in [AI consulting engagements](https://www.atilab.io/ai-consulting), where the readiness audit exists to say where AI should *not* be applied as much as where it should. A measurement baseline first; a build only where the baseline says one is justified. We wrote about the same trap on the delivery side in [measuring AI agents on engineering teams](https://www.atilab.io/blog/ai-agents-developer-kpis) — the metric that is easy to collect is rarely the one that changes a decision.\n\nA tool that measures whether AI answer engines mention and cite your brand when people ask questions in your category. A useful one captures three fields per query: whether an AI answer triggered at all, whether you were cited, and which domains were cited instead. Adding your classic organic position for the same query turns those observations into an instruction.\n\nNo — commercial tools start around $99–$165 a month and cover more surfaces than a script does. Building it yourself is worth it when you want the raw data, control of the prompt set, and month-over-month files you own. The build described here is a single stdlib Python file plus one SERP API account.\n\nOur 23-prompt Google run cost $0.092 in API credit, at an API-reported $0.0040 per query, plus retrieval-API usage on its own plan. Run monthly, that is a little over a dollar a year for the Google signal. The real cost is the hour you spend building the prompt set properly.\n\nNot in real user sessions. No public API exposes what an assistant told a specific user. Tools reporting this are sampling their own prompts through an API, which is a reasonable proxy that should be labelled as one.\n\nMostly not, on our data: 74% of the AI Overview citations we traced also ranked in the classic organic top 20 for the same query. A real but minority share of citations goes to pages that do not rank — that is where answer-shaped formatting earns its keep. Treat AEO as a formatting and structure layer on top of ranking work, not as a replacement for it.\n\nTwenty to thirty is enough to be diagnostic and small enough that you will actually keep it current. Weight it towards non-branded commercial questions, tag each prompt with the page that should serve it, and never change a prompt once it has history.\n\nThe reason we built this rather than bought it was not the $99. It was that we wanted the raw rows, including the ones that made us look bad — and on our first run, nineteen of them did. If you would like a read on where AI visibility sits against the rest of your growth priorities, [book a strategy call](https://www.atilab.io/booking) and we will go through your own numbers rather than ours.", "url": "https://wpnews.pro/news/build-your-own-ai-visibility-tracker-without-a-subscription", "canonical_source": "https://dev.to/lachezar_dimitrov_ec7c7d9/build-your-own-ai-visibility-tracker-without-a-subscription-9n8", "published_at": "2026-08-14 07:05:01+00:00", "updated_at": "2026-08-14 07:46:30.938853+00:00", "lang": "en", "topics": ["developer-tools", "ai-products", "artificial-intelligence", "natural-language-processing"], "entities": ["ATI Lab", "DataForSEO", "Rankscale", "Semrush", "Google AI Overviews"], "alternates": {"html": "https://wpnews.pro/news/build-your-own-ai-visibility-tracker-without-a-subscription", "markdown": "https://wpnews.pro/news/build-your-own-ai-visibility-tracker-without-a-subscription.md", "text": "https://wpnews.pro/news/build-your-own-ai-visibility-tracker-without-a-subscription.txt", "jsonld": "https://wpnews.pro/news/build-your-own-ai-visibility-tracker-without-a-subscription.jsonld"}}