# Measuring whether AI search names your product: six ways the measurement lied to me > Source: > Published: 2026-10-11 23:34:01+00:00 I build DOS AI (dosai.pro), an AI assistant that answers a small business's customers in WhatsApp, Instagram and Telegram and keeps the leads in a built-in CRM. Search consoles show where a page ranks. Nobody shows you what an AI assistant says when a buyer asks it for a recommendation, so I wrote a script for that. The script is under 300 lines. Getting it to tell the truth took longer than writing it. Here are the six places where it, or the things around it, gave me a wrong picture. Thirty-five questions a buyer would type, in seven groups: choosing a service, a task to solve, price, agencies, our brand, Kazakh, English. Thirty-two of them do not contain our name. Each goes to a model with the web search tool switched on and an approximate location set to our city. For every answer the script records the text and the cited URLs, then counts where we appear and which sites the answer was built from. A run costs about a dollar. I have four runs now: one in mid-September, two on October 10 and one on October 12. The first version counted an answer as "they named us" if the text contained our name. One of the brand questions is "What is DOS AI?". The model answered that it could be AI software for MS-DOS or a denial-of-service attack. Our name was in the text, because the model repeated the question. The script scored it as a hit. Two of three "hits" in that run were echoes. The rule now: for a question that contains our name, only a citation of our domain counts. The old ruler still prints as a second line, so earlier runs stay comparable. `--limit` takes a number. Without one it became `NaN`, `slice(0, NaN)` returned nothing, and the run finished with "named in 0 of 0". At a glance that reads as "nobody names us". A bad flag is now a launch error with exit code 2, and a run with zero answers exits with 1. On October 10 the script ran twice, four minutes apart. Same 35 questions, different answers. In the first run the model answered the brand question from memory. In the second it searched and found a different company with the same name on another domain, then quoted that company's prices as ours. The score was the same both times, so the baseline holds. A difference of one answer between runs is noise, because the model decides for itself whether to search. I only treat a change as real when the unbranded count moves, or when three runs move the same way. A third to a half of the answers involve no search at all: between 11 and 17 of 35, depending on the run. For those questions, nothing you publish this month matters. The answer comes from training data. Between the last two runs we published eight articles on the site and several on outside platforms. The score did not move, and two days is too short for that. I checked each platform anyway, and the reasons turned out to be more specific than "wait". One platform serves every article of a free account with `x-robots-tag: noindex`, and the links I checked there carry `nofollow`. Those articles do not exist for a search engine. Another keeps a new author's first article in a moderation queue with no public URL. I had counted both as "published". The check is one request per article: read the response headers and the robots meta tag. I now do it before counting a publication as a publication. Search Console has an API. It told me Google last fetched our sitemap six days before the run, which is before any of the new pages existed. Of eight new articles Google knew five, found through links. Resubmitting the sitemap is one `PUT` request with a write scope on the same service account that reads the reports. I had assumed that a sitemap with fresh dates gets picked up within a day. The last-download date is in the API response, and it takes one call to look. Our home page carries a `sameAs` list in its structured data: the profiles that are also us. It exists to tie the name to the domain, which matters when another company shares the name. One of those profiles returned 404. A comment in the code said every address had been checked by a live probe, with the date. The check was real. It ran once, two months ago. I wrote a checker that reads the list from the live page and fails on 404 or 410. Its first version reported five profiles, all alive except one. The page lists eleven. A dangling `else` had attached itself to the inner `if`, so the walker skipped most of the tree. I caught it only because I compared the count with the source. A check that prints "all fine" for five items out of eleven is worse than no check. Print the count, and compare it with a number you got some other way. By domain, the assistant finds us and describes the product correctly. By name alone, it finds the namesake. For the 32 questions without our name it does not mention us, and its answers to "which services exist" are assembled from roundup posts on other vendors' blogs. So the problem sits outside our site. We have enough pages. What is missing is pages on other domains that say what DOS AI (dosai.pro) is. That is slower work than writing articles, and the script will show whether it pays off. The next run is in about two weeks. Decide what counts as a hit before the first run, and test the counting rule on an answer where the model only repeats your question. Then check the things you believe are already true: that your publications are indexable, that the sitemap was fetched, that your own links open. Three of the six problems above were in that second group. The product these numbers are about: [dosai.pro](https://dosai.pro). The page written for AI assistants, with the short description of what it is and what it is not, is at [dosai.pro/llms.txt](https://dosai.pro/llms.txt).