Tactical Solutions For The Biggest AI Search Measurement Challenges

STAT Director of Search Product Strategy Tom Capper told a webinar that brands should track AI answer mentions as entities rather than treating citation counts as the key performance indicator, arguing that citations have URLs while "mentions have entities." Capper recommended measuring presence, recommendation and context, and SERP prominence alongside business outcomes, and building a broader, more representative prompt sample using sources such as People Also Ask questions, with STAT collecting answers through the web interface because API responses can differ from what users see. He also advised using citations to investigate which third-party sources shape competitor recommendations rather than as a headline metric.

An AI answer can recommend your product without linking to your website. A customer might then search for your brand, visit directly, or buy offline. The purchase still matters to the business, even when analytics cannot connect it to the earlier answer. That was the measurement problem Tom Capper, Director of Search Product Strategy at STAT, addressed in Tactical Solutions For The Biggest AI Search Measurement Challenges regWatch . In this webinar, Capper explained why zero-click search was also an attribution challenge, not simply a traffic problem. His response was to measure visibility alongside business outcomes regWatch rather than expect clicks to describe the entire customer journey. Separate Brand Mentions From Source Citations Capper illustrated the distinction with an AI-generated car recommendation. Audi was the brand being recommended, while the cited links pointed to other websites. A report focused only on whether Audi’s domain earned a citation would miss the recommendation itself. The implication was practical: define the brand entities you want to track, including relevant sub-brands, rather than assume your domain captures every valuable appearance. Apply the same discipline to competitors so the comparison is meaningful. Capper drew the distinction directly: “So yeah, citations have URLs. That’s still valuable tactical data, but I don’t think it should be your KPI. Mentions have entities.” — Tom Capper. Watch the webinar, 24:10–24:30 regWatch . For his measurement approach, being mentioned or recommended was the outcome to understand; the cited URLs helped explain which sources might be shaping it. Build A More Representative Prompt Sample Different answers to the same prompt do not make AI visibility impossible to measure. Capper treated that variability as a sampling problem, drawing a comparison with the location- and device-specific results SEOs already tracked. A small set of prompts can give an unstable picture, particularly when the business already monitors a much larger keyword set. The goal was not to reproduce every individual’s exact conversation. It was to sample the relevant situations broadly enough to recognize patterns. That also meant asking realistic questions. Capper cautioned against simply putting a keyword list into a prompt tracker. He suggested People Also Ask questions as one possible starting point, then considering the personas, locations, budgets, and needs behind those questions. The webinar’s prompt-measurement discussion regWatch also addressed why the environment used to collect an answer mattered: Capper said STAT used the web interface because API responses could differ from what users saw. Use Citations To Investigate What Changed De-emphasizing citations as a headline KPI does not mean ignoring them. They can make campaign work more concrete. Capper described starting with pages cited when competitors were recommended. Those pages revealed third-party sources worth investigating, such as review sites or industry publications. A team could then work to improve its representation on relevant sources and monitor whether those sources appeared alongside changes in brand recommendations. This created a more useful question than whether the citation count increased: which sources changed, what work did the team do, and how did the tracked answers change afterward? Keep the prompt sample and collection context in view when making that comparison, rather than treating an isolated answer as the whole result. Add Visibility Metrics Alongside Business Outcomes Capper recommended a broader set of leading indicators to support, not replace, traditional performance reporting: - Presence: Whether the brand appears in the relevant search result or AI answer. - Recommendation And Context: Whether the answer recommends the brand, merely names it, or associates it with a particular strength or problem. - SERP Prominence: In traditional search, how far down the page a result appears and how much visible space it occupies. The last distinction mattered even when an organic ranking had not changed. Capper showed examples where a lower-ranked result was visible above the fold while a higher-ranked result was pushed below it. Position alone missed that difference. He also urged caution with sentiment scores. A chart that swings sharply among positive, neutral, and negative may be amplifying small differences in otherwise neutral answers. The qualitative themes behind a mention can be more useful than a volatile summary score. Use Surveys To Improve The Attribution Story During the Q&A, Loren Baker asked how teams could connect this visibility to a customer journey when multi-touch attribution could not observe the earlier exposure. Capper’s answer was to combine evidence rather than expect one analytics model to explain everything. Post-purchase surveys could reveal customers who said an AI recommendation helped them discover or choose a brand. Campaign tracking could separately show whether that brand was being recommended more often. He suggested passing survey answers into analytics as a dimension so the team could compare self-reported discovery with the channel credited by analytics. He also warned that showing survey options in the same order could favor the first option. This was a fuller account of influence, not a promise to reconstruct every person’s path or to classify all direct traffic as AI-driven. Put A More Useful Measurement Plan In Place The full session regWatch connected these measurement choices with examples and audience questions. To put the discussion to work, start with a few concrete changes to your next reporting cycle: - Audit What Your Report Misses: Identify where the current dashboard measures a visit or domain citation but not a brand mention or recommendation. - Define A Consistent Comparison: Document your brand and sub-brand entities, competitors, prompt contexts, and collection environment before comparing results over time. - Connect Source Research To Campaign Work: Record the cited pages you investigate or influence, then monitor how those sources and relevant brand recommendations change. - Bring Business Evidence Back In: Compare visibility trends with revenue, conversions, and post-purchase survey answers instead of presenting visibility as a substitute for commercial results. Join Us For Our Next Webinar AI Shopping: Get Your Products Found in Google & ChatGPT Join us as Chris Sachs, VP of Value Enablement, and Aris Vrakas, Product Lead, at seoClarity, show how AI agents already decide which products shoppers consider.