Comparing AI and Google searches among the same group of users over a month to examine how their behavior differs between platforms. #
TL;DR: I matched 16,310 users across two Datos panels to compare how each user searched with Google and LLMs and understand how we need to track prompts differently than keywords. On Google, 87% of searches were lookups, and only 8% of their searches were questions. On AI, questions were 27% of prompts, which is 3.3x the Google rate. For the same need, AI prompts were almost twice as long as Google searches on average. When asking AI, users added over 6x more personal detail. While AI Search only made up 2.7% of the user’s queries, the prompt usually stood on its own, with no matching Google search before or after. The results showed a massive difference in user behavior on these two search surfaces, enough that traditional Google search behavior is a poor predictor of AI Search behavior.
—–
Tracking AI visibility is only as powerful as the prompt set itself. How is your team choosing what to track? Some teams expand keyword lists into questions, some generate prompts from personas, and some just ask an LLM to do the work. The strongest starting point is data on what your audience actually asks: sales calls, lead inquiries, customer questions, and FAQs can all tell you what people care about and the language they use.
But none of these methods tells you, on its own, whether the prompts you’re tracking look like the prompts users actually give to LLMs.
Mike King made the case recently that there is no accuracy in AI search tracking, only precision, and that the prompt set is one of the methodological choices that shapes the number. In the research he cites, brand visibility fell by about half when a prompt’s wording drifted far enough from the original. If small changes in wording can move the measurement that much, the question isn’t just whether your prompt set covers the right topics. It’s whether it resembles the way people actually use AI.
Instead of arguing about which method is right, I looked at the behavior itself. I took the same 16,310 users, followed what they typed into traditional Google search and into AI search for a month, and compared them, user by user. The findings showed a much bigger gap between the two than a keyword list would suggest.
What I Wanted to Test #
I’ve developed my own methodology for building prompt sets for iPullRank clients, one that accounts for topic coverage, funnel stage, persona, and contextual qualifiers. But any prompt set, mine included, is only useful if it reflects how your audience actually behaves and the language they use. A dashboard can look equally authoritative whether it’s reporting on real user behavior or on queries nobody types.
Primary research question:
How different do users behave on traditional Google search vs with LLMs?
Secondary questions:
- When the same person searches Google and prompts an LLM, how different is the query intent?
- How do AI prompts and Google searches differ in length?
- How much qualifying context (role, budget, tool stack, constraints) do users add in each search surface?
- How much of a user’s total searching actually happens on LLMs right now?
- Can you predict how someone prompts an LLM from how they search on Google?
The Hypothesis #
Even before I worked in SEO, I knew I had to kind of game Google into giving me the result I actually wanted for traditional search. That meant typing a specific way: short phrases instead of sentences, stripped down to the nouns I figured would match a page. Now, given my career, I know those are keywords, and that there was a real reason I had to type like that.
With an LLM, I know I can say as much as I want and it doesn’t even have to make sense. I front-load my background, goal, budget, constraints. I make typos and keep going. I pile on detail until I get what I’m after.
Remember typing dog + tree + image into Google? That progressed to “dog under tree,” and then you clicked over to the Images tab. The same goes for buying decisions. On Google, you type “best laptop,” skim a few listicles, then narrow down your options yourself across two or three more searches: “best laptop for video editing,” then “best laptop for video editing under 1500.” With an LLM, you skip all of that and open with “I’m a freelance video editor on a $1,500 budget, I mostly cut 4K in Premiere, and I need something I can carry to client shoots, what should I buy?”
It’s the same person with the same goal, but it isn’t the same behavior, and the second version carries things a keyword export never will: a job, a budget, a software stack, a use case. I expect LLM prompts to be longer and filled with much more detail than a Google search.
Methodology #
Dataset
I combined two Datos panels, matched at the user level, to compare how the same 16,310 users behave on traditional Google search versus AI search.
The first dataset is the Google clickstream panel behind my analysis of 13 billion Google searches. The second is the Datos AI Prompts and Queries panel, which recorded the prompts that real users typed into Perplexity, Copilot, and you.com. Both panels are filtered the same way: US, desktop, November 2024.
Both panels assign the same user ID, and 16,947 of the 33,829 users in the AI panel (50.1%) also appear in the Google panel that month. A 50% overlap between two randomly assigned ID spaces doesn’t happen by chance, so the panels clearly share an ID system. I confirmed the match a second way: of the 14,325 matched users with a recorded city, 100% sit in the same city in both. These are desktop panels, so an ID identifies a device or browser profile rather than a person, which is the right level of caution to read these numbers with.
What I Measured
For each of the 16,310 matched users, I compared traditional Google and AI search activity on five aspects:
- Intent: The type of each query/prompt (navigational, informational, etc.)
- Length: How many words the user query/prompt contained
- Context: How much background detail users added to their queries
- Share: The portion of each user’s total search activity that occurred on each surface
- Predictability: Whether a user’s traditional Google search style predicts their AI search style
Analysis Approach
A small number of panel members ran the same query hundreds of times, which is automation rather than behavior (the busiest single user logged 13,899 queries; the median person logged 2). I counted each unique person-plus-query combination once, so a script firing “best crm software” 500 times carries the same weight as one person typing it once.
When I compared individual behavior (how long each user’s queries were and whether their Google phrasing predicted their AI phrasing), I first calculated each user’s own numbers and then averaged them, so every user counts once whether they ran 3 queries that month or 300.
I sorted every query by what the user was doing, using the words the query opens with and a few structural cues. Queries fell into 13 types, which I grouped into three jobs: Find (look up a keyword, navigate to a site, shop, or search by location), Ask (a question, a how-to, a comparison, a question that opens with your situation, or one that opens by talking about yourself), and Do (hand off a task, create this, summarize or fix this, or “act as”). The same rules ran on both surfaces. I checked them against real examples from each side and fixed the ones that misfired, like song lyrics on Google being counted as “act as”.
Then I checked that classification from the opposite direction. I ignored the 13 types and counted the literal first word of every query. In AI search, question words like “what” and “how” rose to the top on their own, with “i,” “can,” “write,” and “give” close behind. On Google, the top openers were generic nouns and brand names.
I also removed two kinds of queries that would have flattered AI. First, 15 prompts that AI tools suggest users click (like Copilot’s “what are some daily habits for happiness?”) showed up word-for-word across dozens of users, so I took them out. Second, some users pasted a Google search query straight into the LLM within the same minute. I reported that as its own number and excluded those copies from the comparisons.
Every headline difference is statistically significant (p < 0.001), and each holds when I restrict the analysis to users with at least 5 queries on each side, where per-user rates are most reliable.
Finding 1: Questions are 3.3x more common in AI search than on Google #
To break down user intent, I divided searches into three categories:
- Find: keyword / entity, navigational, transactional, local
- Ask: informational question, how-to, recommendation / comparison, scenario / context-led, first-person context
- Do: request / command, task / generation, pasted context, role prompt
On Google, 84.8% of queries were keywords or entities, 8.1% were informational questions, 2.1% were how-tos, and every other type came in under 2%. Hand-offs like “write,” “create,” or “summarize” barely registered on Google (under 0.5% combined). On LLMs, the same users used a keyword or entity 46.3% of the time, asked an informational question 26.8% of the time, made a request 6.9% of the time, and opened with first-person context 5.7% of the time. Out of the 13 query types, AI search spreads much more evenly than Google does.
Google trained all of us to compress a need into a few nouns, so SERP research is built on keywords that represent these n-grams. LLMs give users more room to spell out what they actually want. They’re able to create, write, compare, and hand off tasks. So, the breakdown of use cases changes. “Write me a comparison of these three tools for my team” has no Google search equivalent, so no amount of search data will surface it. If your prompt set only translates keyword behavior into questions, you’re leaving out a meaningful part of what users actually do with AI.
Finding 2: AI prompts are almost twice as long as Google queries for the same need #
Keep in mind that this finding drops every Perplexity row, since Perplexity truncates prompts to 30 characters in this dataset. That leaves the 5,138 users out of 16,310 who wrote at least one full-text AI prompt.
Across all their queries, the same users averaged 8.4 words in AI search and 4.2 on Google. But the average hides something: the typical user writes about 3 words on both. A minority of users who write long, detailed prompts pull the AI average up.
The difference becomes clearer when you hold the need constant. I paired each person’s Google search with an AI prompt about the same thing (same user, shared key terms), which gave 267 matched pairs. For the same need, the AI version ran a median of 9 words vs. 5 on Google search, and it was the longer of the two 60% of the time.
Look at these actual examples of how four users typed in the two search surfaces:
With LLMs, people talk the way they’d talk to a person, adding the context (a 32-person wedding at an Airbnb), the constraint (a 3.5mm plug, not 2.5mm), and the actual question. With Google search, they shorten their queries to a minimum.
Jan Ehrlinspiel, Malte Landwehr, and Tomek Rudzki analyzed nearly 38,000 responses across five engines and found that brand visibility held steady when a prompt was reworded close to the original, and fell by about half when the phrasing drifted further from it. A 4-word prompt and a 28-word prompt about the same need aren’t two versions of one measurement. They’re two different measurements. The takeaway isn’t to make every tracked prompt long, because most users still type short queries into LLMs too. It’s that a prompt set should carry both shapes: the quick lookup and the detailed version the same person writes when the need gets specific.
Finding 3: About 1 in 12 AI prompts carries personal context, vs. about 1 in 100 Google searches #
Length alone doesn’t tell you what those extra words are doing. With AI search, users aren’t just writing longer queries; they’re giving the model information about themselves that can change what a useful answer looks like.
Measuring that turned out to be hard. Keyword rules couldn’t do it: a regex caught fewer than half of the real cases and flagged plenty of false ones. So I hand-labeled a random sample of unique queries from the same users and counted one specific thing: whether the query revealed the asker’s role, situation, or purpose.
About 1 in 12 AI prompts did (8.0%). On Google, it was about 1 in 100 (1.2%). In iPullRank’s Personal Intelligence experiment, personal signals changed which brands AI Mode recommended. When someone writes “I used to use Mint, but it’s gone now,” they’ve told the model what they were using, why they’re switching, and what the replacement has to do. “Best budgeting app” asks a different question and can get a different set of brands back.
Finding 4: AI made up under 3% of searches, and 96% of prompts had no matching Google query #
How often did the same person use AI search over traditional Google search?
The typical user in the panel who used both surfaces sent 2 AI prompts in November vs. 92 Google searches. LLMs accounted for 2.7% of their searches, and 62.7% of users did less than 5% of their searches on LLMs.
When users did use AI search, most prompts had no matching Google search: for 96% of AI prompts, the same user didn’t search that topic on Google within 30 minutes.
Zoom out to the whole month, and users clearly use both. About 1 in 5 brought the same need to traditional Google search and LLMs at some point, usually within an hour, with no fixed order: Google came first 51.9% of the time, so neither surface is reliably the starting point. A small group, 3.7% of users, ran an identical query on both within the same minute, which is one search carried straight over rather than two separate needs.
Most of what happens in AI search leaves no footprint in Google data. You can’t reverse-engineer these prompts from Search Console or keyword tools because, for most of them, the Google query never happened. That’s why iPullRank’s measurement framework treats visibility as a channel metric to validate against real outcomes, and why the prompts you track need to be modeled on AI behavior.
Finding 5: Google habits explain only 4% of how someone prompts AI #
I wanted to know whether someone’s Google habits could predict how they prompt an LLM. For each person I measured how often they opened a query four ways: as a question, as a command, by talking about themselves, or as a bare keyword. These are the opening shapes rather than the 13 intent types, since what carries over between surfaces is how someone starts a query. Then I checked whether their Google rate predicted their AI rate for the same behavior.
I included only users with at least 5 queries on each side (3,643 users), since fewer than that gives noisy rates.
No habit carried over in a meaningful way. The strongest was the keyword habit: how often someone typed a bare keyword on Google explained 8.2% of how often they did it on AI. Question phrasing explained 6.3%. Commands explained 0.4%, and opening by talking about themselves 0.1%, which is effectively nothing. Take all four habits together and someone’s full Google profile explains 4% of their AI behavior.
A plainer way to say it: if Google habits carried over, a person’s AI behavior should look more like their own Google behavior than like a stranger’s. It did 56.9% of the time, against a coin-flip baseline of 50%.
The sharpest version of this is the users who never phrased a Google search as a question at all. There were 796 of them, 22% of the group. On AI, 41% of them asked at least one question, and questions made up 13.5% of their prompts on average. Users who do ask Google questions averaged 32.7%. So the habit matters a little, but the surface matters much more.
What This Means for Brands #
If your prompt set is supposed to represent how your audience uses AI, then the set itself needs to be treated as a research input, not a list of keywords with question marks added. Here’s what that means in practice:
- Mirror the job mix. In AI search, roughly half of prompts are lookups and 4 in 10 are asks. If your set is almost all lookups, it’s modeling traditional Google search.
- Carry both shapes. Include the short version of a need and the detailed version, because they’re different measurements. The same need can produce materially different prompts depending on how much context the user gives the model.
- Write context in. About 1 in 12 real prompts says who’s asking or why. Persona prompts should reflect actual roles, situations, and purposes (the way iPullRank’sprompt recipes do), not just demographic labels.
- Use search data for topics, not phrasing. It tells you what your audience cares about, but it can’t tell you how they’ll ask.
- Hold the set still once it’s built.In AI search, there’s no accuracy, only precision : a representative set you keep stable tells you more over time than a perfect one you keep rewriting.
What This Study Does and Does Not Prove #
What This Study Supports
- The same users behave measurably differently on LLMs than on Google: a different mix of jobs, longer queries for the same need, and far more personal context.
- At the person level, search behavior is a weak predictor of prompt behavior.
- In late 2024, AI accounted for a small share of searches among users who used both, and most AI prompts had no matching Google search.
What This Study Cannot Prove
- That a given prompt set produces different visibility numbers. I measured the inputs, not the answers.
- How users behave today. The data is from November 2024 on US desktops.
Caveats
- 70% of Perplexity rows are truncated to 30 characters; 97k rows are non-Perplexity
- The panel skews heavily to ages 35-44 (65%), males (65%), and US desktop (99.9%)
- ChatGPT only makes up 0.1% of the AI panel
- The typical user ran 92 Google searches and 2 AI prompts in the month.
The iPullRank POV #
What I found is simple: the same person doesn’t search the same way when they move from traditional Google search to AI Search. They stop compressing. They ask questions, hand off tasks, and explain what they’re trying to do.
That has a direct implication for prompt tracking. The prompt set is part of the measurement instrument. If the prompts don’t resemble how users actually use AI, the number you’re measuring can be precise without being particularly useful.
So however you build your set — keyword expansion, personas, LLM generation, customer research, or some combination — test it against real behavior before you trust what it tells you.