# Simulate Google's Query Fan-Out to Find Your Content Gaps (Python) > Source: > Published: 2026-10-22 06:00:00+00:00 When someone asks Google's AI Mode "what's the best way to host a Next.js app for a small team?", Google doesn't run one search. It splits the question into sub-questions and runs those in parallel. Google calls this **query fan-out**. Its docs say AI Overviews and AI Mode "may use a 'query fan-out' technique, issuing multiple related searches across subtopics and data sources, to develop a response" ([Search Central](https://developers.google.com/search/docs/appearance/ai-features)). Microsoft lets you see this happening. Bing Webmaster Tools' AI Performance report lists **grounding queries**, the background queries Copilot generated before citing your pages. They're shorter and more specific than what users type. That changes what "covering a topic" means. Your page isn't competing for one query. It's competing for 5–15 machine-generated ones, and every sub-question you skip is a retrieval you hand to someone else. You can't see Google's real fan-out. But you can approximate it closely enough to find gaps. Here's a script that does exactly that. ``` seed question ──► LLM generates likely sub-queries │ your page ──► split into passages ──► embed │ for each sub-query: best-matching passage + score │ coverage report (gaps first) ``` Two parts: ``` pip install anthropic pydantic sentence-transformers requests beautifulsoup4 export ANTHROPIC_API_KEY=... ``` I'm using Claude for generation and a small open-source embedding model for scoring. Both are swappable. Structured output keeps this clean: we ask for a typed list and get a validated Python object back, with no regex over model text. ``` python # fanout.py from typing import Literal import anthropic from pydantic import BaseModel class SubQuery(BaseModel): query: str intent: Literal["definition", "comparison", "how-to", "pricing", "constraint", "alternative", "follow-up"] class FanOut(BaseModel): sub_queries: list[SubQuery] client = anthropic.Anthropic() PROMPT = """You are simulating the query fan-out step of an AI search engine. Given a user's question, list the 12 short web searches the engine would most likely run in parallel to build a complete, trustworthy answer. Cover different facets: definitions, comparisons, pricing, constraints (size, budget, region, stack), how-to steps, alternatives, and the follow-up a careful user would ask next. Write each one the way a search engine would (short, keyword-like), not as a full sentence. User question: {question}""" def fan_out(question: str) -> list[SubQuery]: response = client.messages.parse( model="claude-opus-5-5", max_tokens=16000, output_config={"effort": "low"}, # this is a simple task; keep it cheap messages=[{"role": "user", "content": PROMPT.format(question=question)}], output_format=FanOut, ) return response.parsed_output.sub_queries ``` Run it on a real question: ``` for sq in fan_out("best way to host a Next.js app for a small team"): print(f"[{sq.intent:11}] {sq.query}") [comparison ] vercel vs netlify vs cloudflare pages nextjs [pricing ] vercel pro pricing per seat 2026 [constraint ] nextjs self hosting docker small team [how-to ] deploy nextjs app router to cloudflare [constraint ] nextjs ISR support hosting providers [alternative] nextjs hosting without vercel [follow-up ] nextjs hosting cold starts serverless ... ``` Generated output varies between runs. That's fine. Run it 3–5 times and merge the results, because real fan-out isn't deterministic either. AI systems retrieve **passages**, not whole pages. We'll split by headings and paragraphs, keeping each heading attached to its text so the passage still makes sense on its own. ``` python # passages.py import requests from bs4 import BeautifulSoup def page_passages(url: str, min_words: int = 25) -> list[str]: html = requests.get(url, timeout=20, headers={"User-Agent": "fanout-audit/1.0"}).text soup = BeautifulSoup(html, "html.parser") for tag in soup(["script", "style", "nav", "footer", "header", "aside"]): tag.decompose() root = soup.find("main") or soup.find("article") or soup.body passages, heading = [], "" for el in root.find_all(["h1", "h2", "h3", "p", "li", "td"]): text = " ".join(el.get_text(" ", strip=True).split()) if el.name in ("h1", "h2", "h3"): heading = text continue if len(text.split()) >= min_words: passages.append(f"{heading}: {text}" if heading else text) return passages ``` Note that this reads the **raw HTML**, the way many AI fetchers do. If your content only appears after JavaScript runs, this returns almost nothing, and that's a finding in its own right (see Part 9). ``` python # coverage.py from sentence_transformers import SentenceTransformer, util model = SentenceTransformer("BAAI/bge-small-en-v1.5") def coverage(sub_queries, passages): q_emb = model.encode([sq.query for sq in sub_queries], normalize_embeddings=True) p_emb = model.encode(passages, normalize_embeddings=True) sims = util.cos_sim(q_emb, p_emb) # [n_queries, n_passages] rows = [] for i, sq in enumerate(sub_queries): best = int(sims[i].argmax()) rows.append({ "query": sq.query, "intent": sq.intent, "score": float(sims[i][best]), "best_passage": passages[best][:110] + "…", }) return sorted(rows, key=lambda r: r["score"]) # gaps first python # main.py import sys from fanout import fan_out from passages import page_passages from coverage import coverage question, url = sys.argv[1], sys.argv[2] # Merge a few runs to smooth out randomness. seen, subs = set(), [] for _ in range(3): for sq in fan_out(question): key = sq.query.lower() if key not in seen: seen.add(key) subs.append(sq) passages = page_passages(url) print(f"{len(subs)} sub-queries · {len(passages)} passages\n") for r in coverage(subs, passages): flag = "GAP " if r["score"] < 0.55 else "weak" if r["score"] < 0.68 else "ok " print(f"{flag} {r['score']:.2f} [{r['intent']}] {r['query']}") if flag != "ok ": print(f" closest: {r['best_passage']}") python main.py "best way to host a Next.js app for a small team" https://your-site.com/blog/nextjs-hosting ``` *(Illustrative output. Your numbers will differ.)* ``` 33 sub-queries · 41 passages GAP 0.41 [pricing] vercel pro pricing per seat 2026 closest: Why we moved: Our bill grew faster than our traffic… GAP 0.47 [constraint] nextjs ISR support hosting providers closest: Deploying to Cloudflare: The adapter handles most… weak 0.61 [follow-up] nextjs hosting cold starts serverless ... ok 0.79 [comparison] vercel vs netlify vs cloudflare pages nextjs ``` `bge-small` on English technical content. Calibrate them by checking a few passages yourself: find the score below which the "closest passage" clearly doesn't answer the question. If you want a reality check, compare your generated sub-queries with Bing's real grounding queries (Bing Webmaster Tools → AI Performance, CSV export). Matching phrasing there is the closest thing to ground truth you'll get today. Coverage tells you whether your page *can* answer the sub-questions. It doesn't tell you whether AI engines actually *do* pick you. For that you need to run the real prompts across ChatGPT, Perplexity, Gemini and others over time and track who gets mentioned. We built [Vista AI's AI rank tracker](https://www.vistaai.io/features/ai-rank-tracker) for that part. The script above is the free, local half of the loop: find gaps, fix them, then measure. The full script is ~120 lines. If you adapt it (different embedding model, multilingual, or batch mode over a sitemap), share it in the comments. Next in the series: **Track ChatGPT, Perplexity and Claude referrals properly.**