{"slug": "i-tested-5-ai-engines-on-my-own-sites-none-agreed", "title": "I Tested 5 AI Engines On My Own Sites. None Agreed.", "summary": "Developer Dan Nwaneri extended his open-source LLM visibility checker to support five AI engines—Claude, ChatGPT, Gemini, Perplexity, and Bing Copilot—by integrating SearchApi's endpoints. Testing the tool on his own domains revealed that no two engines agreed on citations: Gemini cited his personal site for two queries, while ChatGPT cited his VPN site for one, and the other engines returned zero citations. The results highlight inconsistencies in how AI engines reference web content.", "body_md": "In June I wrote that my open-source LLM visibility checker tested Claude only and that multi-model support was \"planned but not yet implemented.\" That's the kind of line that's easy to write and easy to forget.\n\nI didn't forget. I just needed a reason to finish it.\n\nThe reason showed up when SearchApi launched endpoints for ChatGPT and Gemini, on top of the Perplexity and Bing Copilot endpoints they already had, and their growth engineer Sam Gale offered API credits to test them. Around the same time, Sam posted his own tool in SearchApi's Discord: `ai-visibility-tracker`\n\n, a dashboard that scores brand mentions across ChatGPT, Perplexity, Gemini, Copilot, and Google AI Mode. Everything except Claude.\n\nThat's not a coincidence. His tool was missing the one engine mine already had. Mine was missing the four his already had. So instead of building a second comparison tool from scratch, I extended the one I'd already shipped.\n\n`llm_visibility.py`\n\nused to do one thing: send a query to Claude Haiku, regex-match your domain against the response, print a score. Simple, and honest about its limits. Claude only knows what was in its training data, so anything published recently was invisible to it by design.\n\nNow it does five things. Claude still runs the same way: direct API call, training-data knowledge, same regex match. The other four go through a new client, `searchapi_client.py`\n\n, that hits SearchApi's `/api/v1/search`\n\nendpoint with `engine=chatgpt`\n\n, `engine=gemini`\n\n, `engine=perplexity`\n\n, or `engine=bing_copilot`\n\n. All four share one endpoint shape and return a `reference_links`\n\narray (title, link, source) that I check against your domain the same way I check Claude's response text.\n\nChatGPT only returns cited sources if you pass `web_search=true`\n\n. Without it, you get an answer with no citations at all. Caught it in SearchApi's docs before I ran anything, so it's been in the client from the first version, but easy to miss if you're skimming past the optional parameters.\n\nOne more gotcha, unrelated to SearchApi: my existing `serp_features.py`\n\nmodule already talked to an API called SerpApi, for classic Google SERP feature detection. SearchApi and SerpApi are two different companies with confusingly similar names. I kept the two clients in separate files with separate env vars (`SERPAPI_KEY`\n\nvs `SEARCHAPI_KEY`\n\n) on purpose, and I'd recommend anyone doing this kind of work do the same before they mix up a bill.\n\nThe mix-up isn't hypothetical, either. When I asked Perplexity about open-source SEO agent tools during testing, it cited `github.com/serpapi/seo-research-agent`\n\n, SerpApi's own official project, built the same way as mine (LLM plus search API). Two similarly-named companies, two similarly-shaped tools, and an AI engine happily citing both without distinguishing them.\n\nI ran it against two of my own domains. The queries came from real Google Search Console exports, not ones I picked to make a point.\n\n**dannwaneri.com**, 10 queries (mostly \"hire freelance [library] developer\" searches plus my own name):\n\n| Engine | Score |\n|---|---|\n| Claude | 0/10 |\n| ChatGPT | 0/10 |\nGemini |\n2/10 (20%) |\n| Perplexity | 0/10 |\n| Copilot | 0/9 — 1 error |\n\nGemini was the only engine that cited me at all. It got \"daniel nwaneri\" right, correctly linking my homepage. It also cited me for \"hire freelance scipy developer,\" except the page it pulled was `/hire-python-developer/`\n\n, not a scipy-specific page. Close, not exact. That gap is the difference between an engine understanding your content and one pattern-matching on adjacency.\n\n**naija-vpn.com**, 10 queries (real buyer-intent searches: Twitch payments, Fiverr payouts, dollar accounts for Nigerian freelancers):\n\n| Engine | Score |\n|---|---|\n| Claude | 0/10 |\nChatGPT |\n1/10 (10%) |\n| Gemini | 0/10 |\n| Perplexity | 0/10 |\n| Copilot | 0/9 — 1 error |\n\nHere it flipped. ChatGPT was the only one that cited me, correctly pulling `/twitch-payments-nigeria`\n\nfor \"how to receive money from twitch in nigeria.\" Gemini, the engine that carried dannwaneri.com, found nothing on this domain at all.\n\nTwo domains. Two different engines doing the only citing. Zero overlap between them. Claude, in both cases, found nothing. That tracks: neither domain existed in a form Claude's training data would have caught.\n\nIf I'd only tested Claude, like the July version of this tool did, I'd have told you both domains were invisible to AI. If I'd only tested Gemini, I'd have said dannwaneri.com was fine and naija-vpn.com wasn't. Backwards, if you'd only checked ChatGPT.\n\nWrong story either way. Each one only saw a fifth of the picture. The only way to know your actual AI visibility is to check all of them, because you can't predict which engine will happen to cite you this month.\n\nThat's the whole argument for a tool like this existing as multi-engine from the start, and it's the same argument for using one API across five engines instead of scraping each separately.\n\nCopilot errored on both test runs: a `503`\n\n(\"unable to generate an answer for this query\") on one, a request timeout on the other. Different failures, same engine, two separate runs. If you're building on top of SearchApi's Copilot endpoint, plan for it to occasionally just not answer, and don't let one failed query kill the whole batch. Mine logs the error against that query and keeps going.\n\n`modules/searchapi_client.py`\n\nand `modules/llm_visibility.py`\n\n*This piece was produced as part of SearchApi's Developer Ambassador program. They provided API credits; I built and tested the integration myself.", "url": "https://wpnews.pro/news/i-tested-5-ai-engines-on-my-own-sites-none-agreed", "canonical_source": "https://dev.to/dannwaneri/i-tested-5-ai-engines-on-my-own-sites-none-agreed-4013", "published_at": "2026-08-19 10:34:41+00:00", "updated_at": "2026-08-19 10:42:11.756170+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-tools", "developer-tools"], "entities": ["Dan Nwaneri", "SearchApi", "Claude", "ChatGPT", "Gemini", "Perplexity", "Bing Copilot", "SerpApi"], "alternates": {"html": "https://wpnews.pro/news/i-tested-5-ai-engines-on-my-own-sites-none-agreed", "markdown": "https://wpnews.pro/news/i-tested-5-ai-engines-on-my-own-sites-none-agreed.md", "text": "https://wpnews.pro/news/i-tested-5-ai-engines-on-my-own-sites-none-agreed.txt", "jsonld": "https://wpnews.pro/news/i-tested-5-ai-engines-on-my-own-sites-none-agreed.jsonld"}}