{"slug": "i-kept-asking-chatgpt-to-read-my-website-until-its-aeo-got-better", "title": "I Kept Asking ChatGPT to Read My Website Until Its AEO Got Better", "summary": "A developer improved their website's machine readability for AI answer engines by repeatedly asking ChatGPT, Claude and Perplexity to fetch and summarize pages from ihatereading.in, fixing the site each time retrieval failed. Using Vercel's is-agentic npm package, the site reached a score of 96, but the developer argues manual chatbot testing reveals how a site fails from an AI's perspective in ways an automated score cannot. The account stresses that AEO depends first on basic HTTP responses, HTML availability, discoverable links and extractable content before any higher-level optimization.", "body_md": "I found a pretty simple way to test whether a website is actually ready for AI search.\n\nI stopped looking at my website like an SEO.\n\nInstead, I started looking at it like ChatGPT, Claude and Perplexity.\n\nThe process was basically:\n\nCan you fetch my website?\n\nThen:\n\nFetch the latest articles from [https://ihatereading.in](https://ihatereading.in)\n\nFetch more details about [https://ihatereading.in](https://ihatereading.in)\n\nGet all links from [https://ihatereading.in](https://ihatereading.in)\n\nAnd every time the answer was incomplete, wrong, empty or failed completely, I went back to the code and fixed something.\n\nI kept doing this until the answers started getting much better.\n\nThat sounds almost too simple.\n\nBut there is something important behind it.\n\nWe talk a lot about AEO — Answer Engine Optimization.\n\nWe discuss:\n\nAll of that matters.\n\nBut there is another test I don't see people talking about enough:\n\n**Can an AI actually read your website?**\n\nNot theoretically.\n\nActually.\n\nGive the URL to ChatGPT and ask it to retrieve something.\n\nIf it can't, you've found a problem.\n\n`is-agentic` to test the website\nI also used [`is-agentic`](https://is-agentic.com), the npm package from Vercel, to test the site's agentic readiness.\n\nI managed to bring the website to **96**.\n\nBut the more interesting part wasn't the number.\n\nIt was what happened when I started manually testing the website through AI chatbots.\n\nBecause an automated score can tell you that something is wrong.\n\nAn AI chatbot can show you **how the website is failing from the AI's perspective.**\n\nThat's much more useful.\n\nThis became my basic workflow:\n\n```\nWebsite\n   ↓\nAsk ChatGPT / Claude / Perplexity\n   ↓\nSee what it can actually retrieve\n   ↓\nFind what is missing\n   ↓\nFix the website\n   ↓\nAsk again\n   ↓\nRepeat\n```\n\nFor example, I would ask:\n\nIf the AI couldn't find the articles, I didn't immediately assume the AI was bad.\n\nI checked my website.\n\nMaybe the article content wasn't available in the initial HTML.\n\nMaybe the links weren't discoverable.\n\nMaybe the routing wasn't exposing the content correctly.\n\nMaybe the page returned an unexpected response.\n\nMaybe the content structure made it difficult to extract.\n\nSo I fixed it.\n\nThen I asked again.\n\nOnce the basic request worked, I started increasing the difficulty.\n\nFor example:\n\nThen I could ask things like:\n\nWhat topics does iHateReading cover?\n\nWhat are the latest articles?\n\nFind articles about AI agents.\n\nWhat developer tools are mentioned on the website?\n\nGive me the URLs of relevant articles.\n\nThe point isn't that these are sophisticated prompts.\n\nThe point is that **they expose the actual machine readability of your website.**\n\nThis is where I think AEO gets misunderstood.\n\nA website can look perfect to a human.\n\nThe navigation works.\n\nThe animations work.\n\nThe articles look great.\n\nThe content loads when you open the page.\n\nBut an AI system may see something very different.\n\nImagine an AI trying to answer:\n\nWhat is iHateReading?\n\nIt needs to discover the relevant information first.\n\nThen imagine:\n\nWhat are the latest articles on iHateReading?\n\nNow it needs to find article URLs, understand dates and titles, and retrieve the content.\n\nFind me an iHateReading article about AI agents.\n\nNow it needs to connect the topic with the site's content.\n\nThese are different retrieval problems.\n\nAnd each one can expose a different technical issue.\n\nOne thing that stood out to me was how important basic HTTP responses are.\n\nIf an AI crawler gets something like:\n\n```\n400\n500\n204\ncontent not found\nempty response\n```\n\nthere isn't much AEO magic you can do afterward.\n\nThe AI can't recommend content it can't retrieve.\n\nThis is why I think **AEO starts much lower in the stack than people think.**\n\nBefore worrying about whether an LLM understands your brand positioning, make sure it can actually get the content.\n\n```\nHTTP response\n     ↓\nHTML\n     ↓\nLinks\n     ↓\nContent\n     ↓\nStructure\n     ↓\nMeaning\n     ↓\nAnswer\n```\n\nIf you fail at the first few layers, the later ones don't matter.\n\nThis is probably the biggest takeaway for me.\n\nWe sometimes treat AI search like something completely separate from SEO.\n\nBut there is a huge overlap.\n\nIf an AI needs to recommend your website, it needs to:\n\nThat's not completely different from search.\n\nThe interface changed.\n\nThe retrieval and understanding problems didn't disappear.\n\nThis is the part I'd actually recommend trying.\n\nDon't ask ChatGPT:\n\nIs my website AEO optimized?\n\nThat question is too broad.\n\nAsk it to **do something with your website.**\n\n```\nFetch the latest articles from https://yourwebsite.com\nGet all discoverable links from https://yourwebsite.com\nFind the pages about [topic] on https://yourwebsite.com\nSummarize what this website does.\nFind the most relevant article on this website for [question].\nGive me the URL and title of that article.\n```\n\nNow you're testing multiple parts of the system.\n\nAnd when something fails, **don't just try another prompt.**\n\nFix the website.\n\nThen ask again.\n\nThat's the mental model I like most.\n\nInstead of treating AEO as a checklist you complete once, treat it like testing an API.\n\nYou have an input:\n\n```\n\"Find articles about AI agents\"\n```\n\nYou expect:\n\n```\n{\n  \"title\": \"...\",\n  \"url\": \"...\",\n  \"content\": \"...\"\n}\n```\n\nBut the AI gives you:\n\n```\nI couldn't find relevant articles.\n```\n\nThat's a failed test.\n\nSo you investigate.\n\nMaybe the links aren't discoverable.\n\nMaybe your HTML is mostly an application shell.\n\nMaybe your article content isn't server-rendered.\n\nMaybe your URLs aren't consistent.\n\nMaybe your metadata is missing.\n\nMaybe your internal linking is weak.\n\nMaybe the response itself is broken.\n\nYou fix it.\n\nRun the test again.\n\nThat's a much more practical way to think about AEO.\n\nThere is one important distinction, though.\n\nI'm not trying to make ChatGPT say nice things about my website.\n\nThat's not the goal.\n\nThe goal is to make the underlying website **easy for machines to retrieve and understand.**\n\nIf I ask:\n\nFetch all links from my website.\n\nand it returns them correctly, that's useful.\n\nFind my latest article about AI agents.\n\nand it finds the correct page, that's useful.\n\nWhat does iHateReading publish?\n\nand it can accurately explain the site, that's useful.\n\nThe answer itself is the test.\n\nYou don't need some massive AEO tool to start.\n\nOpen ChatGPT, Claude or Perplexity.\n\nGive it your URL.\n\nThen keep asking questions.\n\nStart simple:\n\n```\nFetch my homepage.\nFetch my latest articles.\nGet the links to those articles.\nFind articles about [your topic].\nExplain what this website is about.\nWhich page on this website best answers [your customer's question]?\n```\n\nEvery failure is a debugging clue.\n\nAnd every successful answer is another indication that your website is becoming easier for an answer engine to use.\n\nThis is probably my favorite part of the whole experiment.\n\nInstead of asking:\n\n\"Is my website optimized for AI?\"\n\njust ask the AI to **use your website.**\n\nIf it can't use it, fix the website.\n\nAsk again.\n\nIf it still can't, fix something else.\n\nKeep going.\n\nThat's what I did with iHateReading.\n\nI used [`is-agentic`](https://is-agentic.com) to get a measurable signal, but then I used actual AI conversations to find the problems that mattered.\n\n**Your AEO score is interesting.**\n\n**Whether an AI can actually find, retrieve and understand your content is more interesting.**\n\nAnd the easiest way to find out is to simply ask.\n\nThat would be enough for today, for more details visit our website [iHateReading](//ihatereading.in)\n\nCheers\n\nShrey", "url": "https://wpnews.pro/news/i-kept-asking-chatgpt-to-read-my-website-until-its-aeo-got-better", "canonical_source": "https://dev.to/shreyvijayvargiya/i-kept-asking-chatgpt-to-read-my-website-until-its-aeo-got-better-1246", "published_at": "2026-09-24 09:16:58+00:00", "updated_at": "2026-09-24 09:30:39.839111+00:00", "lang": "en", "topics": ["generative-engine-optimization", "ai-search", "ai-crawlers", "agent-protocols"], "entities": ["ChatGPT", "Claude", "Perplexity", "iHateReading", "Vercel", "is-agentic"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-kept-asking-chatgpt-to-read-my-website-until-its-aeo-got-better", "markdown": "https://wpnews.pro/news/i-kept-asking-chatgpt-to-read-my-website-until-its-aeo-got-better.md", "text": "https://wpnews.pro/news/i-kept-asking-chatgpt-to-read-my-website-until-its-aeo-got-better.txt", "jsonld": "https://wpnews.pro/news/i-kept-asking-chatgpt-to-read-my-website-until-its-aeo-got-better.jsonld"}}