{"slug": "context-is-the-scarce-resource-and-your-ai-still-can-t-see-your-website", "title": "Context is the scarce resource — and your AI still can't see your website", "summary": "A developer building an AI-native platform argues that context, not model quality, determines LLM output, and describes shipping two features to improve it: trimming an MCP server's tool count from 59 to 43, and adding a feature that makes existing website context visible to AI assistants. The developer notes that tool sprawl taxes context windows and that the leaner tool surface outperformed the larger one in internal tests.", "body_md": "*Part 2 of a series on building an AI-native platform. This one is about the thing that actually determines output quality — and a feature we shipped because of it.*\n\nCode is deterministic. Same input, same output, every time. That property is the whole reason we can test it, cache it, and reason about it.\n\nAI is not that. Same prompt, different answer, and no amount of shouting in capital letters changes the underlying fact that you're sampling from a distribution. If you've built anything real on top of an LLM, you already know this in your bones: the demo works, the eval is flaky, and the difference between \"ship it\" and \"delete it\" often comes down to something you can't quite point at.\n\nSo here's the question that matters for anyone building with these models: **if the model doesn't guarantee quality, where does quality come from?**\n\nThe honest answer, after a couple of years of building on this stuff, is context. Not prompt-engineering folklore. Not a marginally smarter model. How much of the actual situation the model can see at the moment it answers.\n\nThe clearest way I know to feel this: take the same task and run it twice.\n\n```\n# Run A — no context\n\"Add pagination to the product list.\"\n\n# Run B — same task, repo in context\n\"Add pagination to the product list.\"\n# ...with the actual ProductList component, the API client,\n#    and the existing useProducts hook in the window.\n```\n\nRun A gives you a plausible, generic answer that assumes a framework you're not using and an API shape you don't have. Run B gives you a diff you can apply. Same model, same temperature, same prompt string. The only thing that changed is what it could see.\n\nThat's not a subtle effect. In practice it's most of the difference between output you ship and output you throw away. And it has a property that quietly governs everything downstream: **context is finite.** There's a budget. Every token you spend on one thing is a token you didn't spend on another.\n\nOnce you accept that, there are exactly two levers for better results. We shipped both, and the second one is the feature this post is really about.\n\nWe expose an MCP server. At one point it had 59 tools. It now has 43, and we intend to keep it lean.\n\nThat reduction was counterintuitive to argue for internally, because on a feature page \"more tools\" reads as \"more capable.\" But watch what actually happens in a session. Every tool an MCP client loads is a schema — a name, a description, a parameter list — and all of it goes into the context window *before the model does anything*. Fifty-nine tools is a menu the model reads end to end on every turn, whether or not the task needs any of them.\n\nTool sprawl looks like capability and behaves like noise. Trim the surface to the tools that carry their weight, merge the overlapping ones, and the model spends its attention on your project instead of on your API reference. We measured the effect the boring way — running the same build tasks across models before and after — and the lean surface came out ahead on every one of them.\n\nIf you're shipping an MCP server, this is the lever nobody talks about: **your tool count is a tax your users' models pay on every call.** Design it like context is expensive, because for the thing consuming it, it is.\n\nHere's the thing that quietly annoyed me for a year.\n\nFor anyone with an existing website, the single densest piece of context they own has been completely unreachable to their AI. Your information architecture. Your copy, in your voice. Your visual system. The hundred small decisions you made and then forgot you made. All of it sitting on a server, and none of it visible to the assistant you're asking for help.\n\nSo the industry's advice became: **start over.** Open a blank prompt, describe your site from memory, and hope the result resembles what you already had.\n\nThat's not a fresh start. It's a lossy re-encode of something that already existed, run through a narrow text channel. You re-litigate decisions you'd already settled, and you silently lose the ones you can't remember making. For a category of tool whose entire value proposition is *use your own words*, telling people to throw away their most contextful asset was backwards.\n\nSo we built the importer. You give it the URL of a site you own; it produces a real, editable project on the platform, and then you build on top of it with whatever assistant you already use.\n\nThe naive way to clone a site is to flatten everything into one directory and rewrite every reference to match the new layout. It works, but it's a rewrite bomb: every `href`\n\n, every `src`\n\n, every `url()`\n\nin CSS, every path buried in a JSON blob has to be found and mutated, and each mutation is a chance to break something.\n\nWe took the other road. Pages get clean URLs — `about.html`\n\nbecomes `/about`\n\n, `contact.php`\n\nbecomes `/contact`\n\n, trailing slashes get normalised — because those are the URLs a human will see and share. But **same-origin assets keep their original paths.** The stylesheet that referenced `../../uploads/2019/hero.jpg`\n\nstill references exactly that, because that file still lives at exactly that path.\n\nThe payoff is that the rewriter shrinks to almost nothing. Instead of remapping every reference, you're doing host-level normalisation and a small set of page-slug rewrites. Three passes:\n\n`https://site.com`\n\n, `//site.com`\n\n, and same-origin absolute paths to one internal form.`.html`\n\n/`.php`\n\n→ clean URL), applied longest-match-first so a longer path never gets clipped by a shorter one that's a prefix of it.`robots`\n\ndirectives, its canonical tags, platform-injected boot scripts.\nFewer rewrites, fewer edge cases, fewer ways to end up with a 99%-imported (i.e. broken) site.The concept is a weekend. The reason it's *not* a weekend is that the web is thirty years of accumulated weirdness, and a site that imports 99% correctly is a broken site. A sampling of what actually decides whether this works:\n\n**Escaped JSON-LD.** WordPress + Yoast emits structured data with escaped slashes:\n\n```\n{\"@type\":\"CoffeeShop\",\"url\":\"https:\\/\\/site.com\\/\"}\n```\n\nA rewriter that only understands `href=\"...\"`\n\nwalks straight past that URL and leaves a dead reference inside your schema graph. You have to parse the escaped form.\n\n**Extension vs. DOM context.** A file can lie about what it is.\n\n```\n<link rel=\"stylesheet\" href=\"/theme/css/variables.php\">\n```\n\nThat's a `.php`\n\nfile being served as `text/css`\n\n. If you route on the extension you file it as a page and the site loses its styling. The `<link rel=\"stylesheet\">`\n\ncontext is the truth; the extension is not. Context wins — you store it as a stylesheet and fetch it from the original URL, query string intact.\n\n**Percent-encoded filenames.** `café-interior.jpg`\n\nshows up in the HTML as `caf%C3%A9-interior.jpg`\n\n. Decode it wrong, or decode it twice, or hand it to a slug sanitiser that strips non-ASCII, and the reference and the stored file no longer point at each other. (Aside: we caught our *own* asset store silently stripping the `é`\n\non upload during this work. Your own stack is one of the thirty years of weirdness.)\n\n**Protocol-relative and srcset.** `//host/img.jpg`\n\nand the comma-separated `srcset`\n\nladder both hide references in places a naive `src`\n\n-only scan misses.\n\n**Redirect chains.** `/old`\n\n→ `/older`\n\n→ `/current`\n\n, and somewhere in the chain a hop wanders off to a host you don't control. You follow it to the end *or* you refuse it and say so — what you don't do is silently drop it and report success.\n\nThat last one is the real principle. **If you bound coverage, say what you dropped.** A report that says \"100%, 0 failed\" while the page has no CSS isn't a report, it's a lie with a progress bar.\n\nA static import cannot bring across dynamic behaviour, and pretending otherwise just moves the disappointment downstream. The source's forms post to an origin that no longer exists. AJAX endpoints, server-rendered personalisation, anything that needed a backend — none of it survives the copy, because there's no backend on the other side to receive it.\n\nSo the importer flags those rather than faking them. Forms come in as markup; you reconnect them to the platform's form layer afterward, which is a minute of work once you know which forms need it. The skip report is where that list lives.\n\nThere's also a real boundary worth stating for anyone about to test this: **it's same-host.** Assets served from a separate CDN subdomain — `cdn.yoursite.com`\n\n, `static.yoursite.com`\n\n— are not pulled in by default. If your theme lives on one host and your media on another, that's a case to plan around, not assume away. I'd rather you read that here than discover it mid-import.\n\nThe line I'd offer other people building in this space: before you build a *migration* feature, check whether you're actually building a *context* feature. It changes what \"done\" means.\n\nA migration is done when the bytes moved. A context feature is done when the thing you moved is now somewhere the model can reason over it — clean URLs it can address, SEO fields it can edit, structure it can navigate. Same crawl, different definition of success, and the second one is the one your users actually feel.\n\nContext is the scarce resource. Spend it carefully, and fill it with the user's own work.\n\n*We build WebsitePublisher — an MCP platform where your AI builds real sites with a real backend: auth, payments, forms, structured data. The import feature is live. If you try it and something doesn't come across, the skip report will tell you what and why — and I'd genuinely like to hear about the cases we didn't anticipate.*", "url": "https://wpnews.pro/news/context-is-the-scarce-resource-and-your-ai-still-can-t-see-your-website", "canonical_source": "https://dev.to/megberts/context-is-the-scarce-resource-and-your-ai-still-cant-see-your-website-3g0h", "published_at": "2026-08-13 05:04:40+00:00", "updated_at": "2026-08-13 05:14:21.353197+00:00", "lang": "en", "topics": ["large-language-models", "ai-agents", "developer-tools", "ai-infrastructure"], "entities": [], "alternates": {"html": "https://wpnews.pro/news/context-is-the-scarce-resource-and-your-ai-still-can-t-see-your-website", "markdown": "https://wpnews.pro/news/context-is-the-scarce-resource-and-your-ai-still-can-t-see-your-website.md", "text": "https://wpnews.pro/news/context-is-the-scarce-resource-and-your-ai-still-can-t-see-your-website.txt", "jsonld": "https://wpnews.pro/news/context-is-the-scarce-resource-and-your-ai-still-can-t-see-your-website.jsonld"}}