# Making 1,200 static pages readable by AI agents: MCP tools + NLWeb on Cloudflare Pages Functions

> Source: <https://dev.to/ohadfarkash/making-1200-static-pages-readable-by-ai-agents-mcp-tools-nlweb-on-cloudflare-pages-functions-oep>
> Published: 2026-11-20 06:36:24+00:00

**Recommendation:** Expose site content to agents with a keyword-scored MCP search tool and a get_guide tool that converts live pages to Markdown, rather than embedding full text in a static index.
More in the playbook "Designing agent systems": <https://wpnews.pro/playbooks/agent-architecture>

I run [OneFindMe](https://onefindme.com/en/), a free AliExpress product search that works in 21 languages. Over the past months the site grew a library of about 1,200 static pages: size guides, import-tax pages per country, buying guides, home-problem fixes, all written per language.

The site already had an [MCP](https://modelcontextprotocol.io/) server at `https://onefindme.com/mcp` with a `search_products` tool. It is listed in the official MCP registry and in about 30 directories. But I noticed a gap: an assistant connected to it could search products, yet it could not read a single one of our guides. When someone asked "13x4 or 13x6 lace wig?", the agent had no way to reach the page that answers exactly that.

Here is how I closed the gap without a database, a vector store or a second service.

Pages Functions have no file system at runtime, so the index is generated into a JS module that the function imports. A small Python script walks `sitemap.xml`, skips `noindex` pages and the search home pages, and keeps only what each page already publishes about itself:

```
out.append({"u": path, "l": lang, "t": h1, "d": meta_description,
            "h": " | ".join(h2_headings), "q": " | ".join(faq_questions), "m": date_modified})
```

The FAQ questions come from the page's own `FAQPage` JSON-LD. Result: 1,213 entries, about 900 KB, written to `functions/_guides.js` as `export default [...]`. The leading underscore keeps it from becoming a route.

I deliberately did **not** store the full text. More on that below.

`search_guides` and `get_guide`
Search is plain keyword scoring with field weights. No embeddings: the titles and FAQ questions are written in the reader's language, so they already contain the words people type.

``` js
const m = 4 * hit(e.t, w) + 2 * hit(e.u, w) + 2 * hit(e.h, w) + 2 * hit(e.q, w) + 1.5 * hit(e.d, w);
// ...
s *= matched / qs.length;                 // pages matching every word rank first
if (want && g.l === want) s *= 1.6;       // requested language
else if (!want && g.l === "en") s *= 1.25; // Latin script, no language given: English first
```

`hit()` accepts exact tokens and, for words of four letters or more, prefix matches, which covers most plurals across European languages. Tokenising with `\p{L}\p{N}` and stripping combining marks after `NFD` lets "perucke" match "Perücke" and keeps Hebrew, Arabic, Japanese and Korean intact. When the query is in a non-Latin script, the language is detected from the script itself.

One rule is product-specific: an Arabic query never gets Hebrew pages, and the reverse. That is a two-line filter, and it matters more to the readers than any ranking tweak.

`get_guide` takes a URL from the search results and returns the page as Markdown. This is why the index holds no text: the function reads the live page through the static-asset binding and converts it on the fly.

``` js
let res = await env.ASSETS.fetch(new Request(page));
if (res.status >= 300 && res.status < 400 && res.headers.get("Location"))
  res = await env.ASSETS.fetch(new Request(new URL(res.headers.get("Location"), page)));
let md = htmlToMarkdown(await res.text(), url);
```

The same `htmlToMarkdown` helper already serves `Accept: text/markdown` requests in the site middleware, so agents that ask for Markdown get it on every page. The answer can never be staler than the page.

[NLWeb](https://github.com/nlweb-ai/NLWeb) is Microsoft's open protocol for asking a website a question in natural language. The minimum is an `/ask` endpoint returning schema.org-shaped results. With the search already written, it is about 30 lines:

``` js
return json({ query_id: crypto.randomUUID(), results: r.results.map((x) => ({
  url: x.url, name: x.title, site: "onefindme.com", score: x.score, description: x.description,
  schema_object: { "@type": "Article", headline: x.title, url: x.url, inLanguage: x.language } })) });
```

It is advertised with `Link: </ask>; rel="nlweb"` on the home pages, next to the existing `api-catalog`, `service-desc` (OpenAPI) and MCP server-card relations. Try it: `https://onefindme.com/ask?query=dyson+v15`.

[GitMCP](https://gitmcp.io/) turns any public GitHub repo into an MCP server, and it reads `llms.txt` first. Our archive repo had stopped updating, so a coding agent asking GitMCP about OneFindMe got a stale README. One commit fixed it: an `llms.txt` plus one Markdown index per language (title, URL, summary, sections, questions). I kept it to a single commit through the Git Data API (`POST /git/trees` with inline contents, then a commit and a ref update): five API calls for 23 files.

The same index is also an open dataset (CC BY 4.0) on [Hugging Face](https://huggingface.co/datasets/fufu1976/onefindme-shopping-guides-21-languages) and [Kaggle](https://www.kaggle.com/datasets/onefindme/onefindme-shopping-guides-21-languages), with the FAQ answers included and no product links.

Connect it to any client that supports remote MCP servers:

```
{ "mcpServers": { "onefindme": { "type": "http", "url": "https://onefindme.com/mcp" } } }
```

*Disclosure: OneFindMe is free to use; product links on the site and in `search_products` results are affiliate links, at no extra cost to the buyer. The guide tools return no product links of their own.*
