{"slug": "cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data", "title": "Cloudflare AI Search: give your agents a search engine for your data", "summary": "Cloudflare announced developer experience improvements to Cloudflare AI Search, enabling users to index data, skip sitemaps with the Discover parsing option, get public endpoints for searching across namespaces, and add custom domains. The company also previewed pricing where embedding and reranking are free with default models, and introduced the Cloudflare Dev Stack MCP for coding agents.", "body_md": "# Cloudflare AI Search: give your agents a search engine for your data\n\nToday, we’re excited to announce a few developer experience improvements to [ Cloudflare AI Search](https://developers.cloudflare.com/ai-search/) to make it easy to manage a search solution out of the box. Previously, you had to stitch together components of the Cloudflare primitives (Workers AI, AI Gateway, Vectorize, R2, Browser Run) but now, AI Search can do this automatically, and better. Our goal is to give your agents their own search engine, where they can easily find data to provide better answers for themselves and their humans.\n\nWe’re also sharing an early preview of pricing for customers of AI Search so you can learn how this scales. We modeled pricing in a way that makes it predictable and scalable: embedding and reranking are free when you use the default models, so no need to worry about predicting token count.\n\nIn AI Search, users can now:\n\n**Index a collection of data for your agent:** Make structured and unstructured data easily accessible for your agent to build with, from individual files to websites you own. (Today, it must be a zone on your Cloudflare account, but with more ways to verify ownership coming soon.)**Skip the sitemap for your websites:** Previously, AI Search required that websites have a sitemap to use the website integration. Now you can select the “Discover” parsing option to add a website without a sitemap as a source.**Get a single public endpoint for searching across a namespace:** When you enable public URLs on your namespace, you can get a`/search`\n\nand`/mcp`\n\nendpoint that can search through multiple instances or websites at once without authentication, so you can share easily with your customers.**Put your own custom domain over public endpoints:** You can now add your own domains over your public URLs, so you can brand your`/search`\n\nand`/mcp`\n\nendpoints (e.g.,`search.example.com/mcp`\n\n). You can also addto create private search instances.__Cloudflare Access__**Add semantic search to your sites built on EmDash with AI Search plugin:** If your site runs on, our open-source CMS, the__EmDash__adds semantic search over your content.__AI Search plugin__**Preview the new pricing model for AI Search:** We want pricing to be predictable and to scale with you, so we built in the cost of embedding and reranking: they’re free when you use select models from the Workers AI catalog.\n\nFinally, we will also share examples of how AI Search is used across our own platform including [Cloudflare.com](http://Cloudflare.com), our Developer Docs, with EmDash, in Cloudflare Dev Stack MCP — and even the blog post you’re reading right now (try cmd+K).\n\n## AI Search in action: powering the new Cloudflare Dev Stack MCP\n\nOne of the ways we use AI Search is in our new **Cloudflare Dev Stack MCP**, which you can try today in our [ AI Playground](https://playground.ai.cloudflare.com). It gives coding agents current, cited docs from across the Cloudflare developer ecosystem, so they build on the latest features and fixes instead of stale training data.\n\nHere's how we built it using the features available today in AI Search:\n\n### 1. Index each surface\n\nWe created one AI Search instance per Cloudflare-owned surface: Docs, Blog, API Docs, Community, Astro, Vite, Vitest, Hono, Replicate, OpenNext. (Each of these is Cloudflare-owned.)\n\nThey span different domains, but, because Cloudflare owns the website data, AI Search is able to treat them as a single set and ingest them all the same way. Point AI Search at a site, or set of sites, and it handles crawling, ingestion, embedding, and retrieval. Creating an instance is a single command, and for a site without a sitemap you add --parse-type discover to find pages by following links (powered by /crawl from Browser Run):\n\n```\nnpx wrangler ai-search instance create cloudflare-community \\\n  --namespace dev-stack \\\n  --source https://community.cloudflare.com \\\n  --type web-crawler \\\n  --parse-type discover\n```\n\n### 2. Combine the instances into one search\n\nNow the interesting part: answering a single query across all 10 instances. There are two ways to do it.\n\n**Option A: in a Worker (what we did for Cloudflare Stack MCP)**\n\nWe bound the namespace to a Worker to create a [ remote MCP server](https://developers.cloudflare.com/agents/model-context-protocol/guides/remote-mcp-server/) and made one multi-instance call across all 10 instances. We took this path because we're adding the stack search into\n\n[, so it ships as a tool alongside the Cloudflare tools agents already connect to.](https://github.com/cloudflare/mcp-server-cloudflare)\n\n__Cloudflare's MCP server__The binding, in `wrangler.jsonc:`\n\n```\n{\n  \"ai_search_namespaces\": [\n    { \"binding\": \"AI_SEARCH\", \"namespace\": \"cloudflare-stack\" }\n  ]\n}\n```\n\nThen a single tool makes one call that fans out across the instances you name:\n\n```\n// One tool, one call that searches every surface in the namespace at once.\ncontext.registerTool(\n  'search_dev_stack',\n  {\n    description: 'Search current docs across the Cloudflare stack.',\n    inputSchema: z.object({ query: z.string() }),\n  },\n  async ({ query }) => {\n    const res = await context.env.AI_SEARCH.search({\n      query,\n      ai_search_options: {\n        instance_ids: ['developers-cloudflare-com', 'astro', /* ...every surface */],\n        retrieval: { max_num_results: 10 },\n        reranking: { enabled: true },\n      },\n    })\n    // res.chunks come back cited and tagged with the instance they came from.\n    return { content: [{ type: 'text', text: format(res.chunks) }] }\n  }\n)\n```\n\n**Option B: flip on public endpoints (no code)**\n\nIf you'd rather not write a Worker at all, enable public URLs on the namespace. You immediately get /search and /mcp endpoints that query every instance, with no auth and nothing to deploy.\n\nReach for the Worker when you're folding search into an existing app or MCP server, as we are. Or reach for the public endpoint when you just want a shareable search endpoint in one click.\n\n### 3. Brand it and lock it down\n\nPublic endpoints come with a default public URL, but you can put your own **custom domain** over them to brand the endpoint (e.g., `search.example.com/mcp`\n\n).\n\nIf the search should be private, add **Cloudflare Access** in front of the domain. The endpoint now requires a login, so only authorized people (or agents) can query it.\n\n## Try it yourself: use the Dev Stack MCP\n\nWith the Cloudflare Dev Stack MCP Server, you can ask about any tool, or describe an app you want to build, and you'll get back current, cited answers on how best to build it on the Cloudflare stack.\n\nThe [ AI Playground](https://playground.ai.cloudflare.com) is worth checking out, but the real magic is wiring the MCP into your coding agent, so the stack's current docs are one tool call away. That replaces the usual fallback (web search then fetching full pages), which is slow, token-heavy, and often lands on the wrong or stale source. To use with your agent of choice, drop the Dev Stack MCP URL into your MCP configuration. For example:\n\n```\n{\n  \"mcpServers\": {\n    \"dev-stack\": { \"url\": \"https://stack.mcp.cloudflare.com/mcp\" }\n  }\n}\n```\n\n## Powering search on our Blog, Developer Docs, and [Cloudflare.com](http://Cloudflare.com)\n\nWe build with AI Search the same way our customers would: Cloudflare Blog's search already runs on it, and today Developer Docs and [Cloudflare.com](http://Cloudflare.com) join it. All of it uses hybrid search, semantic and keyword together in one query, so it handles both open-ended \"what does this do\" questions and exact lookups of names or keywords. We recently rebuilt the Blog on EmDash, our new open-source CMS, and our new\n\n[ EmDash AI Search integration](https://docs.emdashcms.com/deployment/cloudflare/#cloudflare-ai-search) is what powers that search now. You can also add it to your own EmDash site and get the same search over your content out of the box.\n\n## AI Search respects all bot policies\n\nAI Search is powered by Browser Run `/crawl`\n\nin the background, but goes a step further to identify itself with its own bot identity: `Cloudflare-AI-Search`\n\n. Just like Browser Run, it follows robots.txt, identifies itself with an immutable, public user agent, and will respect whatever bot controls a site has in place.\n\n## Preview pricing: pricing you can predict\n\nAI Search is currently free while in beta, and billing is not yet enabled; we'll email you with plenty of notice before it starts. As we move toward general availability, here's a preview of pricing across ingestion, storage, and queries, plus embedding and reranking (*preview prices are subject to change before billing begins)*:\n\n|\n| |\n|---|---|---|\n| ||\nBase Ingestion | $0.75 / 1M tokens | 5M tokens † |\nImage processing (add-on) | +$0.50 / 1M tokens | 5M tokens † |\n| ||\nStored data | $2.00 / GB-month | 10 GB |\n| ||\nSemantic (hybrid and vector search) | $0.75 / 1k queries | 2,000 queries ‡ |\nFull-text | $0.10 / 1k queries | 2,000 queries ‡ |\n| ||\nIngestion and query | Free with select Workers AI models; third-party billed separately | N/A |\n\n*† A single pool of 5M ingestion tokens per month, covering any file type currently supported (e.g., text, images). ‡ A single pool of 2,000 queries per month, shared across both query types. *\n\nOur goal is to provide pricing you can predict, starting with the models your search leans on. Embedding turns your text into the vectors that search matches on, and reranking reorders results so the most relevant come first. Both run free with AI Search defaults or when using select models from the Workers AI catalog, so the models behind indexing and every search are not a cost you have to worry about. Answer generation and query rewriting are optional steps that run on a model you choose, billed as Workers AI usage, or you can use AI Gateway credits with any model/provider.\n\n### Example bill with preview pricing\n\nHere's a sample monthly bill on the Workers Paid plan for creating a new AI Search instance for a 20,000-document data source (about 20M tokens of text) plus 1,000 images (assume about 1,000 tokens each), with 30,000 semantic queries a month using the default AI Search embedding and reranking model. Ingestion is chunked with roughly 10% overlap, which shows up as the × 1.1 below:\n\n|\n|\n|\n|\n|---|---|---|---|\nBase ingestion | (20M tokens of text + 1M tokens of images) × 1.1 - 5M free = 18.1M tokens | $0.75 / 1M tokens | $13.58 |\nImage add-on | 1M tokens of images × 1.1 = 1.1M tokens | $0.50 / 1M tokens | $0.55 |\nStorage | ~1.2 GB (within 10 GB free) | $2 / GB-mo | $0 |\nQueries (semantic) | 30,000 - 2,000 free = 28K | $0.75 / 1k | $21.00 |\nEmbedding (Workers AI) | Usage included with selected Workers AI model | $0 | $0 |\nReranking (Workers AI) | Usage included with selected Workers AI model | $0 | $0 |\n|\n|\n\nImages count toward base ingestion and also incur the image add-on cost. Storage assumes about 10 KB per document and 1 MB per image. Indexing is largely a one-time cost, so later months are mostly queries, closer to $21.\n\n## Get started today\n\nAI Search is available to enable and use today. Point it at your site, turn on hybrid search for both semantic and keyword matching, and you have a search engine for your own data, ready for your agents. Spin one up with one command:\n\n```\nnpx wrangler ai-search create my-search \\\n  --namespace my-namespace \\\n  --source https://my-website.com \\\n  --type web-crawler \\\n  --hybrid-search\n```\n\nFrom there, query it, wire it into an agent over `/mcp`\n\n, or put a custom domain on a public `/search`\n\nendpoint to share it with your users. Check out the [ AI Search docs](https://developers.cloudflare.com/ai-search/) for more information.", "url": "https://wpnews.pro/news/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data", "canonical_source": "https://blog.cloudflare.com/ai-search-easier/", "published_at": "2026-08-06 13:00:00+00:00", "updated_at": "2026-08-09 12:53:49.267479+00:00", "lang": "en", "topics": ["ai-products", "ai-tools", "ai-infrastructure", "developer-tools"], "entities": ["Cloudflare", "Cloudflare AI Search", "Workers AI", "AI Gateway", "Vectorize", "R2", "Browser Run", "Cloudflare Dev Stack MCP"], "alternates": {"html": "https://wpnews.pro/news/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data", "markdown": "https://wpnews.pro/news/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data.md", "text": "https://wpnews.pro/news/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data.txt", "jsonld": "https://wpnews.pro/news/cloudflare-ai-search-give-your-agents-a-search-engine-for-your-data.jsonld"}}