Building a RAG Chatbot on Cloudflare Workers (Vectorize + D1 + Workflows)

A developer built chatbot-with-rag, a minimal retrieval-augmented generation chat demo that runs entirely on Cloudflare Workers using Vectorize, D1, and Workflows. The project links D1's auto-increment chunk id to the corresponding Vectorize vector id for retrieval, and uses Cloudflare Workflows' step.do() durable checkpoints to give each chunk its own retry rather than re-running an entire batch on failure.

I first saw RAG in 2024, at a hackathon. The moment that stuck with me was realizing an LLM could "search" through documents I'd just uploaded and answer questions about them - not from its training data, from my content. I was genuinely astonished; it felt like a different category of capability than just chatting with a model. Today, that's not a novelty anymore - it's close to a baseline expectation for any chatbot that needs to answer questions about content it wasn't trained on. A chatbot only knows what it was trained on - ask it about your own docs, your own product, your own notes, and it either hallucinates an answer or tells you it doesn't know. Retrieval-Augmented Generation RAG is the standard fix: before the model answers, you go find the actual relevant text and hand it over as context. I wanted to build the smallest version of that pattern that still behaves honestly - retrieves the right thing, admits when it can't, and doesn't lose data halfway through ingesting a document. This is a walkthrough of chatbot-with-rag https://github.com/palermo-777/chatbot-with-rag , a minimal RAG chat demo running entirely on Cloudflare: No LangChain agent framework, no vector DB to self-host, no separate backend - one Worker, four bindings. Two flows, running through the same Worker. The two stores are linked by a shared id: D1's auto-increment id for a chunk is the exact same value used as that chunk's Vectorize vector id. A similarity search gives you back an id; a D1 lookup on that id gives you the actual text. No metadata duplication, no second index to keep in sync - just one id, two stores, one source of truth each for what they're good at Vectorize for "what's similar," D1 for "what does it actually say" . The obvious version of ingestion is: split the doc, loop over the chunks, embed and insert each one, done. The problem is what happens when chunk 7 of 12 fails - a rate-limited embedding call, a transient D1 failure, doesn't matter. A plain loop either crashes the whole request losing all 12 chunks or needs you to implement retry logic. Cloudflare Workflows solve this by making each step.do call a durable checkpoint. If a step fails, only that step retries - everything before it is already saved and never redone: export class RAGWorkflow extends WorkflowEntrypoint<Env, RagWorkflowParams { async run event: WorkflowEvent<RagWorkflowParams , step: WorkflowStep { const { data } = event.payload; const texts = await step.do 'split text', async = { // split data into chunks with RecursiveCharacterTextSplitter } ; for const i, text of texts.entries { const record = await step.do store in D1 db: ${i}/${texts.length} , async = { // insert the chunk's text into D1, return its new row id } ; const vector = await step.do Generate Embeddings: ${i}/${texts.length} , async = { // embed the chunk's text via Workers AI bge-base-en-v1.5 } ; await step.do Insert Vector: ${i}/${texts.length} , async = { // upsert { id: record, values: vector } into Vectorize } ; } } } One thing worth being explicit about, because it's an easy trap: the loop is around step.do , not inside it. Put a loop of several inserts inside a single step.do call and Workflows can only checkpoint the whole step as one unit - a failure on item 7 means the entire step retries, re-inserting items 1 through 6 all over again. Giving each chunk its own uniquely-named step store in D1 db: 3/12 , not a bare store in D1 db reused across iterations - step names are the cache key Workflows uses to track what's already done is what actually buys you per-chunk retry instead of per-batch retry. Notice too that record - the id step.do returns from the D1 insert: js const record = await step.do store in D1 db: ${i}/${texts.length} , async = { / ... / } ; reappears two steps later in the Vectorize upsert: env.VECTORIZE.upsert { id: record.toString , values: vector } ; That's not incidental: whatever a step.do callback returns is exactly what Workflows durably persists as that step's result, so a later step can reuse it for free instead of recomputing or re-fetching it. It's the same relationship the architecture diagram's dashed chunk id arrow is pointing at - a value produced by one step, consumed by a later one, with nothing in between needing to know where it came from. Four bindings in wrangler.jsonc - Workers AI, Vectorize, D1, and the Workflow itself: { "ai": { "binding": "AI" }, "vectorize": { "binding": "VECTORIZE", "index name": "vector-index" } , "d1 databases": { "binding": "database", "database name": "database", "database id": "..." } , "workflows": { "name": "rag", "binding": "RAG WORKFLOW", "class name": "RAGWorkflow" } , "assets": { "directory": "./public/", "binding": "ASSETS" } } Vectorize needs to exist before anything can be upserted into it: npx wrangler vectorize create vector-index --dimensions=768 --metric=cosine 768 dimensions because that's what @cf/baai/bge-base-en-v1.5 - the embedding model - outputs. If you swap embedding models later, the dimension has to match, or upserts fail outright. This is the part that actually determines whether the chatbot is trustworthy or just confidently wrong. Two decisions matter here: how many chunks to retrieve, and what to do when none of them are actually relevant. js const RELEVANCE THRESHOLD: number = 0.58; const TOP K: number = 3; export function filterRelevant matches: VectorizeMatch , threshold = RELEVANCE THRESHOLD { return matches.filter match = match.score threshold ; } async function QueryVector question: string, c: Context<AppEnv { const modelResp = await c.env.AI.run '@cf/baai/bge-base-en-v1.5', { text: question } ; const vector = modelResp.data 0 ; // async/queued-response guard omitted for brevity let { matches } = await c.env.VECTORIZE.query vector, { topK: TOP K } ; matches = filterRelevant matches ; const notes: string = ; const citeIds: number = ; for const match of matches { // look up match.id's text in D1, push it and its id onto notes / citeIds } const contextMessage = notes.length ? Context:\n${notes.map note = - ${note} .join '\n' } : ''; return { contextMessage, citeIds }; } topK: 3 retrieves up to three candidates - but filterRelevant can still drop all of them if none clear the threshold, which is the point: Vectorize will always hand back something close to your query vector, even if "closest" still means "not actually related." Force-using the nearest match regardless of score is how you get a chatbot that answers every question with unwarranted confidence. That empty-context case gets handled explicitly on the generation side, not silently: async function LlmWithRag c: Context<AppEnv , question: string, contextMessage: string { const systemPrompt = contextMessage.length 0 ? You are a helpful assistant. Answer using the context provided.\n\n${contextMessage} : 'You are a helpful assistant. No relevant notes were found for this question - say so plainly rather than guessing.'; const modelResp = await c.env.AI.run '@cf/qwen/qwen3.8-27b', { messages: { role: 'system', content: systemPrompt }, { role: 'user', content: question }, , } ; return modelResp.choices 0 ?.message?.content; // empty-response handling omitted for brevity } Two different system prompts, chosen by whether retrieval actually found anything - not one prompt with a vague "use context if relevant" hedge and a hope the model interprets "no context" correctly on its own. The /api/query route ties it together, and returns which notes the answer is grounded in, not just the answer text: js app.get '/api/query', async c = { const question = c.req.query 'text' ; if question return c.text 'specify text in ?text query', 400 ; const { contextMessage, citeIds } = await QueryVector question, c ; const answer = await LlmWithRag c, question, contextMessage ; return c.json { llmAnswer: answer, citedNoteIds: citeIds }, 200 ; } ; That's a small thing, but it's what makes the admin panel's delete button actually useful - if an answer traces back to a bad or stale note, you know exactly which row to remove. A few things need to be in place first: the D1 migration applied, and a real ADMIN TOKEN set the admin endpoints below won't authenticate without one . npx wrangler d1 migrations apply database --local cp .dev.vars.example .dev.vars then set a real ADMIN TOKEN npm run dev With that running: Ask a question curl -G "http://127.0.0.1:8787/api/query" --data-urlencode "text=How long is a Personal Access Token valid for?" → {"llmAnswer":"...", "citedNoteIds": 3 } Upload a document raw markdown as the body - no JSON wrapping needed for a single field curl -X POST http://127.0.0.1:8787/admin/ingest \ -H "Content-Type: text/markdown" \ -H "Authorization: Bearer $ADMIN TOKEN" \ --data-binary @my-notes.md → {"message":"Ingestion started"} List everything currently ingested curl -H "Authorization: Bearer $ADMIN TOKEN" http://127.0.0.1:8787/admin/notes Delete a chunk from both D1 and Vectorize curl -X DELETE -H "Authorization: Bearer $ADMIN TOKEN" http://127.0.0.1:8787/admin/notes/3 Worth noting on that upload: the body is raw markdown text, not JSON - since a browser File object can be handed straight to fetch as the body, and /admin/ingest just reads it with c.req.text . No JSON escaping of arbitrary file content, no encoding surprises to debug - the fewer places multi-line text has to survive being stringified and parsed back out, the fewer ways it can get mangled in transit. Everything above is the same /api/query call you just made with curl - the actual chat widget is a plain HTML form wrapping it, no framework: js formEl.addEventListener 'submit', async e = { e.preventDefault ; const question = inputEl.value.trim ; if question return; inputEl.value = ''; addMessage 'user', question ; submitBtn.disabled = true; try { const response = await fetch '/api/query?' + new URLSearchParams { text: question } ; if response.ok { addMessage 'assistant error', Request failed ${response.status} ; return; } const { llmAnswer, citedNoteIds } = await response.json ; addMessage 'assistant', ${llmAnswer}\n\n from note ${citedNoteIds.join ', ' } ; } finally { submitBtn.disabled = false; } } ; citedNoteIds — the same array QueryVector builds up earlier — is what turns into that from note 62, 74, 63 line under the answer. It's a small thing, but it's the difference between a chatbot that just asserts something and one that shows its work. A collapsible panel gated behind ADMIN TOKEN , stored in sessionStorage cleared on tab close - short-lived by design, since it's an admin credential typed into a page anyone with the URL can load . Once unlocked: a file picker for .md uploads, and a live table of every ingested chunk with a delete button per row. Worth being precise about what that lock actually protects, since it's easy to overstate: the client-side gate is a convenience wrapper, not the real security boundary. The actual enforcement is bearerAuth on the server - anyone who has the token can hit /admin/ingest or DELETE /admin/notes/:id directly with curl , regardless of what the UI shows. The lock exists so you don't have to remember curl syntax to manage your own content, not to keep out someone who already has the token. topK alone isn't a quality control - a similarity search always returns step.do - loop around it, not inside it, or a single failure forces a full batch to redo. Full code: github.com/palermo-777/chatbot-with-rag https://github.com/palermo-777/chatbot-with-rag . Happy to hear what you'd tune differently on the threshold or the chunking strategy - drop a comment.