Building a RAG Chatbot on Cloudflare Workers (Vectorize + D1 + Workflows)

I first saw RAG in 2024, at a hackathon. The moment that stuck with me was realizing an LLM could "search" through documents I'd just uploaded and answer questions about them - not from its training data, from my content. I was genuinely astonished; it felt like a different category of capability than just chatting with a model.

Today, that's not a novelty anymore - it's close to a baseline expectation for any chatbot that needs to answer questions about content it wasn't trained on. A chatbot only knows what it was trained on - ask it about your own docs, your own product, your own notes, and it either hallucinates an answer or tells you it doesn't know. Retrieval-Augmented Generation (RAG) is the standard fix: before the model answers, you go find the actual relevant text and hand it over as context. I wanted to build the smallest version of that pattern that still behaves honestly - retrieves the right thing, admits when it can't, and doesn't lose data halfway through ingesting a document.

This is a walkthrough of chatbot-with-rag, a minimal RAG chat demo running entirely on Cloudflare:

No LangChain agent framework, no vector DB to self-host, no separate backend - one Worker, four bindings.

Two flows, running through the same Worker.

The two stores are linked by a shared id: D1's auto-increment id for a chunk is the exact same value used as that chunk's Vectorize vector id. A similarity search gives you back an id; a D1 lookup on that id gives you the actual text. No metadata duplication, no second index to keep in sync - just one id, two stores, one source of truth each for what they're good at (Vectorize for "what's similar," D1 for "what does it actually say").

The obvious version of ingestion is: split the doc, loop over the chunks, embed and insert each one, done. The problem is what happens when chunk 7 of 12 fails - a rate-limited embedding call, a transient D1 failure, doesn't matter. A plain loop either crashes the whole request (losing all 12 chunks) or needs you to implement retry logic.

Cloudflare Workflows solve this by making each step.do() call a durable checkpoint. If a step fails, only that step retries - everything before it is already saved and never redone:

export class RAGWorkflow extends WorkflowEntrypoint<Env, RagWorkflowParams> {
    async run(event: WorkflowEvent<RagWorkflowParams>, step: WorkflowStep) {
        const { data } = event.payload;

        const texts = await step.do('split text', async () => {
            // split `data` into chunks with RecursiveCharacterTextSplitter
        });

        for (const [i, text] of texts.entries()) {
            const record = await step.do(`store in D1 db: ${i}/${texts.length}`, async () => {
                // insert the chunk's text into D1, return its new row id
            });

            const vector = await step.do(`Generate Embeddings: ${i}/${texts.length}`, async () => {
                // embed the chunk's text via Workers AI (bge-base-en-v1.5)
            });

            await step.do(`Insert Vector: ${i}/${texts.length}`, async () => {
                // upsert { id: record, values: vector } into Vectorize
            });
        }
    }
}

One thing worth being explicit about, because it's an easy trap: the loop is around step.do(), not inside it. Put a loop of several inserts inside a single step.do() call and Workflows can only checkpoint the whole step as one unit - a failure on item 7 means the entire step retries, re-inserting items 1 through 6 all over again. Giving each chunk its own uniquely-named step (store in D1 db: 3/12, not a bare store in D1 db reused across iterations - step names are the cache key Workflows uses to track what's already done) is what actually buys you per-chunk retry instead of per-batch retry.

Notice too that record- the id step.do() returns from the D1 insert:

const record = await step.do(`store in D1 db: ${i}/${texts.length}`,
 async () => { /* ... */ });

reappears two steps later in the Vectorize upsert:

env.VECTORIZE.upsert([{ id: record.toString(), values: vector }]);

That's not incidental: whatever a step.do() callback returns is exactly what Workflows durably persists as that step's result, so a later step can reuse it for free instead of recomputing or re-fetching it. It's the same relationship the architecture diagram's dashed chunk id arrow is pointing at - a value produced by one step, consumed by a later one, with nothing in between needing to know where it came from.

Four bindings in wrangler.jsonc - Workers AI, Vectorize, D1, and the Workflow itself:

{
    "ai": { "binding": "AI" },
    "vectorize": [{ "binding": "VECTORIZE", "index_name": "vector-index" }],
    "d1_databases": [{ "binding": "database", "database_name": "database", "database_id": "..." }],
    "workflows": [{ "name": "rag", "binding": "RAG_WORKFLOW", "class_name": "RAGWorkflow" }],
    "assets": { "directory": "./public/", "binding": "ASSETS" }
}

Vectorize needs to exist before anything can be upserted into it:

npx wrangler vectorize create vector-index --dimensions=768 --metric=cosine

768 dimensions because that's what @cf/baai/bge-base-en-v1.5 - the embedding model - outputs. If you swap embedding models later, the dimension has to match, or upserts fail outright.

This is the part that actually determines whether the chatbot is trustworthy or just confidently wrong. Two decisions matter here: how many chunks to retrieve, and what to do when none of them are actually relevant.

const RELEVANCE_THRESHOLD: number = 0.58;
const TOP_K: number = 3;

export function filterRelevant(matches: VectorizeMatch[], threshold = RELEVANCE_THRESHOLD) {
    return matches.filter((match) => match.score > threshold);
}

async function QueryVector(question: string, c: Context<AppEnv>) {
    const modelResp = await c.env.AI.run('@cf/baai/bge-base-en-v1.5', { text: question });
    const vector = modelResp.data[0]; // (async/queued-response guard omitted for brevity)

    let { matches } = await c.env.VECTORIZE.query(vector, { topK: TOP_K });
    matches = filterRelevant(matches);

    const notes: string[] = [];
    const citeIds: number[] = [];
    for (const match of matches) {
        // look up match.id's text in D1, push it (and its id) onto notes / citeIds
    }

    const contextMessage = notes.length ? `Context:\n${notes.map((note) => `- ${note}`).join('\n')}` : '';
    return { contextMessage, citeIds };
}

topK: 3 retrieves up to three candidates - but filterRelevant can still drop all of them if none clear the threshold, which is the point: Vectorize will always hand back something close to your query vector, even if "closest" still means "not actually related." Force-using the nearest match regardless of score is how you get a chatbot that answers every question with unwarranted confidence.

That empty-context case gets handled explicitly on the generation side, not silently:

async function LlmWithRag(c: Context<AppEnv>, question: string, contextMessage: string) {
    const systemPrompt = contextMessage.length > 0
        ? `You are a helpful assistant. Answer using the context provided.\n\n${contextMessage}`
        : 'You are a helpful assistant. No relevant notes were found for this question - say so plainly rather than guessing.';

    const modelResp = await c.env.AI.run('@cf/qwen/qwen3.8-27b', {
        messages: [
            { role: 'system', content: systemPrompt },
            { role: 'user', content: question },
        ],
    });

    return modelResp.choices[0]?.message?.content; // (empty-response handling omitted for brevity)
}

Two different system prompts, chosen by whether retrieval actually found anything - not one prompt with a vague "use context if relevant" hedge and a hope the model interprets "no context" correctly on its own.

The /api/query route ties it together, and returns which notes the answer is grounded in, not just the answer text:

app.get('/api/query', async (c) => {
    const question = c.req.query('text');
    if (!question) return c.text('specify text in ?text query', 400);

    const { contextMessage, citeIds } = await QueryVector(question, c);
    const answer = await LlmWithRag(c, question, contextMessage);

    return c.json({ llmAnswer: answer, citedNoteIds: citeIds }, 200);
});

That's a small thing, but it's what makes the admin panel's delete button actually useful - if an answer traces back to a bad or stale note, you know exactly which row to remove.

A few things need to be in place first: the D1 migration applied, and a real ADMIN_TOKEN set (the admin endpoints below won't authenticate without one).

npx wrangler d1 migrations apply database --local
cp .dev.vars.example .dev.vars   # then set a real ADMIN_TOKEN
npm run dev

With that running:

curl -G "http://127.0.0.1:8787/api/query" --data-urlencode "text=How long is a Personal Access Token valid for?"

curl -X POST http://127.0.0.1:8787/admin/ingest \
  -H "Content-Type: text/markdown" \
  -H "Authorization: Bearer $ADMIN_TOKEN" \
  --data-binary @my-notes.md

curl -H "Authorization: Bearer $ADMIN_TOKEN" http://127.0.0.1:8787/admin/notes

curl -X DELETE -H "Authorization: Bearer $ADMIN_TOKEN" http://127.0.0.1:8787/admin/notes/3

Worth noting on that upload: the body is raw markdown text, not JSON - since a browser File object can be handed straight to fetch() as the body, and /admin/ingest just reads it with c.req.text(). No JSON escaping of arbitrary file content, no encoding surprises to debug - the fewer places multi-line text has to survive being stringified and parsed back out, the fewer ways it can get mangled in transit.

Everything above is the same /api/query call you just made with curl - the actual chat widget is a plain HTML form wrapping it, no framework:

formEl.addEventListener('submit', async (e) => {
    e.preventDefault();
    const question = inputEl.value.trim();
    if (!question) return;
    inputEl.value = '';
    addMessage('user', question);

    submitBtn.disabled = true;
    try {
        const response = await fetch('/api/query?' + new URLSearchParams({ text: question }));
        if (!response.ok) {
            addMessage('assistant error', `Request failed (${response.status})`);
            return;
        }
        const { llmAnswer, citedNoteIds } = await response.json();
        addMessage('assistant', `${llmAnswer}\n\n (from note #${citedNoteIds.join(', ')})`);
    } finally {
        submitBtn.disabled = false;
    }
});

citedNoteIds — the same array QueryVector builds up earlier — is what turns into that (from note #62, 74, 63) line under the answer. It's a small thing, but it's the difference between a chatbot that just asserts something and one that shows its work.

A collapsible panel gated behind ADMIN_TOKEN, stored in sessionStorage (cleared on tab close - short-lived by design, since it's an admin credential typed into a page anyone with the URL can load). Once unlocked: a file picker for .md uploads, and a live table of every ingested chunk with a delete button per row.

Worth being precise about what that lock actually protects, since it's easy to overstate: the client-side gate is a convenience wrapper, not the real security boundary. The actual enforcement is bearerAuth on the server - anyone who has the token can hit /admin/ingest or DELETE /admin/notes/:id directly with curl, regardless of what the UI shows. The lock exists so you don't have to remember curl syntax to manage your own content, not to keep out someone who already has the token.

topK alone isn't a quality control - a similarity search always returns step.do() - loop around it, not inside it, or a single failure forces a full batch to redo. Full code: github.com/palermo-777/chatbot-with-rag. Happy to hear what you'd tune differently on the threshold or the chunking strategy - drop a comment.