# The Database I Deleted: Migrating an Agentic RAG App to AWS Serverless #3

> Source: <https://dev.to/dmitriy_trunov_9a09a497b1/the-database-i-deleted-migrating-an-agentic-rag-app-to-aws-serverless-3-k0d>
> Published: 2026-09-19 19:54:30+00:00

[Part 1](https://dev.to/dmitriy_trunov_9a09a497b1/migrating-an-agentic-rag-app-to-aws-serverless-d2k) covered why an agentic RAG assistant moved off a single EC2 box onto AWS serverless, and the resulting architecture. [Part 2](https://dev.to/dmitriy_trunov_9a09a497b1/the-database-i-deleted-migrating-an-agentic-rag-app-to-aws-serverless-n18) walked through five decisions that didn't survive contact with AWS's fine print: a database that turned out to be a file, why there's no streaming, an auth design CloudFront can't do, what Bedrock batch inference can't give you, and where the vector text actually lives. This part is the payoff — what the migration found, what it cost, and what's still unverified.

Re-deriving the corpus surfaced two problems that had nothing to do with AWS.

The original derives a chunk id as `{repo}::{file_path}::{section}`. Two chunks in one file can share a composed heading — a repeated "Usage", an "Example" under two parents. I measured it:

```
doc_id collisions (repo::file::section): 303   (of 24,775 chunks)
```

RRF dedupes by `doc_id`. So 303 chunks were silently shadowing each other during fusion — one of each pair could never surface, and which one survived depended on iteration order. A pre-existing retrieval bug, invisible from the outside, sitting in a system whose evaluation numbers I'd published.

Adding a per-file ordinal to the hash takes it to zero.

Chunk sizes look fine at the median and awful in the tail:

```
          chunks    p50     p90      p99      max
before    24,775    602   2,797   10,483  119,786   bytes
after     25,482    621   3,096    7,905    7,998
```

That 119 KB chunk is roughly 30,000 tokens. **Titan Text Embeddings caps input at 8,192.** It would not have embedded — it would have failed, or been silently truncated, and simply been missing from the vector index.

The fix splits on paragraph boundaries, with a hard fallback for unbroken tables and code blocks, because "never split mid-paragraph" is a nice principle right up until it leaves you with a chunk that no embedding model will accept.

The crawler paces itself against GitHub's quota and sleeps until reset when it runs low. That's correct for a local run that owns the whole crawl. Inside a Lambda it is wrong twice over:

`maxConcurrency` bounds concurrent So the crawler now raises instead of sleeping, carrying the reset time, and the state machine catches it into a `Wait` — which costs nothing and can span an hour:

```
crawl_task.add_catch(
    wait_for_quota.next(task("CrawlRepoRetryTask", crawl_repo)),
    errors=["RateLimitedError"],
    result_path="$.error",
)
```

`GITHUB_RATE_LIMIT_MODE` keeps the sleeping behaviour for local runs. And there is a test asserting the *class name* — Step Functions matches on the error name the runtime reports, so renaming the exception would silently break the Wait path with no import error to warn you.

The original's proudest feature was an atomic budget reservation: not "read the total, compare, spend" — which races under concurrent traffic — but one `UPDATE ... WHERE ... RETURNING` that Postgres serializes.

That property transfers exactly onto a DynamoDB conditional write:

```
get_table().update_item(
    Key={"pk": PK_BUDGET, "sk": window},          # window = today, UTC
    UpdateExpression="ADD total_spent :amt",
    ConditionExpression=(
        "attribute_not_exists(total_spent) OR total_spent <= :headroom"
    ),
    ExpressionAttributeValues={
        ":amt": amt,
        ":headroom": Decimal(str(cap)) - amt,
    },
)
```

Same one-round-trip check-and-deduct, single-digit milliseconds, zero idle cost. Verified against a real DynamoDB implementation (via `moto`) rather than reasoned about — 40 concurrent requests against a cap allowing 5 reservations grant **exactly 5**.

The `attribute_not_exists` branch is not decoration: `ADD` creates the attribute, but the condition still has to pass *before* it exists. Without that clause, every new day starts permanently blocked.

And one deliberate behaviour change, which is really a bug fix. The original cap was a **lifetime** total. That means one scraper — or one enthusiastic afternoon — exhausts it forever, and every future visitor sees "budget used up" for the life of the demo. Keying the budget item by UTC date makes the window structural: yesterday's spend is simply a different item, nothing has to reset anything, and the worst case is bounded to one day.

Streamlit is a WebSocket server holding per-session state. That doesn't survive contact with Lambda, so the UI became a static SPA on S3 behind CloudFront.

It is three files — `index.html`, `styles.css`, `app.js` — with no build step and no framework. That's a deliberate match to the rest of the architecture: nothing idles, few moving parts, CI syncs three files. The API is the real contract, so React later changes nothing server-side.

The design has exactly one idea. The assistant answers two fundamentally different ways, and **that distinction is the point of the project**, so colour and structure encode which mechanism answered: amber for exact metadata queries, teal for document retrieval, split down the bar when a question needed both. The same coding runs through the question index, so you can see *before* clicking which path a question will take.

I built it, then looked at it in a browser, which caught four things reasoning had not:

`.view { display: grid }` beats the browser's `[hidden] { display: none }`, so the History view rendered underneath the chat. Invisible in review, obvious in a screenshot.
None of those are hard. All of them needed eyes.

|  | Original (EC2) | Serverless | 
|---|---|---|
| Idle cost | ~$15/month | **~$2/month** | 
| At ~500 questions/month | ~$15/month | **~$5–8/month** | 
| Deployable artifact | multi-GB image | **~7 MB zip** | 
| Runtime dependencies | torch, sentence-transformers, minsearch, streamlit, pandas, psycopg, openai | **boto3** + scrape/parse stack | 
| Cold start | n/a (always on) | artifact download + init | 
| Infrastructure as code | none | 5 CDK stacks | 

The ~7 MB bundle is the single most satisfying number. Dropping the two local torch models — Titan for embeddings, Amazon Rerank for reranking — is what turns a container image into a zip.

*Unit prices are estimates and need re-checking before anyone quotes them.*

The original write-up was honest that its deep evaluation was scaffolded but unrun. This one has more to admit, and burying it would make everything above less trustworthy:

**Neither quality gate has produced a number.** And these are the two that matter:

Until they run, the claim that this migration preserved answer quality is *unmeasured*. The original's published retrieval numbers are fiction for this stack.

What *is* verified: the DynamoDB behaviour, against a real implementation, including the concurrency property above; the SQL port, against the real 279-project artifact (all 11 tools, with the leaderboard and top-projects rankings coming back genuinely different — which is the distinction the router exists to preserve); and the retrieval path end to end over the real 25,482-chunk corpus at 3–8 ms per keyword query.

Swapping OpenAI for Nova Pro is not a like-for-like substitution, and the place it will show is routing.

The router makes a 12-way decision with genuinely subtle distinctions. "Top project score" and "leaderboard total score" are different numbers that must not be conflated. A person lookup is *required* to chain into a follow-up document search, because a bare theme label is an uninformative answer. Those instructions were tuned against a stronger model.

Here's why the obvious evaluation won't catch a regression: ask "who's the top scorer on the leaderboard?", have the model call `top_projects_by_score` instead of `top_authors_by_total_score`, and you get a fluent, confident, well-grounded-sounding answer built on **the wrong number**. An LLM judge scoring answer-against-context will happily mark it RELEVANT. It *is* consistent with its context. It's just answering a different question.

Only the tool trace exposes that. Which is why the gate compares routing rather than relevance, and why the pass bar gets set before the eval runs rather than after.

**Price the floor, not the feature.** Aurora Serverless v2 is the right answer to a lot of questions. It was the wrong answer to "where do I put 88 KB of read-only data that a pipeline rebuilds," and the only way to see that was to look at what the database actually held rather than what it was called.

**Check what your runtime can actually do before designing around it.** Python Lambda can't stream. CloudFront OAC can't sign a POST body. Bedrock batch can't constrain output. Each of those invalidated a design I'd already committed to on paper, and each took minutes to discover once I looked.

**A migration re-derives your data, which audits it.** Two real bugs in the original — 303 unreachable chunks and one chunk too large to embed — surfaced not from careful review but from rebuilding the corpus and measuring the result. If you migrate something, measure the thing you rebuilt against the thing you replaced.

**When two systems must agree, assert it in a test.** The vector filter and the SQL filter accepting the same fields is not something either system's own tests can catch. Neither is a Step Functions error-name match. Cross-system invariants need a test that exists specifically to hold them together.

**Say what you haven't verified.** A migration that "works" in the sense of compiling, synthesizing, and passing 142 tests is still a migration where no model has answered a real question. Those are very different claims, and only one of them is true here.