Stuck at "Preparing… 0/0" Forever. My State Machine Had No Exit.

This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry.

TextStack is an open-source reader for technical books, built in .NET. It has an "Ask this book" feature: RAG over the book you uploaded. Before you can ask, the book must be indexed: split into chunks, and for PDFs each page goes through a paid vision parse that takes minutes. The code is public: github.com/mrviduus/textstack.

A user clicks "Ask this book". The screen shows "Preparing… 0/0". And for some books it stayed there forever. Not slow. Forever. The retry button did nothing. The only cure was me poking the database by hand.

Here is what the old flow did:

POST /me/books/{id}/index
  -> flip rag_status to Indexing        (in the endpoint)
  -> run the whole chunking inline      (minutes of vision parsing, still in the HTTP request)
  -> return 202 when done

Two problems living inside one line: the work ran inside the HTTP request, and the status flip happened before the work.

A vision parse of a big PDF runs for many minutes. Cloudflare cuts the request long before that. The API container also restarts on every deploy. In both cases the process doing the work dies quietly, and the row stays like this:

rag_status = Indexing, rag_chunk_count = 0

And here is the real bug. The retry path only re-claimed rows in status NotIndexed

or Failed

:

// old claim: an Indexing row can never be claimed again
if (book.RagStatus is RagIndexStatus.NotIndexed or RagIndexStatus.Failed) { ... }

Indexing

with zero chunks was a state with no exit. Not failed, so no retry. Not ready, so no answers. A dead end that looks like progress. "Preparing… 0/0", forever.

The shape of the fix: the endpoint only claims and answers, a background worker does the work, and every path ends in a terminal state. (Full code: RagIndexingService.cs and RagIndexingWorker.cs.)

1. The endpoint returns 202 immediately. It flips the status and goes home. No paid work on the HTTP path, nothing for Cloudflare to kill.

2. A worker claims the row atomically. No locks, no queue infrastructure. One UPDATE where the WHERE clause is the whole concurrency story:

var claimed = await db.Database.ExecuteSqlInterpolatedAsync($"""
    UPDATE user_books
    SET rag_indexing_started_at = now()
    WHERE id = {bookId}
      AND rag_status = 1              -- Indexing
      AND rag_chunk_count = 0
      AND rag_indexing_started_at IS NULL;
    """, ct);
if (claimed == 0) return;             // someone else owns it, walk away

If two workers pick the same row, one gets rowcount 1 and proceeds, the other gets 0 and returns. Postgres is the referee.

3. A sweep recovers dead rows. Every 30 seconds the worker also checks: any row still Indexing

with zero chunks whose rag_indexing_started_at

is older than 15 minutes means the process that claimed it died mid-parse. That row flips to a terminal Failed

with a human-readable reason:

UPDATE user_books
SET rag_status = 3, rag_error = 'indexing interrupted, retry'
WHERE rag_status = 1 AND rag_chunk_count = 0
  AND rag_indexing_started_at < {staleCutoff};

Note what it does NOT do: auto-requeue. A vision parse is real paid spend. The row fails loudly, the user re-triggers deliberately. Retry is a human decision when every retry costs money.

The failure path had its own bug waiting. When indexing dies because the worker is shutting down, the cancellation token is already cancelled. If you write the Failed

status using that same token, the error write gets cancelled too, and you are back to the forever-Indexing dead end you just fixed.

// the terminal write runs on a FRESH context with CancellationToken.None:
// the token that killed the work must not kill the record of its death
await using var db = await dbFactory.CreateDbContextAsync(CancellationToken.None);

The same cancellation also produces a second trap: the chunker returns 0 chunks on shutdown, which looks identical to a genuinely empty book. Without checking ct.IsCancellationRequested

, a perfectly good book gets labeled "No chapters to index" and a user believes their upload is broken. Zero is the least trustworthy number in a distributed system.

A status enum is not a state machine until every state has an exit. Indexing

was a real status, rendered nicely in the UI, and it was also a trap: no transition out except success. The fix was not clever code. It was drawing the diagram and asking one question about every state: how does a row leave here if the process dies right now?

And the operational rule on top: never let an HTTP request own work that outlives an HTTP request. The request is a messenger, not a worker.

What is the longest a "temporary" status has survived in your database? I found rows that had been "Preparing" for weeks.

I build TextStack, an open-source reader for technical books, in .NET. This fix is from the indexing pipeline behind its "Ask this book" feature. github.com/mrviduus/textstack