Stuck at "Preparing… 0/0" Forever. My State Machine Had No Exit.

A developer fixed a state machine bug in TextStack, an open-source .NET reader for technical books, where a RAG indexing process could get stuck in a 'Preparing… 0/0' state forever. The issue occurred because the indexing work ran inside the HTTP request and could be killed by Cloudflare or a deploy, leaving the database row in an 'Indexing' status with zero chunks—a state with no exit. The fix moves the work to a background worker, uses atomic row claiming, and adds a sweep to recover dead rows.

This is a submission for DEV's Summer Bug Smash: Clear the Lineup powered by Sentry. TextStack is an open-source reader for technical books, built in .NET. It has an "Ask this book" feature: RAG over the book you uploaded. Before you can ask, the book must be indexed: split into chunks, and for PDFs each page goes through a paid vision parse that takes minutes. The code is public: github.com/mrviduus/textstack https://github.com/mrviduus/textstack . A user clicks "Ask this book". The screen shows "Preparing… 0/0". And for some books it stayed there forever. Not slow. Forever. The retry button did nothing. The only cure was me poking the database by hand. Here is what the old flow did: php POST /me/books/{id}/index - flip rag status to Indexing in the endpoint - run the whole chunking inline minutes of vision parsing, still in the HTTP request - return 202 when done Two problems living inside one line: the work ran inside the HTTP request, and the status flip happened before the work. A vision parse of a big PDF runs for many minutes. Cloudflare cuts the request long before that. The API container also restarts on every deploy. In both cases the process doing the work dies quietly, and the row stays like this: rag status = Indexing, rag chunk count = 0 And here is the real bug. The retry path only re-claimed rows in status NotIndexed or Failed : // old claim: an Indexing row can never be claimed again if book.RagStatus is RagIndexStatus.NotIndexed or RagIndexStatus.Failed { ... } Indexing with zero chunks was a state with no exit. Not failed, so no retry. Not ready, so no answers. A dead end that looks like progress. "Preparing… 0/0", forever. The shape of the fix: the endpoint only claims and answers, a background worker does the work, and every path ends in a terminal state. Full code: RagIndexingService.cs https://github.com/mrviduus/textstack/blob/main/backend/src/Worker/Services/RagIndexingService.cs and RagIndexingWorker.cs https://github.com/mrviduus/textstack/blob/main/backend/src/Worker/Services/RagIndexingWorker.cs . 1. The endpoint returns 202 immediately. It flips the status and goes home. No paid work on the HTTP path, nothing for Cloudflare to kill. 2. A worker claims the row atomically. No locks, no queue infrastructure. One UPDATE where the WHERE clause is the whole concurrency story: js var claimed = await db.Database.ExecuteSqlInterpolatedAsync $""" UPDATE user books SET rag indexing started at = now WHERE id = {bookId} AND rag status = 1 -- Indexing AND rag chunk count = 0 AND rag indexing started at IS NULL; """, ct ; if claimed == 0 return; // someone else owns it, walk away If two workers pick the same row, one gets rowcount 1 and proceeds, the other gets 0 and returns. Postgres is the referee. 3. A sweep recovers dead rows. Every 30 seconds the worker also checks: any row still Indexing with zero chunks whose rag indexing started at is older than 15 minutes means the process that claimed it died mid-parse. That row flips to a terminal Failed with a human-readable reason: UPDATE user books SET rag status = 3, rag error = 'indexing interrupted, retry' WHERE rag status = 1 AND rag chunk count = 0 AND rag indexing started at < {staleCutoff}; Note what it does NOT do: auto-requeue. A vision parse is real paid spend. The row fails loudly, the user re-triggers deliberately. Retry is a human decision when every retry costs money. The failure path had its own bug waiting. When indexing dies because the worker is shutting down, the cancellation token is already cancelled. If you write the Failed status using that same token, the error write gets cancelled too, and you are back to the forever-Indexing dead end you just fixed. // the terminal write runs on a FRESH context with CancellationToken.None: // the token that killed the work must not kill the record of its death await using var db = await dbFactory.CreateDbContextAsync CancellationToken.None ; The same cancellation also produces a second trap: the chunker returns 0 chunks on shutdown, which looks identical to a genuinely empty book. Without checking ct.IsCancellationRequested , a perfectly good book gets labeled "No chapters to index" and a user believes their upload is broken. Zero is the least trustworthy number in a distributed system. | Before | After | | |---|---|---| | Work runs | inside the HTTP request | in a background worker | | Request dies mid-parse | row stuck Indexing forever | sweep flips it to Failed in 15 min | | Retry button | dead for stuck rows | works, Failed is claimable | | Shutdown during parse | "No chapters to index" wrong | "indexing interrupted, retry" | | Concurrency control | none needed and none possible | one atomic UPDATE, rowcount as the lock | A status enum is not a state machine until every state has an exit. Indexing was a real status, rendered nicely in the UI, and it was also a trap: no transition out except success. The fix was not clever code. It was drawing the diagram and asking one question about every state: how does a row leave here if the process dies right now? And the operational rule on top: never let an HTTP request own work that outlives an HTTP request. The request is a messenger, not a worker. What is the longest a "temporary" status has survived in your database? I found rows that had been "Preparing" for weeks. I build TextStack, an open-source reader for technical books, in .NET. This fix is from the indexing pipeline behind its "Ask this book" feature. github.com/mrviduus/textstack