{"slug": "the-ai-demo-failed-but-the-database-remembered-half-of-it", "title": "The AI demo failed, but the database remembered half of it", "summary": "A developer found a critical atomicity bug in Formbricks' AI example-response generator, where a failed operation left partial data in the database and blocked retries. The fix wraps all persistence in a single transaction, ensuring one request has one outcome.", "body_md": "*This is a submission for DEV's Summer Bug Smash: Clear the Lineup.*\n\nThe error message said the operation had failed. PostgreSQL told a different\n\nstory.\n\nI was reading Formbricks' AI example-response generator when I found a sequence\n\nof writes that looked individually reasonable: create a tag, create a display,\n\ncreate a response, evaluate quotas, link the tag, repeat. But each completed\n\npiece could commit before the next one began.\n\nSo I made the third response fail.\n\nThe action rejected, exactly as the UI would expect. Underneath it, the first\n\ntwo responses were still there. So were three displays, a newly created tag and\n\ntwo tag links.\n\n| Record | Expected | Observed |\n|---|---|---|\n| Response | 0 | 2 |\n| Display | 0 | 3 |\n| newly-created Tag | 0 | 1 |\n| TagsOnResponses | 0 | 2 |\n\nThe third display had been written before response creation failed. The error\n\nwas real, but so was half the dataset.\n\nThen came the part that made this more than a cleanup problem. Formbricks only\n\nallows example generation while a survey has zero responses. After the\n\nexisting rate-limit window, retrying the operation hit that guard. The failed\n\nattempt had left two responses behind, so its own debris prevented recovery.\n\nThe operation reported failure, changed the database and then used that\n\nchange as the reason the user could not try again.\n\nThat is the invariant I set out to restore: one request to generate example\n\nresponses must have one persistence outcome. Either the complete synthetic\n\ndataset commits, or none of that attempt remains.\n\n[Formbricks](https://github.com/formbricks/formbricks) is an open-source\n\nexperience-management platform for building surveys and analyzing responses.\n\nIts Survey Summary can generate example responses so a team can explore the\n\nanalytics experience before collecting real data.\n\nThat sounds like a small demo feature. Its persistence path is not small.\n\nOne generation creates a generated-response tag, one Display and one Response\n\nfor each synthetic submission, response timestamps, quota-evaluation links,\n\ntag links and additional impression-only Displays. The production path\n\ngenerates 20 example responses.\n\nFrom a person's perspective, that is one button press. Before this patch, the\n\ndatabase saw a collection of separately committed operations.\n\nThe distinction matters because a user-level operation does not become atomic\n\njust because every helper has a transaction somewhere inside it. If helper A\n\ncommits, helper B commits and helper C rolls back, the database has faithfully\n\nprotected three different operations. The user only asked for one.\n\nI wanted a reproduction I could run on demand. I replaced the external model\n\nboundary with a deterministic four-response dataset and injected a failure\n\nwhile response 3 was being created. Four responses keep the baseline small\n\nenough to inspect; the final validation separately exercises the\n\nproduction-sized batch of 20.\n\nBefore the fix, response 1 and response 2 committed. Their tag links committed.\n\nThe Display belonging to response 3 also committed because it was created\n\nbefore the injected failure. The enclosing action rejected, but there was no\n\nenclosing database transaction capable of undoing the earlier work.\n\nThe [baseline characterization\ntest](https://github.com/JuanTorchia/formbricks/blob/d5c5f381c994bafa2efdb7662e5468e6c887843d/apps/web/integration/ai-example-response-atomicity.test.ts)\n\nThis proves a mechanism, not its production frequency. I do not know how many\n\nusers have encountered a mid-batch failure, how often it happens or what it has\n\ncost. I did not find this through a production incident report. The defensible\n\nclaim is narrower: under a deterministic failure, the operation left the\n\nmeasured partial state above and made a later retry fail the zero-response\n\nguard.\n\nMy first fix was the fix most of us would sketch immediately: start one outer\n\nPrisma transaction before persistence and pass its client into the writes.\n\nIt solved the first rollback test. Then I constrained Prisma to one database\n\nconnection.\n\nThe test expired after about five seconds.\n\nThe outer transaction owned the only connection, but organization, workspace,\n\nsurvey and quota services still performed reads through the global Prisma\n\nclient. Those reads asked the pool for another connection. There was no other\n\nconnection. The transaction waited on code inside itself until it expired.\n\nIncreasing the timeout would only make the deadlock-shaped wait longer. The\n\nproblem was not that the transaction needed more patience. The problem was\n\nthat I had drawn a boundary in one function while the call graph quietly\n\ncrossed it.\n\nThat one-connection test changed the implementation. Stable survey, quota and\n\nworkspace-to-organization context is now loaded once through the transaction\n\nclient. Mutable quota counts and every write also stay on that client. Existing\n\ncallers keep their cached, global path; the generated-response path opts into a\n\nnarrow persistence context tied to its caller-owned transaction.\n\nIt left me with a rule I trust more than the green test I had before:\n\nA transaction boundary is only real if every database operation inside it\n\nuses the same transaction client.\n\nPutting the model call inside the transaction would hold a database connection\n\nand lock while waiting on an external service. That makes atomicity expensive\n\nin exactly the wrong place, so generation remains outside.\n\nBut that creates a race. Two collaborators can both observe zero responses,\n\nstart model generation and return with valid datasets. A real respondent can\n\nalso submit while the model is running. The survey may be archived. Ownership\n\nmay no longer match the snapshot used to start the action.\n\nThe transaction therefore acquires the survey lock *after* generation and\n\nrevalidates the state that authorizes persistence:\n\nOnly then does it load the stable persistence context and write the synthetic\n\nbatch.\n\nMy first instinct was `SELECT ... FOR UPDATE`\n\n. It serializes competing\n\ngenerators, but it can also conflict with the `FOR KEY SHARE`\n\nlock PostgreSQL\n\nuses when a normal Response or Display insert validates its foreign key to the\n\nSurvey.\n\nThe example generator should protect its own batch. It should not make a real\n\nrespondent wait just because a synthetic demo is being persisted.\n\nThe final design uses `FOR NO KEY UPDATE`\n\n:\n\n``` js\nreturn await prisma.$transaction(\n  async (tx) => {\n    await tx.$queryRaw`\n      SELECT id\n      FROM \"Survey\"\n      WHERE id = ${survey.id}\n      FOR NO KEY UPDATE\n    `;\n\n    // Revalidate archive state, ownership and zero Responses.\n    // Load stable Survey and quota context through tx.\n    // Persist every synthetic entity through tx.\n  },\n  { timeout: 10_000 }\n);\n```\n\nThat lock still serializes competing generators and Survey updates, while\n\nremaining compatible with the foreign-key lock used by normal inserts.\n\nI tested the distinction directly: pause example persistence at response 3,\n\ninsert a real Response from a second connection, and verify that the real\n\ninsert completes before the example transaction is released.\n\nThe winning lock was not the one with the most intimidating name. It was the\n\nweakest lock that protected the invariant without placing the demo ahead of a\n\nreal person.\n\nQuota evaluation introduced another boundary. For normal response intake,\n\nFormbricks historically treats some quota database errors as best-effort: log\n\nthe problem and continue accepting the response. Changing that globally would\n\nturn this bug fix into an unrelated compatibility decision.\n\nFor an atomic generated batch, however, swallowing a quota-link failure would\n\ncommit another kind of partial dataset.\n\nThe dedicated example-response context therefore selects strict propagation.\n\nA real PostgreSQL `P2003`\n\nforeign-key error during quota-link creation is\n\nre-thrown and aborts the outer transaction. Existing response callers preserve\n\ntheir original API and best-effort behavior.\n\nThat asymmetry is intentional. Reliability work is not making every path\n\nstricter. It is deciding which failures each path is allowed to survive.\n\nThe model request stays outside the transaction. Inside it, the patch locks and\n\nrevalidates the Survey, loads stable evaluation context through `tx`\n\n, creates\n\nall 20 responses and their related records through that same client, bulk\n\ninserts tag links and adds the remaining impression-only Displays before\n\ncommit.\n\nThe candidate patch also adds runtime guards so the atomic context cannot be\n\nused without its transaction, across surveys, with quotas from another survey\n\nor for contact-linked responses. A type that looks correct at one call site is\n\nnot enough protection for shared persistence code.\n\nThe completed regression matrix is broader than “the happy path still worksâ€Â:\n\n| Scenario | Verified result |\n|---|---|\n| failure while creating response 3 | zero synthetic rows; later retry succeeds |\nquota-link foreign-key failure (`P2003` ) |\ncomplete rollback |\ntransaction expiration (`P2028` ) |\ncomplete rollback; later retry succeeds |\n| two generators race | one complete dataset; one domain rejection |\n| real Response arrives during model generation | generated batch rejected |\n| real Response arrives during persistence | real insert is not blocked |\n| Survey archived during generation | generated batch rejected |\n| generated Tag already exists | preexisting Tag survives rollback |\n| Prisma pool has one connection | complete response and quota path succeeds |\n\nThe real PostgreSQL harness covers the production schema, ownership checks,\n\nDisplay creation, v1 Response persistence, quota lookup and evaluation,\n\nquota-link writes and bulk tag assignment.\n\n```\nFocused compatibility run: 5 files passed, 158 tests passed\nPostgreSQL harness:         2 files passed,   8 tests passed\nOne-connection full path:  1 file passed,    2 tests passed\n```\n\nThe PostgreSQL results were recorded before a later type-only follow-up. That\n\nfollow-up changed how the same quota-evaluation payload is constructed so\n\nTypeScript preserves its discriminated union; it did not change the runtime\n\nvalues or transaction behavior. I attempted a PostgreSQL rerun, but my local\n\ndatabase service was offline, so I am stating the gap instead of pretending the\n\nrerun happened.\n\nOn the final commit, the fork's hosted Formbricks web build, official unit\n\ntests, linters and Helm validation passed. The hosted E2E job stopped in one\n\nsecond without executing test steps in the fork environment, and SonarQube\n\ncould not authenticate without its repository secret. I will not compress that\n\nmixed result into “CI is green.â€Â\n\nFor the tested failure modes, a generated dataset commits completely or rolls\n\nback completely. A failed transaction no longer leaves synthetic Responses\n\nthat create the persistent zero-response retry block. Competing persistence is\n\nserialized per Survey, state is revalidated after generation, quota capacity\n\nordering is preserved and normal response insertion is not blocked by the\n\nSurvey lock.\n\nThe external model call is outside the database transaction. Two callers may\n\nstill pay for duplicate model work before persistence serialization lets one\n\nwin and rejects the other. This is not exactly-once generation.\n\nThe explicit 10-second transaction timeout was validated locally. It is not a\n\nuniversal promise for every remote or heavily loaded PostgreSQL deployment.\n\nThis work also does not cover a process crash outside PostgreSQL transaction\n\nsemantics, establish production frequency or prove that a lease, queue or\n\nworkflow engine is needed. Those are different claims requiring different\n\nevidence.\n\nI would rather leave a limitation visible than make the patch sound larger\n\nthan it is. Atomicity is already a strong promise. It does not need borrowed\n\ncertainty.\n\nFormbricks currently says community code contributions are accepted only in\n\nrare exceptions. I reported the reproducible bug and asked maintainers whether\n\nthey wanted the prepared fix and where they would prefer the PostgreSQL\n\nregression to live.\n\nAs of August 21, 2026, [the upstream issue remains\nopen](https://github.com/formbricks/formbricks/issues/8722), labeled as a bug,\n\nA contest deadline does not create a right to someone else's review queue. The\n\npatch and its evidence can stand in public without turning persistence into\n\npressure.\n\nThe first rollback test went green quickly. If I had stopped there, I would\n\nhave proposed a transaction that could starve its own connection pool. If I\n\nhad chosen the strongest row lock without testing a real insert beside it, I\n\nmight have protected synthetic data by delaying an actual respondent.\n\nThe difficult part was never typing `$transaction`\n\n.\n\nIt was discovering where the operation really begins and ends. It was tracing\n\nevery quiet database read that crossed that boundary. It was accepting that an\n\nexternal model belongs outside the lock, then distrusting every piece of state\n\nthat might have changed while it ran. It was choosing a lock for the work that\n\nneeded protection without making unrelated work pay for it.\n\nThere is a small debugging exercise here that travels well beyond Formbricks.\n\nFind a feature your product presents as one action, especially one that writes\n\nin a loop. Fail it in the middle. Then ignore the error message and ask the\n\ndatabase what actually happened.\n\nIf those two answers disagree, the bug is not merely a missing rollback. It is\n\na boundary nobody finished drawing.", "url": "https://wpnews.pro/news/the-ai-demo-failed-but-the-database-remembered-half-of-it", "canonical_source": "https://dev.to/jtorchia/the-ai-demo-failed-but-the-database-remembered-half-of-it-3m4a", "published_at": "2026-08-21 14:40:26+00:00", "updated_at": "2026-08-21 14:46:38.391822+00:00", "lang": "en", "topics": ["developer-tools", "artificial-intelligence"], "entities": ["Formbricks", "PostgreSQL", "Prisma", "JuanTorchia"], "alternates": {"html": "https://wpnews.pro/news/the-ai-demo-failed-but-the-database-remembered-half-of-it", "markdown": "https://wpnews.pro/news/the-ai-demo-failed-but-the-database-remembered-half-of-it.md", "text": "https://wpnews.pro/news/the-ai-demo-failed-but-the-database-remembered-half-of-it.txt", "jsonld": "https://wpnews.pro/news/the-ai-demo-failed-but-the-database-remembered-half-of-it.jsonld"}}