{"slug": "the-card-said-one-column-the-apply-wrote-two", "title": "The card said one column. The apply wrote two.", "summary": "A developer found a critical flaw in their open-source tool @hyuga/llm-safe-sql, which lets language models propose SQL updates that are executed in a transaction, measured, and rolled back for human approval. The bug caused the tool to verify only columns that changed, not all columns the statement writes, allowing a concurrent update to be silently overwritten. The issue also affected multi-row updates where rows already matching the target value were never checked. The developer fixed the logic to separate 'changed' columns from 'written' columns and added a guard for engines like PostgreSQL and SQLite that rewrite rows even when values are unchanged.", "body_md": "I have been building a thing that lets a language model propose an `UPDATE`\n\n, then\n\nexecutes it for real inside a transaction, measures the actual before and after\n\nvalues, and always rolls back. A human reads the measurement and decides. Only\n\nthen does anything commit.\n\nThe pitch is one sentence: **what you approve is not the model's description of\nits SQL, it is what the database did when the SQL ran.**\n\nLast week I found that the thing showing you that measurement was showing you a\n\nsubset of it, and had been since the first release.\n\nReal output, from `@hyuga/llm-safe-sql@0.4.0`\n\ninstalled from npm. One row:\n\n`name = 'Tanaka'`\n\n, `postcode = '00100'`\n\n.\n\n```\n  UPDATE customers SET name='Sato', postcode='00100' WHERE id=1\n\nWhat this touches\n  customers — Customer records. The postcode is used for billing address and delivery.\n  1 row would change, across 1 column: name\n\nMeasured by running the statement and rolling it back\n  id = 1\n      name: 'Tanaka' -> 'Sato'\n```\n\nOne row, one column. `postcode`\n\nis not mentioned, and that is *correct* — it is\n\nbeing assigned the value it already holds, so nothing about it changes. The card\n\nis describing the diff accurately.\n\nApprove it. Then, before it is applied, somebody else notices the postcode is\n\nwrong and fixes it:\n\n```\nUPDATE customers SET postcode='90210' WHERE id=1;\n```\n\nNow apply the approved plan:\n\n```\nApplied: UPDATE on customers, 1 row(s), at 2026-08-10T09:49:12.049Z.\nDB now: [{\"name\":\"Sato\",\"postcode\":\"00100\"}]\n```\n\nThe fix is gone. Zero warnings. The word `postcode`\n\nnever appeared on the\n\napproval card, never appeared in the audit record, and never appeared in the\n\ncomparison the tool makes before it commits.\n\nThe diff was built like this:\n\n``` js\nconst changed: string[] = [];\nfor (const c of Object.keys(before)) {\n  if (same(before[c], after[c])) continue;   // drop what did not move\n  if (auto.has(lower(c))) continue;          // drop what the DB maintains itself\n  changed.push(c);\n}\n```\n\nThat is a correct answer to \"what should the card show\". Showing `postcode:`\n\nwould be noise, and worse than noise — it would be the card\n\n'00100' -> '00100'\n\nclaiming a change where there is none.\n\nThe problem is that `changed`\n\nwas *also* the list the apply iterated when it\n\nre-checked that nothing had moved since approval:\n\n``` js\nfor (const c of plan.changed) {\n  if (!same(live[c], plan.before[c])) throw new ApplyRefused('ROW_CHANGED', …);\n}\n```\n\n`postcode`\n\nis not in `changed`\n\n, so it is not checked. And the statement writes it\n\non every execution. **A column that is written and never verified.**\n\n\"The set of columns that change\" and \"the set of columns the statement writes\"\n\nare different sets, and I had used one name for both. `SET x = <the value it`\n\nis not exotic SQL — it is zero-padded codes, defensive\n\nalready has>`status`\n\nassignments, `SET updated_by = 'batch'`\n\n. Every one of those is this shape.\n\nThere is a second version of the same hole, one level up.\n\nWhen a `WHERE`\n\nmatches several rows and one of them already holds the target\n\nvalue, the card says:\n\n```\n  1 row would change, across 1 column: status\n  (1 more match the condition but are already correct.)\n```\n\nThat row's `changed`\n\nis empty, so the verification loop runs zero times — before\n\nthe write and after it. Nothing about that row's contents was ever checked.\n\nMySQL was saved by an accident of its protocol. It reports \"rows matched\" and\n\n\"rows changed\" separately, so a plan that measured one changed row and an apply\n\nthat changed two is a detectable disagreement. PostgreSQL and SQLite rewrite a\n\nrow even when the new values equal the old ones, so those two numbers are the\n\nsame and the comparison says nothing. The reconciliation code knew this:\n\n```\nif (plan.op === 'UPDATE' && plan.rowsChangedIsMeaningful && res.rowsChanged !== plan.rowsChanged) {\n```\n\n`rowsChangedIsMeaningful`\n\nis true on MySQL and false on the other two. **A guard\nthat worked on one of three supported engines was the only thing standing there.**\n\nThe fix was to stop conflating the two sets: `PlanRow`\n\nnow carries `covered`\n\n—\n\nevery column the statement assigns — snapshotted before and after even when the\n\nvalue does not move, included in the tamper digest, and checked at both ends of\n\nthe apply. `changed`\n\nstill drives the display. Same sequence on 0.4.2:\n\n```\nRefused (ROW_CHANGED): Row id=1 no longer holds the value you approved:\n`postcode` was '00100' when the plan was made and is '90210' now.\nNothing was applied — make a new plan against the current values.\n```\n\nI found this because of how the previous audit failed.\n\nI had run an adversarial review over the codebase — several independent passes by\n\ndimension, each finding verified by separate sceptics whose job was to refute it.\n\nTwenty-one findings survived. Adapters, engine, parser, policy. It felt like a\n\ngood day's work.\n\nThen I asked a different question: not \"what did you find\" but **\"what did this\nreview structurally not look at?\"** The first line of the answer:\n\n`src/apply.ts`\n\n(494 lines at the time) produced zero findings. That should be read as\n\n\"nobody opened it\", not as \"it was clean\".\n\n`apply.ts`\n\nis the only code in the library that writes to production. Dry runs\n\nalways roll back. Approval only writes a record. Committing happens in exactly\n\none place, and the review had not been there.\n\nScoped to that one file, the same process returned **twenty-three** findings —\n\nmore than the first pass found in the whole rest of the codebase. Everything\n\nabove came out of it.\n\nThe number 21 had felt like progress. It was a record of where I had looked.\n\nThis week I wrote worked examples: the four database accounts the design assumes,\n\nwith the exact grants, for MySQL and Postgres. I decided to run every line against\n\na real server rather than write what I knew to be true.\n\nThe server disagreed twice.\n\n**A database-wide grant cannot be narrowed.** I had written the obvious thing —\n\ngrant DML on the whole schema, then take it back on the two tables that hold the\n\napproval records:\n\n```\nERROR 1147 (42000): There is no such grant defined for user 'llm_plan'\n                    on host '%' on table 'llm_safe_sql_plans'\n```\n\nMySQL will not revoke a table-level subset of a database-level grant. So\n\n`GRANT ... ON shop.*`\n\nhands the dry-run account write access to the table that\n\nrecords approvals, permanently. A dry run could forge its own approval. The\n\nexamples name each table instead.\n\n**And check did not check.** The tool has a command whose entire job is to say\n\n`ready`\n\nand exited 0 — and the omission surfaced on the first `plan`\n\n, asThat is the likeliest mistake anyone makes on day one, and it was invisible to the\n\ncommand that exists to catch exactly that class of thing. Fixed in 0.4.2: it\n\nreports the missing table and exits non-zero, and a missing store table is a\n\nrefusal on every path rather than a stack trace.\n\nThere was a third, smaller one. `check`\n\nnow asks the catalogue whether the tables\n\nexist rather than issuing `SELECT 1 FROM audit WHERE 1 = 0`\n\n— because the store\n\naccount the examples recommend holds `INSERT`\n\non the audit table and nothing else,\n\nso the select probe reports the table missing *exactly when the credential is as\nnarrow as it is supposed to be.* Writing the recommended configuration is what\n\nAnd then CI failed, because CI runs the README's own quick start verbatim, and my\n\nfix had made the documented order wrong. The instructions said `check`\n\nthen\n\n`migrate`\n\n; `check`\n\nnow exits non-zero before `migrate`\n\nhas run. Something noticed\n\nbefore a reader did.\n\n**Count what you have not looked at.** A list of findings looks like evidence of\n\nthoroughness and is evidence of coverage. The file that should obviously have been\n\nat the top of a risk-ordered list did not appear on the list at all, and absence\n\nlooked exactly like safety.\n\n**Writing the explanation is a test.** Twice now, the act of documenting this\n\nthing has found defects in it that reading the code did not. Not because prose is\n\nmagic — because writing an example means running it, and running it means the\n\nserver gets a vote. Two privilege lists that were obviously right were wrong.\n\n**Watch for one name doing two jobs.** The bug here was not a missing check. It\n\nwas a display set and a verification set sharing a variable, so narrowing the\n\ndisplay for good reasons silently narrowed the check. If you have something that\n\nboth shows a human what will happen and decides whether it may happen, those are\n\ntwo lists, and they should have two names.\n\n**A guard that only works on one backend is not a guard.** It is a coincidence\n\nwith good timing, and it will be silently absent on the day you switch.\n\nIf you are building anything that measures before it asks a human to agree, the\n\nquestion I would now ask it is: *what is the interface not showing me, and how\nwould I know?* The answers worth having are specific — which columns does it\n\nNone of the five layers people usually reach for would have caught this. Not a\n\nrole without write privileges, not a proxy, not an approval dialog, not a stricter\n\nprompt. Every one of these was a legitimate, authorised `UPDATE`\n\non an allowlisted\n\ntable by a credential entitled to run it. Nothing about *what was permitted* was\n\nviolated. What was wrong was what the human was told before they agreed.\n\nCode, and the full list of what changed in each version, at\n\n[github.com/hyuga611/llm-safe-sql](https://github.com/hyuga611/llm-safe-sql).\n\nThe `examples/`\n\ndirectory is the part I would read first — it is the only\n\ndocumentation I have written where every line was executed against a real server\n\nbefore it was committed.", "url": "https://wpnews.pro/news/the-card-said-one-column-the-apply-wrote-two", "canonical_source": "https://dev.to/hyuga611/the-card-said-one-column-the-apply-wrote-two-2pa0", "published_at": "2026-08-10 09:56:46+00:00", "updated_at": "2026-08-10 10:18:43.535896+00:00", "lang": "en", "topics": ["ai-tools", "ai-safety", "developer-tools"], "entities": ["@hyuga/llm-safe-sql", "MySQL", "PostgreSQL", "SQLite"], "alternates": {"html": "https://wpnews.pro/news/the-card-said-one-column-the-apply-wrote-two", "markdown": "https://wpnews.pro/news/the-card-said-one-column-the-apply-wrote-two.md", "text": "https://wpnews.pro/news/the-card-said-one-column-the-apply-wrote-two.txt", "jsonld": "https://wpnews.pro/news/the-card-said-one-column-the-apply-wrote-two.jsonld"}}