How we let an AI read production databases without trusting it

The developer behind TraceX, an AI-powered production-data investigation tool, built a safety architecture that enforces protections in code rather than prompts: connections are rejected unless the database account is verified read-only, the model emits structured lookup requests that are validated against the introspected schema and compiled into parameterized statements, and every query runs in a read-only transaction with a timeout and row cap. The system also uses one connection per investigation, warns when connected to a primary rather than a replica, masks secret-looking fields before prompting, and computes confidence scores from grounded findings rather than asking the model. The developer acknowledged remaining weaknesses, including that read records are still sent to the model provider.

Every engineering team knows this ticket: "I was charged twice." Someone stops what they're doing, queries the database, greps the code, reads recent deploys and checks the logs. Eventually they explain the answer to support, who explain it to the customer. I'm building TraceX https://tracexpro.com , which does that investigation. To do it, it has to read production data, and the obvious reaction to "an AI reads your production database" is "absolutely not". That reaction is reasonable, so most of our design work has gone into one question: How do you make this safe without relying on the model to behave? Here's what we ended up with, including what's still weak. The rule behind all of it: anything that matters is enforced in code, not requested in a prompt. Asking people to "please use a read-only user" isn't a control. So TraceX checks the account's privileges when you connect, and refuses it if it can write: SELECT EXISTS SELECT 1 FROM information schema.tables WHERE table schema NOT IN 'pg catalog', 'information schema' AND has table privilege format '%I.%I', table schema, table name , 'INSERT, UPDATE, DELETE, TRUNCATE, REFERENCES, TRIGGER' UNION ALL SELECT 1 FROM information schema.schemata WHERE schema name NOT IN 'pg catalog', 'information schema' AND has schema privilege schema name, 'CREATE' UNION ALL SELECT 1 WHERE has database privilege current database , 'CREATE' ; If that returns true, the connection is rejected with a message telling you to use a read-only account. This is the decision I'd make again first. Giving a model a SQL tool means your safety depends on parsing whatever it generates. Instead, the model can only ask for a structured lookup: a table, some filters, some columns and a limit. That request is checked against the introspected schema, then compiled into a parameterized statement. An unknown table or column fails validation. There's no string for the model to inject into, because it never produces a query string. You lose flexibility. It can't write a clever five-way join. For "show me the records for this order", which is most of what an investigation needs, that's a trade I'm happy with. Every query runs inside a read-only transaction, with a timeout and a row cap: async with conn.transaction readonly=True : await conn.execute "SET LOCAL statement timeout = 3000" rows = await conn.fetch sql, params at most 20 rows So the database itself refuses writes, even if the checks above had a bug. A commenter on our launch pointed out the next problem: read-only doesn't help if your connection pool is already saturated during an incident. They were right. So TraceX uses one connection per investigation, closes it afterwards, and tells you whether you've connected a replica or a primary: SELECT pg is in recovery ; If it's a primary, you still can connect, but you get a warning suggesting a read replica. You can also add your own ceiling: ALTER ROLE tracex CONNECTION LIMIT 2; Secret-looking fields and values are masked before a prompt is built. But the records TraceX reads do go to the model provider, because that's how it reasons about them. I'd rather say that plainly than imply otherwise. You can limit what it sees by restricting the database user to specific tables, and you can bring your own API key. Ask a model how confident it is and you'll get a confident answer. So the confidence score is computed from the findings, not taken from the model: python def overall confidence findings, supporting ids, contradicting ids= : grounded = f for f in supporting findings, supporting ids if is grounded f if not grounded: return min best confidence, 0.3 no evidence, so it can't sound sure value = max confidence f for f in grounded value += 0.05 independent sources grounded - 1 logs, database and code agree value -= 0.15 len grounded findings against contradicting ids return min value, 0.95 A finding is "grounded" only if it cites real evidence: a record, a code location or a log line. A cause with no evidence is capped at 30%, however persuasive it sounds. The score never reaches 100%. This came directly from another comment we got: A root-cause answer I can't check is just another confident guess, and the ruled-out hypotheses are the bit I'd actually want in front of a customer. When we looked, we found a gap: our dashboard had a "Ruled out" section, but real investigations never filled it in. So now they do, with one strict rule: an explanation is only listed as ruled out if it cites a real finding that rules it out. If the model can't back it up, it's dropped, so "ruled out" can't become another unchecked claim. The same comment asked what happens when evidence points two ways. If a competing cause is supported within 0.15 of the leading one, TraceX shows both side by side, caps confidence at 60%, and doesn't draft a fix. A close call should go to a human. TraceX can write a fix, but it never opens a pull request on its own. A person clicks, and nothing is merged without your team. Risky fixes have stricter rules. A change is high risk if it touches migrations, the schema, auth, payments or config, or runs over 80 lines. We also check the content of the change, so a rename hidden in a file with an innocent name still counts: DDL = re.compile r"\b ALTER|DROP|CREATE|TRUNCATE|RENAME \s+ TABLE|INDEX|COLUMN|TYPE|SCHEMA|CONSTRAINT \b" r"|\bop\. add column|drop column|alter column|create table|drop table|rename table \b", re.I, That's shortened; the real pattern covers Django and Rails migrations too. High-risk fixes open only as draft pull requests, only an admin can open them, and the PR explains why it's flagged. TraceX is in private beta. There's a sandbox https://app.tracexpro.com/sandbox with made-up data and no signup. Open "Refunds stuck in pending since Tuesday's release" and look at the Resolution tab to see the ruled-out explanations. I'd really like to hear from you: what would you need to see before letting a tool like this read your production systems? And which of these guardrails do you think is weakest?