We Built an AI Data Analyst, Then Deleted the AI

A company that built an in-house AI data analyst scrapped the AI layer before shipping and instead exposed its warehouse to Claude through an MCP-over-HTTP server, answering 1,436 questions across 781 tables in nine countries in its first twelve weeks without giving anyone a warehouse login. The server enforces read-only, single-source SELECTs parsed with Postgres's own parser, rewrites every SELECT, CTE and subquery to add the tenant filter, and returns model-readable refusals that tell Claude how to fix a query; 87% of failed queries were followed by a successful one within ten minutes, and Claude searched the attached knowledge bases 454 times. The team said it stopped its original classifier-search-prompt-fixer pipeline because it was "rebuilding Claude, and not as well.

Most companies have the same problem with their data. Someone needs a number, it’s in the warehouse, and only a few people know the warehouse well enough to get it out. So the question becomes a request, and it waits until an analyst or a developer has time. We wanted people to answer those questions themselves, without handing everyone warehouse credentials and without an AI that guesses what our metrics mean. Our first attempt was an AI analyst built into our own app. We stopped before it shipped. What we shipped instead is simpler: Claude does the thinking, and our product gives it safe access to our data and the knowledge of what that data means. Twelve weeks in, it has answered 1,436 questions across 781 tables in nine countries. Teams across the company rely on it, and nobody had to be given a warehouse login. The first version looked like most AI analyst projects: a classifier to spot data questions, a search step to find relevant tables, a long prompt to write the SQL, and a fixer to retry when it failed, plus the usual plumbing for model providers, token budgets and streaming. As it came together, we realised we were rebuilding Claude, and not as well. Claude already plans, recovers from bad queries and keeps track of a conversation, and it gets better at all of that with every release. What it can’t do is see our warehouse, know what an “active listing” means here, or know who is allowed to see which table. So that’s what we built, and we turned our app into something Claude could use. There are three parts now. Claude is the brain: it reads the question, writes the SQL and explains the answer. Our server holds the data and the knowledge. It’s a fairly ordinary web app that speaks MCP over HTTP, so Claude, or any other MCP client, can connect to it. And a skill, a set of written instructions Claude loads for data questions, teaches it how we want those questions handled. The server’s job is to make the right thing easy for Claude and the wrong thing impossible. Claude browses a catalog of schemas, tables and columns, each with a short description written by a person. It’s paged, after our first version dumped more than a thousand schemas into a single response. Claude can run one read-only SELECT at a time, against one data source. The server parses it into a syntax tree with Postgres's own parser rather than pattern-matching the text, checks every table it touches against what the person is allowed to see, and adds the tenant filter by rewriting that tree, in every SELECT, CTE and subquery. Many of our tables hold several markets side by side, so this matters. Claude just says which market it wants. It never writes the filter, so it can't forget it or widen it: -- what Claude wroteSELECT c.city, COUNT FROM shop.orders oJOIN shop.customers c ON c.id = o.customer idGROUP BY c.city; -- what actually runsSELECT c.city, COUNT FROM shop.orders oJOIN shop.customers c ON c.id = o.customer idWHERE o.tenant id = 7 AND c.tenant id = 7GROUP BY c.city; If you build something similar, put that condition in the ON clause on the optional side of a LEFT JOIN, or the join quietly becomes an inner join. Refusals are written for the model, and they say how to fix the query: the aggregate FILTER WHERE … clause is not supported on this warehouse redshift ;rewrite it with CASE inside the aggregate, e.g. SUM CASE WHEN <condition THEN 1 ELSE 0 END Claude reads that and tries again. 87% of failed queries were followed by a successful one within ten minutes. A column called placed at doesn't tell you what counts as a placed order. That knowledge lives in documents, so they sit behind the same connection as searchable knowledge bases. The server passes each search on to wherever that corpus is indexed, so people install one connector instead of one per corpus, and every knowledge base has its own access grant, separate from table access. Claude searched them 454 times in the first twelve weeks instead of guessing. Sign-in is standard OAuth. The first time Claude connects, the server replies with a pointer to its sign-in metadata, Claude registers itself as a client, and the person signs in with their company account. Every request after that carries a token scoped to this server. Permissions are checked on each request rather than baked into the token, so removing someone’s access takes effect straight away. Each tool’s required permission is derived from its name, so a new tool can’t ship without one, and every query is logged. Plenty of answers are worth keeping, so Claude can save them as reports. What gets saved is the queries, not the numbers, so when someone opens the report next week they see next week’s data. They can change the date range and hit refresh too. Claude can build a report in two ways. Most of the time it uses the report tools the server gives it, adding widgets one by one metrics, tables, charts, funnels . We test-run each widget’s query before saving it and send Claude a preview. We also draw every chart ourselves on the server, so all reports look like they came from the same place, and they look the same when exported as a PDF. Sometimes a grid of widgets is the wrong shape, like a written analysis where the numbers sit inside sentences. For that, Claude writes the whole page and saves it on the server along with its queries. We fill in the numbers and run the page in a locked-down sandbox, where it can pull in a charting library but can’t call out to anything. The sharing model is my favourite part. You share a report view-only, and the other person can open and refresh it even if they’ve never had access to those tables. That’s all they get, though. They can’t query that data themselves, and the report can’t show them anything its owner couldn’t see. If the owner loses access, so does the report. Good tools turned out not to be enough. Claude would pick the right table, write perfectly valid SQL, and still hand back the wrong number because it counted test orders, or used UTC when the person meant their own day. Nothing errors when that happens, which is exactly what makes it dangerous. So we wrote a skill. It’s basically a long document in plain English that ships with the connector, and the rule we lean on most is to work out what a metric means before computing it. What counts, for which market, minus what, in which time zone. If Claude can’t answer one of those, it goes and looks it up, or it asks. The skill also tells Claude how to talk. Nobody asking about listings wants to see our table names, so Claude keeps the SQL to itself and just answers, something like “about 12,400 new listings last month, up around 8%, want it by city?” What the skill never does is decide who can see what. People can argue a model out of following instructions, so anything that protects data lives on the server. If someone deleted the skill tomorrow, the answers would get worse, but nothing would leak. In the twelve weeks since launch, people have asked 1,436 questions and run 9,904 successful queries across 781 tables in nine countries. They come from teams all over the company, including shared accounts that whole squads work from. How people use it surprised us. A typical question takes about four queries over three minutes. People rarely ask once and leave. They look at a number, slice it a different way, compare it with last month, and ask a follow-up. That kind of back-and-forth is exactly what our original pipeline would have handled badly, and what Claude handles well. The mistakes that worry us are the quiet ones. A query that runs fine but counts test accounts gives you a clean, confident, wrong number, which is why the definition rules live in the skill. As for what it’s worth, we can only estimate. If each answered question saves an analyst an hour, that’s about 1,436 hours so far, or roughly four full-time analysts at the current pace. Even at half that, it’s a lot of time going back into building instead of pulling numbers. Moving the thinking into the client moved the hard work rather than removing it. OAuth became part of our product, and at one point open registration left us with six identical client apps, all called “Claude”. The protocol and its SDKs kept changing underneath us. And the list of tools became a contract, because clients cache it and the model learns it, so we now test every tool’s name and arguments against the live server. We also can’t tune the model. What we can shape is the skill, the tool descriptions and the error messages, and so far that has been enough. The turning point for us was one question: if the next model release makes this code look worse, why are we writing it? Most of our first version failed that test. The catalog, the permissions, the knowledge bases and the reports didn’t, because no model brings those with it. Claude brings the reasoning, and we bring the data, what it means, and the rules for using it. We Built an AI Data Analyst, Then Deleted the AI https://pub.towardsai.net/we-built-an-ai-data-analyst-then-deleted-the-ai-f2946bc05db7 was originally published in Towards AI https://pub.towardsai.net on Medium, where people are continuing the conversation by highlighting and responding to this story.