{"slug": "build-an-ai-data-analyst-that-thinks-like-a-senior-analyst", "title": "Build an AI Data Analyst That Thinks Like a Senior Analyst", "summary": "A Python toolkit that pushes a data question through six stages—business understanding, hypothesis generation, SQL planning, validation, executive summary, and recommendations—aims to make AI data analysts check their numbers before answering, according to a walkthrough using a 29-row online_orders.csv dataset and DuckDB for deterministic sanity checks. The toolkit works with Anthropic or OpenAI APIs and is designed to prevent confident but unvalidated answers from chatbots.", "body_md": "# Build an AI Data Analyst That Thinks Like a Senior Analyst\n\nA six-stage pipeline that checks its numbers before calling anything an answer.\n\nAsk a **chatbot** \"which promotion should we run more of,\" and it answers in one breath. It picks a number, states it with confidence, and stops. It picks the promotion with the best-looking number and states its choice confidently. But it may never check how much data that number is based on. A promotion that looks great after 10 orders is much less convincing than one that performs well across 1,000 orders.\n\nA **senior analyst** works slower on purpose. They restate the question, form a hypothesis, write the query, then check whether the result has enough data behind it before they say anything to an executive.\n\nWe can build that discipline into code.\n\nIn this walkthrough, we build a small **Python toolkit** that pushes a question through **six stages** instead of one prompt: business understanding, hypothesis generation, SQL planning, validation, an executive summary, and recommendations.\n\nThe toolkit works with either the Anthropic or the OpenAI API, so you bring your own key. Point it at any table, and it runs the same six stages.\n\nAll the code below runs in order, from loading the CSV to the final recommendation, so you can follow along in a notebook against your own data.\n\n## The Data\n\nIn this article, we are going to use a data table called `online_orders.csv`. You can check out this dataset in this **[StrataScratch interview question](https://platform.stratascratch.com/coding/2157-10-monthly-sales-increase?code_type=2&utm_source=blog&utm_medium=click&utm_campaign=kdn+ai+senior+data+analyst)**. It contains 29 rows of order-level data: which product sold, which promotion applied, the per-unit cost, the customer, the date, and the units sold.\n\n| product_id | promotion_id | cost_in_dollars | customer_id | date_sold | units_sold | \n|---|---|---|---|---|---|\n| 1 | 1 | 2 | 1 | 2022-04-01 | 4 | \n| 3 | 3 | 6 | 3 | 2022-05-24 | 6 | \n| 1 | 2 | 2 | 10 | 2022-05-01 | 3 | \n| 1 | 2 | 3 | 2 | 2022-05-01 | 9 | \n| … | … | … | … | … | … | \n| 5 | 2 | 8 | 15 | 2022-05-01 | 2 | \n\nFirst, we load it with **[Pandas](https://pandas.pydata.org/)**:\n\n``` python\nimport pandas as pd\nfrom IPython.display import display\norders = pd.read_csv(\"online_orders.csv\")\nprint(f\"Loaded {len(orders):,} rows and {len(orders.columns)} columns.\")\ndisplay(orders.head())\n```\n\n#### Output\n\n```\nLoaded 29 rows and 6 columns.\n```\n\n29 orders across 3 months, 4 promotions, and 11 products. That is small enough that every group in a `groupby` matters, which is exactly the kind of dataset a fast answer gets wrong.\n\n#### Inspecting the Schema\n\nBefore touching any large language model (LLM), we look at what is actually in the table:\n\n```\nschema_preview = pd.DataFrame({\n    \"column\": orders.columns,\n    \"dtype\": orders.dtypes.astype(str).values,\n    \"missing_values\": orders.isna().sum().values,\n})\ndisplay(schema_preview)\n```\n\n#### Output\n\n| column | dtype | missing_values | \n|---|---|---|\n| product_id | int64 | 0 | \n| promotion_id | int64 | 0 | \n| cost_in_dollars | int64 | 0 | \n| customer_id | int64 | 0 | \n| date_sold | object | 0 | \n| units_sold | int64 | 0 | \n\nNo missing values, and `date_sold` is stored as text rather than a real date.\n\n## A Deterministic Sanity Check\n\nBefore we call any LLM, plain SQL already tells us something. We register the dataframe with **[DuckDB](https://duckdb.org/)**, which lets us run real SQL against it with no database server to set up.\n\n``` python\nimport duckdb\ncon = duckdb.connect()\ncon.register(\"online_orders\", orders)\npreview = con.execute(\"\"\"\n    SELECT\n        promotion_id,\n        COUNT(*) AS n_orders,\n        SUM(units_sold) AS total_units,\n        SUM(cost_in_dollars * units_sold) AS total_revenue,\n        ROUND(AVG(units_sold), 2) AS avg_units_per_order\n    FROM online_orders\n    GROUP BY promotion_id\n    ORDER BY avg_units_per_order DESC\n\"\"\").df()\ndisplay(preview)\n```\n\n#### Output\n\n| promotion_id | n_orders | total_units | total_revenue | avg_units_per_order | \n|---|---|---|---|---|\n| 4 | 1 | 8.0 | 64.0 | 8.00 | \n| 1 | 12 | 77.0 | 407.0 | 6.42 | \n| 2 | 10 | 55.0 | 199.0 | 5.50 | \n| 3 | 6 | 31.0 | 185.0 | 5.17 | \n\nSorted by average units per order, promotion 4 comes out on top at 8.00.\n\nIt also has exactly 1 order behind it. A \"which promotion has the best average\" answer, asked and answered in one breath, would recommend promotion 4 on the strength of a single order. That is the trap the rest of this pipeline is built to catch.\n\n## The LLM Wrapper\n\nThe pipeline should not care whether you hand it an Anthropic client or an OpenAI client. A thin wrapper takes the provider explicitly and calls the matching method. For Anthropic, a reply can come back as more than one content block, so it scans them for the first block of type `text` instead of assuming it comes first.\n\n``` python\nclass LLMClient:\n    def __init__(self, client, model, provider):\n        self.client = client\n        self.model = model\n        self.provider = provider\n\n    def complete(self, prompt):\n        if self.provider == \"anthropic\":\n            response = self.client.messages.create(\n                model=self.model,\n                max_tokens=1024,\n                messages=[{\"role\": \"user\", \"content\": prompt}],\n            )\n\n            for block in response.content:\n                if block.type == \"text\":\n                    return block.text\n\n            raise ValueError(\"No text block found in Claude's response.\")\n\n        if self.provider == \"openai\":\n            response = self.client.chat.completions.create(\n                model=self.model,\n                messages=[{\"role\": \"user\", \"content\": prompt}],\n            )\n            return response.choices[0].message.content\n\n        raise ValueError(f\"Unsupported provider: {self.provider}\")\n```\n\nThis gives the rest of the pipeline a single `complete()` method to work with. The provider-specific response formats stay hidden inside the wrapper, so later stages do not need separate Anthropic and OpenAI code paths. If a provider is unsupported, or Claude returns no usable text block, the wrapper fails explicitly instead of silently passing an invalid response downstream.\n\nEvery stage below asks the model to return JSON, so we need one more helper to pull that JSON out of a text reply. Some replies come back wrapped in a triple-backtick code fence, so the helper strips that first, then falls back to scanning the text for the first valid JSON object or array.\n\n``` python\nimport json\nimport re\ndef parse_json(text):\n    text = text.strip()\n\n    if text.startswith(\"```\"):\n        text = re.sub(r\"^```(?:json)?\\s*\", \"\", text, flags=re.IGNORECASE)\n        text = re.sub(r\"\\s*```$\", \"\", text)\n\n    try:\n        return json.loads(text)\n    except json.JSONDecodeError:\n        pass\n    candidates = []\n    object_match = re.search(r\"\\{.*\\}\", text, re.DOTALL)\n    array_match = re.search(r\"\\[.*\\]\", text, re.DOTALL)\n    if object_match:\n        candidates.append(object_match)\n    if array_match:\n        candidates.append(array_match)\n    candidates.sort(key=lambda match: match.start())\n    for match in candidates:\n        try:\n            return json.loads(match.group(0))\n        except json.JSONDecodeError:\n            continue\n    raise ValueError(f\"No valid JSON found in model output:\\n{text}\")\n```\n\nThe **parser** starts with the simplest case: if the entire reply is valid JSON, it returns it immediately. If that fails, it looks for an object or array embedded in surrounding prose and tries the candidates in the order they appear. This makes the pipeline a little more tolerant of common model formatting mistakes while still raising an error when there is no valid JSON to work with.\n\n## Stage 1: Business Understanding\n\nThe first stage restates the question in terms the table can actually answer, names the grain of the data, and lists limitations before any analysis starts.\n\n```\nclass SeniorAnalyst:\n    MIN_SUPPORT = 3  # minimum orders behind a group before we trust it\n\n    def __init__(self, llm, table_name, dataframe):\n        self.llm = llm\n        self.table_name = table_name\n        self.con = duckdb.connect()\n        self.con.register(table_name, dataframe)\n        self.schema = self.con.execute(f\"DESCRIBE {table_name}\").df()\n\n    def understand_business_context(self, question):\n        row_count = self.con.execute(\n            f\"SELECT COUNT(*) FROM {self.table_name}\"\n        ).fetchone()[0]\n\n        columns = self.schema[\n            [\"column_name\", \"column_type\"]\n        ].to_dict(\"records\")\n        prompt = f\"\"\"You are a senior data analyst. A stakeholder asked: \"{question}\"\nTable: {self.table_name}\nColumns: {columns}\nRow count: {row_count}\n\nRestate the stakeholder question in terms this table can actually answer.\n\nAlso name the grain of the table (what one row represents), and list any\nlimitations you can already see: sample size, date coverage, missing\ndimensions, missing context.\n\nReturn JSON only: {{\"restated_question\": \"...\", \"grain\": \"...\",\n\"limitations\": [\"...\", \"...\"]}}\"\"\"\n        context = parse_json(self.llm.complete(prompt))\n        self.context = context\n        return context\n```\n\nWe ran this with claude-sonnet-5 on the question \"which promotion should we run more of.\" Here is what came back.\n\n#### Output\n\nIt flagged the small sample size before running a single query — the same trap the plain SQL `groupby` above already showed us. That flag is a hint, not a check. The pipeline still needs to enforce it in code, which is what the validation stage below does.\n\n## Stage 2: Hypothesis Generation\n\nThe second stage proposes specific, testable hypotheses using only the columns that exist in the table.\n\n``` python\ndef generate_hypotheses(self, n=2):\n    columns = list(self.schema[\"column_name\"])\n    prompt = f\"\"\"Business context: {self.context}\nPropose {n} specific, testable hypotheses that would help answer the\nrestated question, using only columns in: {columns}.\nEach hypothesis should be something we can test using SQL.\nReturn JSON only: [{{\"hypothesis\": \"...\", \"why\": \"...\"}}, ...]\"\"\"\n    hypotheses = parse_json(self.llm.complete(prompt))\n    self.hypotheses = hypotheses\n    return hypotheses\n```\n\n#### Output\n\nThe pipeline tests the first hypothesis. Notice it is not a raw average: it asks whether the volume leader beats the runner-up by a real margin, which already reads differently from the \"highest average\" query above that put a 1-order promotion on top.\n\n## Stage 3: SQL Planning\n\nThe third stage turns the top hypothesis into an actual query. We ask for a row count alongside any grouped metric, since a group's size is what the validation stage checks next.\n\n``` python\ndef plan_sql(self, hypothesis):\n    columns = list(self.schema[\"column_name\"])\n    prompt = f\"\"\"Table: {self.table_name}\nColumns: {columns}\nHypothesis to test: {hypothesis['hypothesis']}\nWrite one DuckDB SQL query that tests this hypothesis.\nUse only the available columns, do not invent columns, and if the query\ngroups rows, include a COUNT(*) column named n_orders so the result can\nbe checked for sample size before anyone trusts it.\nReturn JSON only: {{\"sql\": \"...\", \"purpose\": \"...\"}}\"\"\"\n    plan = parse_json(self.llm.complete(prompt))\n    return plan\n```\n\n#### Output\n\n```\nGenerated SQL:\n    WITH promo_sums AS (\n        SELECT promotion_id, SUM(units_sold) AS total_units, COUNT(*) AS n_orders\n        FROM online_orders\n        GROUP BY promotion_id\n    ),\n    ranked AS (\n        SELECT promotion_id, total_units, n_orders,\n               RANK() OVER (ORDER BY total_units DESC) AS rnk\n        FROM promo_sums\n    )\n    SELECT\n        r1.promotion_id AS top_promotion_id,\n        r1.total_units AS top_total_units,\n        r1.n_orders AS top_n_orders,\n        r2.promotion_id AS second_promotion_id,\n        r2.total_units AS second_total_units,\n        r2.n_orders AS second_n_orders,\n        (r1.total_units - r2.total_units) * 1.0 / r2.total_units AS pct_difference\n    FROM ranked r1\n    JOIN ranked r2 ON r2.rnk = 2\n    WHERE r1.rnk = 1\n\n   'purpose': 'Identify the promotion_id with the highest total units\n     sold and compare it to the second-highest to test whether it exceeds\n     it by at least 20%, including order counts to assess statistical\n     support.'\n```\n\nRather than a simple `groupby`, the model reached for a common table expression (CTE) with a window function, ranking promotions by total units and pulling the top two into the same row for comparison.\n\n## Stage 4: Validation\n\nThe fourth stage runs the query and checks `n_orders` against a minimum support threshold. This is the one stage that is plain code, not a model call, because the check has to be enforced, not suggested.\n\n``` python\ndef validate(self, sql_plan):\n    result = self.con.execute(sql_plan[\"sql\"]).df()\n    if \"n_orders\" in result.columns:\n        result[\"low_confidence\"] = result[\"n_orders\"] < self.MIN_SUPPORT\n    else:\n        result[\"low_confidence\"] = False\n    return result\n```\n\n#### Output\n\n| top_promotion_id | top_total_units | top_n_orders | second_promotion_id | second_total_units | second_n_orders | pct_difference | low_confidence | \n|---|---|---|---|---|---|---|---|\n| 1 | 77.0 | 12 | 2 | 55.0 | 10 | 0.4 | False | \n\nThis query only produces one row, and it is not flagged. Promotion 1 leads on total units with 12 orders behind it, promotion 2 is the runner-up with 10, and both clear the minimum of 3 we set. The check still ran here — it just had nothing to catch, because this hypothesis compares two well-supported groups instead of resting on promotion 4's single order.\n\n## Stage 5: Executive Summary\n\nThe fifth stage writes the summary, and it is told explicitly to leave any flagged row out of the headline claim.\n\n``` python\n    def summarize(self, hypothesis, validated_result):\n        flagged = validated_result[validated_result[\"low_confidence\"]]\n        prompt = f\"\"\"Hypothesis: {hypothesis['hypothesis']}\n    Query result:\n    {validated_result.to_string(index=False)}\n    Rows marked low_confidence have fewer than {self.MIN_SUPPORT} orders\n    behind them and should not anchor a conclusion.\n    Low-confidence rows: {flagged.to_dict('records')}\n    Write a concise 3 to 4 sentence executive summary of what this result\n    supports. Base the conclusion only on the data shown, explicitly avoid\n    using low-confidence rows as the headline, and do not invent\n    explanations that are not supported by the data.\"\"\"\n        return self.llm.complete(prompt)\n```\n\n#### Output\n\n## Stage 6: Recommendations\n\nThe sixth stage proposes actions, and it is told the same rule applies: no recommendation may rest on low-confidence data or facts the summary did not support.\n\n``` python\n    def recommend(self, summary):\n        prompt = f\"\"\"Executive summary: {summary}\n    Propose 2 to 3 specific business recommendations based only on what the\n    summary supports. Recommendations must follow from the evidence, must\n    not rest on low-confidence data or invented facts, and if the evidence\n    is weak, should recommend further analysis instead of pretending the\n    answer is certain.\"\"\"\n        return self.llm.complete(prompt)\n```\n\n#### Output\n\n## Putting It Together\n\nA `run` method chains the six stages. One call takes a question in and returns every intermediate result: the context, the hypotheses, the SQL plan, the validated table, the summary, and the recommendation.\n\n``` python\n    def run(self, question):\n        context = self.understand_business_context(question)\n        hypotheses = self.generate_hypotheses()\n        top_hypothesis = hypotheses[0]\n        plan = self.plan_sql(top_hypothesis)\n        validated = self.validate(plan)\n        summary = self.summarize(top_hypothesis, validated)\n        recommendation = self.recommend(summary)\n        return {\n            \"context\": context,\n            \"hypotheses\": hypotheses,\n            \"sql_plan\": plan,\n            \"validated_result\": validated,\n            \"summary\": summary,\n            \"recommendation\": recommendation,\n        }\n```\n\n## Calling It\n\nCalling it looks the same regardless of which provider you bring. The provider is set explicitly rather than guessed from the client object, and the pipeline refuses to run if you forget to paste in a real key.\n\n```\n    PROVIDER = \"anthropic\"\n    API_KEY = \"YOUR_API_KEY_HERE\"\n    ANTHROPIC_MODEL = \"claude-sonnet-5\"\n    OPENAI_MODEL = \"gpt-4o\"\n    if API_KEY == \"YOUR_API_KEY_HERE\":\n        raise ValueError(\n            \"Paste your real API key into API_KEY before running the LLM section.\"\n        )\n    if PROVIDER.lower() == \"anthropic\":\n        from anthropic import Anthropic\n        client = Anthropic(api_key=API_KEY)\n        llm = LLMClient(client=client, model=ANTHROPIC_MODEL, provider=\"anthropic\")\n    elif PROVIDER.lower() == \"openai\":\n        from openai import OpenAI\n        client = OpenAI(api_key=API_KEY)\n        llm = LLMClient(client=client, model=OPENAI_MODEL, provider=\"openai\")\n    else:\n        raise ValueError(\"PROVIDER must be either 'openai' or 'anthropic'.\")\n    analyst = SeniorAnalyst(llm, \"online_orders\", orders)\n    result = analyst.run(\"Which promotion should we run more of?\")\n    print(result[\"summary\"])\n    print(result[\"recommendation\"])\n```\n\nSet `PROVIDER` to `openai` instead, drop in an OpenAI key, and the same six stages run against `gpt-4o` unchanged. `LLMClient` is the only piece that knows which API it is talking to.\n\n## Conclusion\n\nNone of the six stages here is complicated on its own. Restating a question, writing SQL, and summarizing a table are things a single prompt already does reasonably well. The value comes from the validation stage between the query and the summary — checking `n_orders` before anything gets called an answer.\n\nOn this dataset, that check already caught something before the LLM was even called: the plain SQL `groupby` above ranked promotion 4 first by average units per order, resting on exactly 1 order. The hypothesis the model chose to test this run compared two well-supported groups instead — 12 orders against 10 — so `validate()` had nothing to flag. The pipeline runs the same `n_orders` check regardless of which comparison the model hands it, so a future table, or a future run that tests an average instead of a total, gets caught by the same line of code.\n\nThis pipeline has 6 methods on one class, and the same 6 run again on the next table you point it at.\n\n \n\n \n\n[**\\[Nate Rosidi\\](https://twitter.com/StrataScratch)**](https://twitter.com/StrataScratch) is a data scientist and in product strategy. He's also an adjunct professor teaching analytics, and is the founder of StrataScratch, a platform helping data scientists prepare for their interviews with real interview questions from top companies. Nate writes on the latest trends in the career market, gives interview advice, shares data science projects, and covers everything SQL.", "url": "https://wpnews.pro/news/build-an-ai-data-analyst-that-thinks-like-a-senior-analyst", "canonical_source": "https://www.kdnuggets.com/build-an-ai-data-analyst-that-thinks-like-a-senior-analyst", "published_at": "2026-09-09 14:00:50+00:00", "updated_at": "2026-09-09 14:43:09.607555+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-tools", "ai-agents"], "entities": ["Python", "Pandas", "DuckDB", "Anthropic", "OpenAI", "StrataScratch"], "alternates": {"html": "https://wpnews.pro/news/build-an-ai-data-analyst-that-thinks-like-a-senior-analyst", "markdown": "https://wpnews.pro/news/build-an-ai-data-analyst-that-thinks-like-a-senior-analyst.md", "text": "https://wpnews.pro/news/build-an-ai-data-analyst-that-thinks-like-a-senior-analyst.txt", "jsonld": "https://wpnews.pro/news/build-an-ai-data-analyst-that-thinks-like-a-senior-analyst.jsonld"}}