SQL vs Pandas vs AI Agents: Which Solves Analytics Problems Best?

A comparison of SQL, Pandas, and AI agents on three analytics interview questions found SQL fastest and most reliable, while AI agents offered natural language convenience but higher latency and hallucination risk. The study measured execution times over 500 runs across eight dimensions, revealing that for simple queries all tools agreed, but for complex multi-step aggregation the agent's schema grounding was critical.

SQL vs Pandas vs AI Agents: Which Solves Analytics Problems Best? Same three analytics problems, three tools, eight dimensions, measured with real execution times and real agent prompts. Introduction We gave the same three interview questions from StrataScratch to SQL, , and a Pandas https://pandas.pydata.org/ agent. Every piece of code executed against the same dataset, and every timing number is a median over 500 runs. The agent's answers are exactly what Claude generated in response to a documented prompt, instead of a hypothetical example of what an agent might produce. Claude https://claude.ai/ The comparison runs across eight dimensions: speed, accuracy, explainability, debugging, scalability, flexibility, hallucination risk, and production readiness. The three questions span Easy, Medium, and Hard difficulty levels. The harder the question, the more the differences between SQL, Pandas, and the agent become visible. How We Ran This Comparison The three questions come from the StrataScratch interview bank and cover Easy, Medium, and Hard difficulty levels. SQL ran on SQLite in-memory, timed over 500 runs, with the median taken. Pandas ran on the same dataset in 3.12, also over 500 runs. The agent is Claude's Python https://www.python.org/ , called via the claude-sonnet-4-6 https://www.anthropic.com/news/claude-sonnet-4-6 . Anthropic API https://www.anthropic.com/api Each question got its own schema-grounded user prompt that included the table names, column names, and a few sample rows. The system prompt below stayed the same for all three calls. Agent response times are measured from the time the request is sent to the first token received. Simple Retrieval: All Three Agree For the first interview question from Meta https://platform.stratascratch.com/coding/9787-user-scroll-up-events?code type=1&utm source=blog&utm medium=click&utm campaign=kdn+sql+vs+pandas+vs+ai+agents , users are asked to find every user who performed at least one scroll up event and return the distinct user IDs. The data lives in a single table called facebook web log . // Data Here's the facebook web log table. | user id | timestamp | action | |---|---|---| | 0 | 2019-04-25 13:30:15 | page load | | 0 | 2019-04-25 13:30:18 | page load | | 0 | 2019-04-25 13:30:40 | scroll down | | 0 | 2019-04-25 13:30:45 | scroll up | | … | … | … | | 0 | 2019-04-25 13:30:40 | page exit | // SQL Coding Solution 0.002 ms SELECT DISTINCT user id FROM facebook web log WHERE action = 'scroll up'; // Pandas Coding Solution 0.40 ms python import pandas as pd result = facebook web log facebook web log 'action' == 'scroll up' .drop duplicates subset='user id' 'user id' // Agent Prompt Table: facebook web log user id INTEGER, action TEXT, timestamp TEXT Sample rows: 1, 'scroll up', '2019-01-01 00:00:00' 2, 'scroll down', '2019-01-01 00:01:00' 3, 'like', '2019-01-01 00:03:00' 2, 'scroll up', '2019-01-01 00:04:00' Question: Find all users who performed at least one scroll up event. Return distinct user IDs. // Agent Output 2 s SELECT DISTINCT user id FROM facebook web log WHERE action = 'scroll up'; Output: All three return users 1 and 2. | user id | |---| | 0 | | 2 | | 1 | On a single-filter problem, the agent matches SQL exactly. The only real risk at this difficulty is column naming. Without the schema in the prompt, action might come back as event type or event name , which returns nothing and throws no error. Multi-Step Aggregation: Where Schema Grounding Matters Most The second question https://platform.stratascratch.com/coding/9792-user-feature-completion?code type=1&utm source=blog&utm medium=click&utm campaign=kdn+sql+vs+pandas+vs+ai+agents is about product feature completion. An app tracks how far each user gets through a set of product features, where every feature has a fixed number of steps. The task is to calculate the average completion percentage for each feature across all users, where a user's completion is their maximum step reached divided by the total steps for that feature, times 100. Users who have never started a feature are counted as 0% complete. Two tables feed this: facebook product features : | feature id | n steps | |---|---| | 0 | 5 | | 1 | 7 | | 2 | 3 | and facebook product features realizations . | feature id | user id | step reached | timestamp | |---|---|---|---| | 0 | 0 | 1 | 2019-03-11 17:15:00 | | 0 | 0 | 2 | 2019-03-11 17:22:00 | | 0 | 0 | 3 | 2019-03-11 17:25:00 | | 0 | 0 | 4 | 2019-03-11 17:27:00 | | ... | ... | ... | ... | | 1 | 1 | 3 | 2019-04-05 13:00:07 | // SQL Coding Solution 0.007 ms WITH max step AS SELECT feature id, user id, MAX step reached AS max step reached FROM facebook product features realizations GROUP BY feature id, user id , calc per feature AS SELECT feats.feature id, n steps, max step reached, COALESCE max step reached, 0 1.0 / n steps AS share of completion FROM facebook product features feats LEFT OUTER JOIN max step ON feats.feature id = max step.feature id SELECT feature id, AVG share of completion 100 AS avg share of completion FROM calc per feature GROUP BY feature id; // Pandas Coding Solution 2.05 ms python import pandas as pd max step per user per feature max step = facebook product features realizations .groupby 'feature id', 'user id' 'step reached' .max .reset index join to features, fill users who never started with 0 df = pd.merge facebook product features, max step, how='outer', on='feature id' .fillna 0 completion % per user-feature row df 'share of completion' = df 'step reached' / df 'n steps' 100 average per feature result = df.groupby 'feature id' 'share of completion' .mean .to frame 'avg share of completion' .reset index // Agent Prompt Tables: - facebook product features feature id INTEGER, n steps INTEGER - facebook product features realizations feature id INTEGER, user id INTEGER, step reached INTEGER, timestamp TEXT Question: For each feature, calculate the average completion percentage across all users. A user's completion = max step reached / n steps 100. Users who never started count as 0% completion. Return: feature id, avg share of completion. // Agent Output 3 s WITH user max steps AS SELECT feature id, user id, MAX step reached AS max step FROM facebook product features realizations GROUP BY feature id, user id SELECT f.feature id, ROUND AVG COALESCE ums.max step, 0 100.0 / f.n steps , 2 AS avg share of completion FROM facebook product features f LEFT JOIN user max steps ums ON f.feature id = ums.feature id GROUP BY f.feature id ORDER BY f.feature id; Output: All three return the same numbers. | feature id | avg share of completion | |---|---| | 0 | 80 | | 2 | 0 | | 1 | 76.19 | The agent got it right here because the prompt said: "Users who never started count as 0% completion." That phrase is load-bearing. Without it, the agent writes an inner join — which drops non-starters — and every average goes up. That failure is silent. The numbers come back clean, and they're wrong. You'd need to know the expected output to catch it. Multiple Tables and Window Logic: All Three Correct, One Much Slower The third question https://platform.stratascratch.com/coding/10084-cum-sum-energy-consumption?utm source=blog&utm medium=click&utm campaign=kdn+sql+vs+pandas+vs+ai+agents covers Meta's data center energy consumption across three regions. Each region has its own table: fb eu energy , fb na energy , and fb asia energy . The task is to combine them, sum consumption by date, and produce two derived columns: the cumulative running total and that total as a percentage of the grand total, rounded to a whole number. // Data Each regional table has the same shape. fb eu energy : | recorded date | consumption | |---|---| | 2020-01-01 | 400 | | 2020-01-02 | 350 | | 2020-01-03 | 500 | | 2020-01-04 | 500 | | 2020-01-07 | 600 | fb na energy : | recorded date | consumption | |---|---| | 2020-01-01 | 250 | | 2020-01-02 | 375 | | 2020-01-03 | 600 | | 2020-01-06 | 500 | | 2020-01-07 | 250 | fb asia energy : | recorded date | consumption | |---|---| | 2020-01-01 | 400 | | 2020-01-02 | 400 | | 2020-01-04 | 675 | | 2020-01-05 | 1200 | | 2020-01-06 | 750 | | 2020-01-07 | 400 | // SQL Coding Solution 0.010 ms WITH total energy AS SELECT recorded date, consumption FROM fb eu energy UNION ALL SELECT recorded date, consumption FROM fb asia energy UNION ALL SELECT recorded date, consumption FROM fb na energy , energy by date AS SELECT recorded date, SUM consumption AS total energy FROM total energy GROUP BY recorded date ORDER BY recorded date ASC SELECT recorded date, SUM total energy OVER ORDER BY recorded date ASC AS cumulative total energy, ROUND SUM total energy OVER ORDER BY recorded date ASC 100.0 / SELECT SUM total energy FROM energy by date , 0 AS percentage of total energy FROM energy by date; // Pandas Coding Solution 1.84 ms python import pandas as pd merged df = pd.concat fb eu energy, fb asia energy, fb na energy energy by date = merged df.groupby 'recorded date', as index=False 'consumption' .sum .sort values 'recorded date' energy by date 'cumulative total energy' = energy by date 'consumption' .cumsum energy by date 'percentage of total energy' = round energy by date 'cumulative total energy' 100 / energy by date 'consumption' .sum energy by date 'recorded date' = pd.to datetime energy by date 'recorded date' .dt.strftime '%Y-%m-%d' result = energy by date 'recorded date', 'cumulative total energy', 'percentage of total energy' // Agent Prompt Tables: - fb eu energy recorded date TEXT, consumption INTEGER - fb na energy recorded date TEXT, consumption INTEGER - fb asia energy recorded date TEXT, consumption INTEGER Question: Combine all three tables. Sum consumption by date. Output: - recorded date - cumulative total energy running sum across dates, ordered by date - percentage of total energy running cumulative / grand total 100, rounded to the nearest whole number // Agent Output 4 s WITH all energy AS SELECT recorded date, consumption FROM fb eu energy UNION ALL SELECT recorded date, consumption FROM fb na energy UNION ALL SELECT recorded date, consumption FROM fb asia energy , daily totals AS SELECT recorded date, SUM consumption AS daily total FROM all energy GROUP BY recorded date SELECT recorded date, SUM daily total OVER ORDER BY recorded date AS cumulative total energy, ROUND SUM daily total OVER ORDER BY recorded date 100.0 / SUM daily total OVER , 0 AS percentage of total energy FROM daily totals ORDER BY recorded date; Output: All three return the same table. | recorded date | cumulative total energy | percentage of total energy | |---|---|---| | 2020-01-01 | 1050 | 13 | | 2020-01-02 | 2175 | 27 | | 2020-01-03 | 3275 | 40 | | 2020-01-04 | 4450 | 55 | | 2020-01-05 | 5650 | 69 | | 2020-01-06 | 6900 | 85 | | 2020-01-07 | 8150 | 100 | The agent used SUM daily total OVER a window function with no ORDER BY as the denominator rather than the scalar subquery in the SQL reference solution. Both approaches are valid. The output matched exactly. How the Three Compare // Speed At this data scale, SQL ran in 0.002-0.010 ms, Pandas in 0.4-2.1 ms. The agent added 2-4 seconds of large language model LLM inference time before any SQL ran. The agent generates code first; that generation time is the end-to-end latency for each query cycle. At warehouse scale, the gap closes to near zero once code is generated; SQL gains further because it runs inside the database engine, and Pandas hits a memory ceiling around 10 million rows and needs Apache Spark or beyond that. Polars https://pola.rs/ // Accuracy and Hallucination Risk SQL and Pandas are deterministic. The same code on the same data gives the same answer every time. With schema-grounded prompts, Claude got all three questions right, but each call produced different SQL different common table expression CTE names, different column aliases, different but equivalent approaches . Without the schema, hallucination risk climbs fast. // Explainability and Debugging A SQL query reads in one block. A bad join condition is visible right in the text. Pandas needs Python fluency, but you can inspect the DataFrame at each step. Agents explain their reasoning in English, then produce code that you may or may not be shown. If the generated SQL is wrong, you're tracing an error through a model's reasoning chain rather than reading a query you wrote. // Flexibility and Production Readiness Pandas is the clearest option for custom transformations, string parsing, and iterative feature engineering. SQL handles set logic cleanly and gets verbose for procedural work. Agents answer plain-English requests well, with the least consistency when schemas are complex or ambiguous. For shipping, SQL is the most proven option in analytics; Pandas is dependable with tests; and agents are dependable today for low-stakes queries or when the output is reviewed before it runs. What the Agent Results Actually Show With a schema-grounded prompt, Claude got all three correct: Easy, Medium, and Hard. The agent's SQL for the Hard question used a different window function pattern than the reference solution and still returned the correct table. Two things limit that finding. First, reproducibility: each API call can return different SQL for the same question. The logic is equivalent, but a team reviewing agent-generated queries needs to verify the outputs rather than trust that today's correct run will match tomorrow's. Second, schema dependency: the prompts above include table and column names, as well as sample rows. Remove those, and the agent guesses. At Easy difficulty, a wrong guess produces an empty result. At Hard difficulty, a wrong guess produces a plausible wrong result with no error. The practical pattern is: provide the full schema, ask for SQL, then run and verify the output before it goes downstream. Conclusion SQL, Pandas, and Claude each got the same three analytics questions right when used correctly. The differences are in speed 0.01 ms vs 4 seconds , reproducibility, and what happens when you reduce the context. SQL fits structured retrieval and set-based logic with millisecond execution and deterministic output. Pandas fits custom transformations and step-by-step notebook work up to about 10 million rows. The agent fits first-draft queries and ad hoc exploration, with the full schema in the prompt and a human reviewing the output. The agent got the Hard question right by using SUM OVER instead of a scalar subquery, which is a valid approach that SQL's reference solution didn't take. That's the honest version of this comparison: the agent can generate correct, creative SQL. It just adds latency, varies between runs, and depends entirely on what you put in the prompt. is a data scientist and in product strategy. He's also an adjunct professor teaching analytics, and is the founder of StrataScratch, a platform helping data scientists prepare for their interviews with real interview questions from top companies. Nate writes on the latest trends in the career market, gives interview advice, shares data science projects, and covers everything SQL. Nate Rosidi https://twitter.com/StrataScratch