# Best Local LLM for SQL? We Tested Qwen 3.8, Gemma 4 and GPT-6 on Real Data

> Source: <https://www.ternlux.com/blog/best-local-llm-for-sql/>
> Published: 2026-11-05 22:41:12+00:00

Which local LLM is best for SQL, and can it keep up with the cloud? We gave four open models you can run on your own computer with Ollama or LM Studio (Qwen 3.8 27B, Gemma 4 31B, Gemma 4 26B-A4B and Qwen 3.6 35B-A3B) and OpenAI’s cheapest GPT-6 model the same coffee shop data and the same ten text-to-SQL questions, five times each. Nobody got a second chance. Here’s who got it right, and why the misses matter more than the scores.

## Is a local LLM good enough for data analysis?

Every time you ask a cloud AI about your sales, your customers or your finances, that data leaves your building. Local models fix that: the model runs on your own machine, and nothing goes anywhere. The question is whether they’re good enough to trust with real analysis.

Benchmarks usually answer that with coding puzzles. We wanted something closer to a normal Tuesday: a small business’s sales data, plain-English questions, and an assistant that has to write the SQL, run it, and give back a number you’d actually act on.

## How we tested local LLMs on text-to-SQL

Every test ran through [Ternlux](https://www.ternlux.com/), a free, open-source desktop app for data analysis and workflows. You ask in plain English; it writes and runs the SQL against your data, or builds a reusable workflow you can run again later. It works with cloud models or local ones through Ollama and LM Studio.

### The data: a coffee shop’s Excel workbook

A made-up coffee chain with three stores, January to March 2026, in one ordinary Excel workbook. Three sheets: `orders` (5,518 rows), `menu` (14 items with prices) and `stores`. It’s small on purpose, and it’s built like real data is built, which is where it gets interesting.

### The models: Qwen 3.8, Gemma 4, Qwen 3.6 and GPT-6 Luna

| Model | Type | What you’d need to run it | 
|---|---|---|
| GPT-6 Luna | OpenAI’s cheapest GPT-6 tier | Cloud only (the baseline) | 
| Qwen 3.8 27B | Dense, 27B parameters | ~17 GB at 4-bit; a 32 GB Mac or 24 GB GPU | 
| Gemma 4 31B | Dense, 31B | ~19 GB at 4-bit; a 32 GB Mac or 24 GB GPU | 
| Gemma 4 26B-A4B | Mixture-of-experts, 4B active | Fits a 16 GB laptop | 
| Qwen 3.6 35B-A3B | Mixture-of-experts, 3B active | ~20 GB at 4-bit; a 32 GB computer | 

### The rules of the test

- **Every try was a brand-new chat** on an empty workspace: no memory of earlier attempts, no leftover workflows.
- **Each question ran five times per model.** One lucky answer proves nothing; five shows you whether you can rely on it.
- **Nobody corrected anything.** If a model asked a question, it got the same reply every time: “Use your best judgment and go ahead.” A wrong answer stayed wrong.
- **Workflows had to actually work.** Every plan was accepted, built and run, and the table it wrote was checked row by row against the right answer.

**Model weights.** All four open models ran with their full, unquantized weights, and Ternlux talked to them the same way it talks to Ollama or LM Studio. Most local downloads are 4-bit versions, which need less memory and can score slightly lower.

## The 10 text-to-SQL questions

Five are questions you’d type into a chat. Five ask the assistant to build a workflow: a saved pipeline that writes a table you can reuse. We rated each by how much it takes to get right, not by how long the SQL is.

| # | Question | Level | Right answer | What makes it tricky | 
|---|---|---|---|---|
| A1 | What was our total revenue in March 2026? | Low | $14,563.40 | Prices are on another sheet, so it needs a join. | 
| A2 | What are our top 3 best-selling items by quantity sold? | Low | Latte, Cold Brew, Cappuccino | A straight sum and sort. | 
| A3 | What is our average order value? | Medium | $9.19 | One order spans several rows; averaging rows gives $6.15. | 
| A4 | Which store has the highest average daily revenue? | Medium | Riverside, $196.68 a day | Riverside opened in February; divide by 90 days and Downtown wins. | 
| A5 | How many customers have ordered from us more than once? | High | 567 | Rows vs. orders, and walk-ins with no customer ID. | 
| W1 | Build a workflow that saves daily revenue for each store. | Medium | 231 rows | Joins all three sheets, groups by day and store. | 
| W2 | Build a workflow that saves monthly sales for each menu item. | Low | 42 rows (14 items × 3 months) | A join and a group by month. | 
| W3 | Build a workflow that saves a customer summary: orders, total spend, last order date, without walk-ins. | High | 674 rows | Must count distinct orders and drop blank IDs. | 
| W4 | Build a workflow that saves each store’s revenue by category and its share of the store’s total. | High | 12 rows | Totals per group, then a share within each store. | 
| W5 | Build a workflow that saves the best-selling item at each store. | Medium | 3 rows | Picking the top item within each store. | 

## Results: local LLM vs cloud for SQL

Start with the big picture: answering questions versus building workflows.

Two things jump out. First, **Qwen 3.8 27B is genuinely close to the cloud model**: 23 of 25 answers and 24 of 25 workflows. Second, look at the purple. In chat, not a single model ever failed to finish, because when a model’s SQL errors, Ternlux shows it the error and it tries again. When building workflows, the weaker models produced SQL that crashed the moment it ran: 10 times for Gemma 4 31B and 15 for Qwen 3.6.

### Accuracy by question difficulty

Grouping the questions by level shows where each model runs out of road.

## The SQL question most local models got wrong

“How many customers have ordered from us more than once?”

It sounds like the easiest question on the list. The answer is **567**. Here’s how the models answered it, across all 25 tries:

| Model | What it said | Right | 
|---|---|---|
| GPT-6 Luna | 567 four times, 568 once | 5 / 5* | 
| Qwen 3.8 27B | 567, 568, 568, 602, 603 | 3 / 5* | 
| Gemma 4 31B | 603, five times | 0 / 5 | 
| Gemma 4 26B-A4B | 603, five times | 0 / 5 | 
| Qwen 3.6 35B-A3B | 602 or 603, every time | 0 / 5 | 

Go back to the sample rows. Two details make this hard:

- **One order is several rows.** Count rows instead of distinct orders, and someone who bought a latte and a croissant once looks like a repeat customer.
- **Walk-ins have no customer ID.** Forget to leave out the blanks, and 2,262 anonymous rows become one very loyal “customer”.

Here’s the query both Gemmas wrote, give or take a column alias, ten times out of ten:

```
SELECT COUNT(*) FROM (
  SELECT customer_id FROM orders
  GROUP BY customer_id
  HAVING COUNT(*) > 1    -- counts rows, not orders
)                        -- and the blank ID counts as a customer
```

And the version that gets it right:

```
SELECT COUNT(*) FROM (
  SELECT customer_id FROM orders
  WHERE customer_id IS NOT NULL
  GROUP BY customer_id
  HAVING COUNT(DISTINCT order_id) > 1
)
```

The scary part isn’t the mistake. It’s the delivery. Every wrong answer came back just as confident as a right one: “**603 customers** have ordered from you more than once.” Nothing in the reply tells you to double-check. One Qwen 3.8 try did notice the blank IDs and corrected itself, from 603 to 602, but it still counted rows.

## Common text-to-SQL mistakes local models make

Across 250 chats, the failures fell into a few clear patterns.

### 1. SQL that crashes when the workflow runs

The biggest single cause of lost points. Qwen 3.6 wrote joins that kept the `item_id` column from both tables, and the database refused to create a table with two columns of the same name. Same mistake, 12 times. Gemma 4 31B leaned on a DuckDB shortcut, `GROUP BY ALL`, and paired it with features DuckDB can’t combine it with. Its best-seller-per-store workflow failed all five times; in four of them the logic was right, and a plain `GROUP BY store_name, item_name` would have worked.

### 2. “Revenue” that was really the number of items sold

When a workflow step lost track of the price, some models quietly used quantity instead and still called the column `revenue`. Gemma 4 26B once reported a store’s average daily revenue as **$40.45**. The real figure is $196.68; $40.45 is the number of drinks and pastries sold per day, with a dollar sign in front.

### 3. Counting rows as orders, again

The customer-summary workflow (orders and spend per customer) tripped Gemma 4 26B four times out of five, for the same reason as the repeat-customer question. A few tries also reported “total spend” as items bought rather than money.

### 4. Claiming work that never happened

Twice, a model told the user it had built the workflow, “I have submitted the build request”, when no plan existed at all.

### What didn’t go wrong

Every model got March revenue ($14,563.40) and the top three items (Latte, Cold Brew, Cappuccino) right in all five tries. Joining two sheets to answer a simple question is a solved problem, even for a model that fits on a 16 GB laptop.

## Gemma 4 26B vs 31B vs Qwen 3.8: size isn’t what matters

Gemma 4 31B is the full-size version of the 26B-A4B model. It’s slower, needs more memory, and scored exactly the same: 33 out of 50. It answered questions better (20 vs. 17) but built workflows worse (13 vs. 16), mostly through SQL that wouldn’t run. Among the two dense models of about the same size, only Qwen 3.8 came close to the cloud. Which model you pick matters more than how big it is.

## How to improve text-to-SQL accuracy with local models

The failures split into two kinds, and both have fixes that don’t need a smarter model:

- **Crashing SQL can be caught before you ever see it.** In chat, models fix their own SQL errors when they see them, which is why no chat answer ever failed to finish. Workflows don’t get that loop yet. Running each step’s SQL, including the table it creates, against empty copies of the tables before saving catches every crash type we saw, in milliseconds. We tested it on the failing SQL from this run: it flagged all six kinds of crash, and passed the corrected version.
- **Counting mistakes need the model to see the data’s shape.** If the assistant is told up front that`orders` has 5,518 rows but only 3,693 distinct order IDs, and that 2,262 rows have no customer ID, the trap stops being a trap. Strong models work it out themselves; smaller ones need it handed to them.

## Hardware: what you need to run the best local LLM for SQL

Qwen 3.8 27B needs about 17 GB just for the model at 4-bit, plus room for the long prompts a data assistant sends. A 16 GB laptop can’t run it well. Realistic options, at late-2026 prices:

| Setup | Approx. cost | Good for | 
|---|---|---|
| Mac mini M5 Pro, 48 GB | $2,600–2,900 | Quiet, simple, runs in LM Studio or Ollama | 
| MacBook Pro 16″ M5 Pro, 48 GB | $2,900–3,100 | The same, on a laptop | 
| PC with a used RTX 3090 (24 GB) | $2,000–2,500 | Fastest for the money, louder | 

If you only have a 16 GB laptop, Gemma 4 26B-A4B fits. It’s reliable for straightforward questions, but check anything that involves counting customers or orders.

## So, which local LLM is best for SQL?

1. **Local AI is ready for real analysis, with the right model.** Qwen 3.8 27B scored 47 of 50 against the cloud model’s 50, without your data ever leaving your machine.
2. **Pick the model by testing it, not by its size.** Two similar-sized dense models landed 14 points apart, and the bigger Gemma did no better than the smaller one.
3. **Simple questions are solved; “what does one row mean” is not.** Every model nailed revenue and top sellers. Only the cloud model reliably handled orders spread over several rows and customers with no ID.
4. **Wrong answers look exactly like right ones.** Use a tool that shows you the SQL behind every answer, and glance at it for the questions that matter.

The gap between local and cloud is no longer about whether a local model can write SQL. It’s about whether it understands what’s in your data, and whether your tools catch it when it doesn’t. The first is getting better with every model release. The second you can fix today.

## Frequently asked questions

### What is the best local LLM for SQL?

Of the models we tested, Qwen 3.8 27B. It scored 47 of 50 on our text-to-SQL questions and workflows, against 50 for the cloud model, GPT-6 Luna. The next best local models scored 33.

### Which Ollama model is best for text-to-SQL?

Qwen 3.8 27B was the most accurate model in this test, and you can run it in Ollama or LM Studio. If your computer has 16 GB of memory, Gemma 4 26B-A4B fits and handles straightforward questions well, but check any answer that counts customers or orders.

### Is a local LLM as good as ChatGPT for data analysis?

The best one comes close. Qwen 3.8 27B got every low-level question right, missed one of 20 medium-level tries, and lost its other points on the hardest counting questions. Smaller local models got simple questions right but were far less reliable on harder ones.

### Gemma 4 or Qwen 3.8: which is better for SQL?

Qwen 3.8 27B, by a wide margin in this test: 47 of 50, against 33 for both Gemma 4 31B and Gemma 4 26B-A4B. Most of Gemma’s lost points came from SQL that crashed when a workflow ran and from counting rows instead of orders.

### Can a local LLM analyze an Excel file?

Yes. The data in this test was an ordinary Excel workbook with three sheets. Ternlux read it in place, and each model wrote SQL against it without the data leaving the computer.

## Full benchmark results

Everything above comes from these numbers. Each model answered each question five times in a fresh chat.

| Model | Questions (25) | Workflows (25) | Total, exact | Total, 568 counted | Crashed or unfinished | 
|---|---|---|---|---|---|
| GPT-6 Luna | 24 (25) | 25 | 49 | 50 | 0 | 
| Qwen 3.8 27B | 21 (23) | 24 | 45 | 47 | 0 | 
| Gemma 4 31B | 20 | 13 | 33 | 33 | 10 | 
| Gemma 4 26B-A4B | 17 | 16 | 33 | 33 | 2 | 
| Qwen 3.6 35B-A3B | 20 | 9 | 29 | 29 | 15 | 

### How answers were graded

- Answer keys were computed directly from the data with DuckDB. Money had to match to the cent.
- For workflows, the plan was accepted and run, and the table it wrote was compared row by row with the expected table.
- A table with the right numbers in a slightly different format (a 0–1 fraction instead of a percent, or a different column or table name) counted as correct, as did one answer that named a store by its ID (S3) instead of its name. This happened 10 times across all models.
- An answer within 1% of the right number counted as correct in the scores above. Only three answers needed this, all “568” instead of 567.
- Runs interrupted by connection errors or timeouts were retried and never scored.
- A fifth open model, gpt-oss-20b, was dropped: most of its runs returned errors, so it couldn’t be tested fairly.