Frontier-level agentic SQL performance now fits on your laptop, even if your laptop is almost five years old with only 16GB RAM! By combining Qwen 3.8 27B and DuckDB, everything happens locally. If you want, one config change moves the query workload to MotherDuck to handle any scale.
How does this change the way we work? What does this say about the future of using AI in data? Oh, and let’s get it set up on your machine while we’re at it. It won’t take long!
The Pareto Frontier Has Shifted
We used a custom harness and OpenRouter to run the DABstep benchmark across a variety of models and compare them to Qwen3.8 27B running locally on my laptop(s). I’ll dig into the details in a moment, but first, how did it do?
To say I was impressed with Qwen 3.8 is an understatement. Look how far to the top-left it is!
For free, or under $0.50 if you count electricity costs, Qwen outperformed OpenAI’s GPT 5.6 Luna Max that cost over $8. That’s over 17 times cheaper. Just a few weeks ago, we were singing Luna’s praises and its revolutionary impact. It’s still really powerful. Things just move fast here in AI land. The Qwen 4 bit run was done on an M5 Macbook Air (32GB RAM), and the 3 bit run on my older M1 Pro Macbook Pro (16GB RAM). Since the Air only uses 30 Watts to the Pro’s 100W, it is even more cost efficient!
Gemini-3-Flash remains the most accurate model, but for an additional 2.3x in price over Luna. These benchmarks are slightly noisy, so it is possible that some small prompt improvements could bring performance closer together.
Things are not looking great for Sonnet 5 here. More expensive and significantly less accurate is not a great combination.
As another open weight model data point, Kimi K3 is quite accurate, but at a high price. It is suboptimal vs. Gemini-3-Flash aside from its openness.
This truly is intelligence too cheap to meter. We are quite literally only metering electricity, and on a 30W Macbook Air, that’s 1 single light bulb! Left a light on last night? Could have answered a whole bunch of data questions!
While this benchmarked questions and answers about data, there are a number of other similar workloads where these models are useful. Background research has a lot of similarities. The first task in both is to use business context to understand the question, then the tool usage for research is just web search instead of kicking off a local DuckDB query. SQL is a good proxy for other lightweight coding tasks like scripting or automation too. These tasks are now free, so feel free to kick them off in the background even if there is only a chance they are useful. With a local model this good, and a laptop that powerful sitting on your desk, every day you wait to set this up is a day you’re burning money on tokens! If you have a fairly powerful laptop, you could get several dollars per day (upwards of $100 per month!) of value out of it for pennies in electricity.
Aside from the raw dollars, completely removing the risk of racking up a big bill overnight by accident is really freeing. The failure mode of running slowly on a big task instead of churning through credits is a big benefit. (Side note, MotherDuck’s billing model works the same way - you can’t accidentally send a query across thousands of machines!).
One of my favorite data workloads that fits that pattern is query optimization. “Hey Qwen! Look at all the queries I ran today and make them faster. You have the next 12 hours to experiment. Use data science best practices. Go!”.
Powerful open models open up other doors around self hosting these open weight models, possibly fine tuning them, or even tapping into a decentralized network of independently owned Macs for cheap inference!
Let’s Get Your Laptop Fans Whirring
So how can you get this set up locally to get in on these benefits? First, what kind of machine do we need?
You’ll want a laptop with over 16GB of unified memory or a discrete graphics card with 16GB of RAM. For things to work most smoothly, having 32GB of RAM does help a fair bit. Macs (especially Macbook Pros) also have higher GPU memory bandwidth so they tend to be higher performing than other laptops.
As a total newcomer to local models, it only took me an hour or so of trial and error to get the basics working. Now you can skip those steps and be up and running in minutes!
Point DuckDB at your data and ask a data question!
I did a number of smaller tests to tweak settings to try and get the most out of my machine. I found the best approach was to ask my agent to build a DOE (design of experiments) to try a variety of combinations of settings. This is important! The settings are very interrelated, so tuning one at a time will not give you the results you’re looking for. I also monitored memory pressure and swapping (meaning I asked Claude to measure that for me!) and asked for ways to lower memory use as well.
Memory is the most precious resource when running a local model, so I would recommend a clean restart before any benchmarking. At least close all your Chrome tabs…
I ended up getting about 5-7 tokens / second when looking at total elapsed time on my M5 Macbook Air, which is close to the 8.8 tokens/sec that memory bandwidth would project.
When using an older M1 Pro Macbook Pro with 16GB RAM, I used a 3bit quant. That means that the model’s weights were rounded even more aggressively in order to save RAM. Accuracy suffered somewhat (only 96.4% accurate instead of 98.6% with the 4bit), but it still worked, even on nearly 5 year old hardware! I saw about 5 tokens/second looking at elapsed time, because the memory bandwidth on the M1 Pro chip is actually higher than the M5 [non-pro] on the Macbook Air, but the M5 has some nice neural accelerators that give it an edge.
What is This MTP Thing?
MTP, or multi-token prediction, is a really cool approach to boosting output by guessing what the next few tokens will be in advance, then checking if they were correct. It’s most similar to speculative execution in CPUs. A CPU will do branch prediction to try and execute calculations ahead of time (exploring the most common side of an if statement, for example). When it gets things right, you get faster speeds. If it’s wrong, it just falls back to running your calculations in series.
Since the tokens are checked, there is no risk of any inaccuracy. I like risk free performance!
On the more modern Macbook, MTP was helpful with a draft size of 3. I saw somewhere around a 30% boost.
On the older Macbook Pro, MTP was actually slowing me down! It’s because the bottleneck was somewhere else in the system - the hardware just wasn’t designed with this kind of workload in mind. It took too much time to guess ahead and check those answers.
Our custom harness uses the DABStep benchmark to simulate how a data team should manage their context layer. First, general documentation (including some SQL snippets) is fed into Claude Fable 5 and converted into MotherDuck Guides. Guides are just markdown living right next to your data that are linked together with relationships. If you want to work locally, markdown files and some links might just do the trick.
So yes, we do use a large frontier model for part of this workload, but it is a fast and inexpensive portion of the flow. The cost sensitive part is when the question and answer sets are evaluated and that’s where the local models come in.
Once an initial context layer is built, it is iteratively tested with the smaller AI model against 26 Q&A pairs as a training set. Each question must match an exact text output. If any questions are answered incorrectly by the smaller model, the context layer is improved to document missing assumptions and business rules. The improvement uses the larger model to try and really refine that central set of definitions.
With a 26/26 score exiting the training and improvement loop, the full test set of over 400 questions is run and scored. This set is run using the smaller model as well because that is the only scalable way to run this kind of process in production.
Nobody should run Fable for their evals at scale unless their company rhymes with “gyroscopic”. I guess they could use Sol if it rhymes with “orange banzai”.
The Rest of the Story
I’m convinced that local models have finally reached the tipping point of being useful. Qwen3.8 27B in particular shocked me and made me rethink what is possible in a 27 billion parameter model.
There are still some things that are not quite equal though. First, the runtime:
Qwen took 5-6 minutes to answer each question on average vs. about 25 seconds for Gemini-3-Flash or 40 seconds for GPT 5.6 Luna. There are some disclaimers here though! I think I could tune things up a little bit more, so there could be another 20-30% performance left on the table. New quantizations keep coming out that push Qwen even further as well.
A key factor though is the hardware. M5 Max Macbook Pros actually have 4x the memory bandwidth of the Macbook Air. If that translates to 4x the performance (a decent assumption), then 90 seconds starts to be a lot closer to the frontier lab models.
However, I was only really able to run 1 prompt at a time locally. I routinely blasted 15 questions at a time through OpenRouter for this benchmark, and you can scale out prompts on the cloud as much as you are willing to pay for. Subagents are quite handy to run in parallel.
This approach also only applies to those with beefy enough hardware. Some brief Claude research estimates around 1/3rd of laptops have over 16GB RAM, which means most folks still aren’t going to be able to take full advantage. If you limit it to just Macs with 16GB+, for their faster memory bandwidth, that number drops to more like 1/30th of folks.
But I’d guess it’s a bit more than 3% of the folks reading this post, so if you have the horsepower, you should try this out!
In this benchmark, the construction of the context layer was also still done with a large frontier model (here, Fable 5). Local models are unlikely to cover all use cases just yet, at least on midrange consumer hardware.
Lastly, we are only considering electricity costs since we assume you already have a laptop. If you include depreciation expense for the laptop itself, the cost of a local model increases to about $6 per 1000 questions answered. What’s the math there? If we assume a laptop costs $2000 and lasts 4 years, then 6 minutes of depreciation per question gives you $5.70 in costs to answer all 1000.
So, don’t buy a Macbook Air just to run Qwen. You should also, you know, use the laptop.
Get That Laptop Quacking!
Local models are strong enough that they are worth the hour they take to get set up. Pair them with DuckDB, and you have a powerhouse data exploration and analysis toolkit for free, right where you’re already working.
If your data is too big to analyze locally, MotherDuck’s hypertenancy architecture gives each agent their own sandbox in the cloud. That comes with nice compute isolation guarantees (no agent can slow down the CEO’s dashboard!) as well as data compartmentalization (your agent’s data is private by default). Plus, costs are per sandbox, so no one query can explode across servers and balloon your spend. MotherDuck’s Guides also give you a super easy way to make sure that your business context is right at your agents’ “fingertips” whenever they need it. Qwen3.8 27B and DuckDB is a uniquely powerful combo to use the laptop you already have to understand your company’s data, and MotherDuck is there if you need more power.
Today Duck Labs, the developers of DuckDB, announced they are being acquired by Amazon. This is big news in the duck-iverse, and many people are wondering what this will mean for everyone’s favorite duck-powered database, MotherDuck.