There’s been a lot of chatter lately about open-source AI models. Some of the latest releases from Qwen, GLM, Moonshot, and even Deepseek are nearly as good as the frontier American models, but much less expensive.
Of course pushing the frontier of intelligence for less money is exciting. But I think improvements in the very cheapest models could prove more transformative. In fact, at the far end of the spectrum, there is some pretty compelling intelligence available for free.
One example are LLMs that can be run locally on regular consumer hardware. These are becoming surprisingly capable, and are effectively “free”, since they run on the computer (or phone) you already have.
There are plenty of powerful open-source models that can run locally on something like a DGX Spark or a high-end graphics card. But most people don’t already have these laying around, so I wouldn’t really consider the models you can run with them “free”. (And yes, you still do have to pay for the electricity to charge your device, but that's pretty much negligible.)
True, on most machines, they’re not quite good enough or fast enough for agentic coding (yet). None of these models are going to impress you compared to what’s state-of-the-art today. But a lot of tasks simply don’t need state-of-the-art intelligence.
I use an M1 MacBook Pro with 16GB of RAM — far from a top-end setup. I've played around a bit with local models, but had just assumed nothing very good would work on my run-of-the-mill machine. But I saw Ornith 1.5 come out the other day, and decided to test out the 9B version. And it was pretty solid!
As one of my quick benchmarks, I asked it to “make a really cool website about ducks as an all-in-one HTML file”. Here’s the result after about 5 minutes (you can click to see the full page):
It’s shocking how far we’ve come in the last year or two. This feels like something I might have gotten from Sonnet 3.7, if not better, and I can run it on my four-year-old MacBook.
It turns out OpenRouter also has several LLMs available for free through their API, which I haven’t really seen discussed much. These do all use your queries for training, and there are pretty strict rate limits. Maybe that’s why more people aren’t excited about this.
One example is a free content-safety model from Nvidia. All it does is classify content as safe or unsafe. But it seems to work well, and it’s pretty cool this is available for literally free. (In fact, I just set this up to auto-flag unsafe user comments for review as part of a new project I’m working on.)
Having some fun with the content-safety classifier
There are plenty of more powerful LLMs available, too — models like Nemotron and GPT-OSS-120b. Plus, you can sometimes find even better models available for free, typically as a promotion for either the model or the inference provider. OpenRouter offers a free models router that will automatically send your request to one of these models at no cost.
I decided to try out Nemotron 3.5 Ultra on my duck website benchmark, using the same prompt as above. It cooked for almost 15 minutes, and eventually came back with this:
Okay, not great. Still not too bad considering it was free. A few follow-up queries or a little debugging and this could be a solid output.
Overall, I’m pretty surprised how good Ornith looks here when compared to Nemotron 3 Ultra, which has 6x more active parameters. I’m eager to play around with Ornith a little more to test its capabilities and explore some local use cases.
OpenAI recently announced they were upgrading the free version of ChatGPT to use GPT-5.6 Luna, which is very capable. It’s a model that even singularitypilled AI-maxxers might use, maybe as a subagent or for easy tasks where speed matters.
Without even needing to log in, I went to chatgpt.com and gave it the same prompt (“Make a really cool website about ducks as an all-in-one HTML file”). In less than 30 seconds, it gave me this:
I mean, this is pretty good. You could nitpick a couple of things, but this was provided for free, almost instantly!
Improvements in free ChatGPT, or eventually in whatever intelligence Apple and Google provide for free, have huge potential for impact. Most people still don't pay for premium AI, so their capabilities are limited to whatever's available for free (or maybe what their employer offers, like Copilot).
Imagine an average non-techie person trying to make a website a couple of years ago. First of all, they’d likely assume they couldn’t do it themselves. If they did try, they’d probably end up paying for something like Squarespace.
Now, without spending a dime, you can just ask for a website and get one back in seconds, plus detailed step by step instructions for how to get it online if you need them. That’s pretty wild.
In a few more years, if scaling and cost decreases continue, Sol- or Fable-level intelligence could be available for free locally, or as part of an ad-supported business model from one of the big providers. What does that world look like?
It’s hard to say for a few reasons:
Still, even without further improvement, there is huge potential for optimization and adoption of the free capabilities that are available today. Maybe someday we’ll all reminisce about tokens like we do about cell phone minutes, laughing that companies used to charge us for them. ⧈