Stop Wasting Cloud Tokens - Local AI FTW

Enterprises should prioritize building datasets and customizing small AI models over relying solely on cloud-based frontier models, according to a tech commentator. The argument emphasizes that customized small models can outperform frontier models and protect intellectual property, with tools for customization expected to become significantly easier within about a year.

Yes, but only if 100% – but there’s two threads here. And that lack-of-labor is why I’m so insanely frustrated with the way we see many CIO/CTOs deploying AI “We can do more with less people ” Noooooooooo There was never enough labor to begin with – to do things the way we want to the way things ought to be done. We can use this as a labor augmentation tool and we must not lose the people that know what remains to be done. So I think based on discussions I can better articulate what’s crystalizing in my mind for enterprise even small/medium 100-1000 staff organizations : If you can get deepseek v4 flash for $1 per million or $5 per mil from a cloud provider you trust then yes, I agree with you, don’t bother with local AI yet, let me bring it back there . Use that to get your dataset. Or don’t and just use the frontier SaaS model for everything. BUT you MUST build the dataset. If you have the dataset THEN you can customize/fine-tune the small model. THAT is the benefit.. that’s the juice that’s worth the squeeze. Your argument holds up if you’re just deploying vanilla qwen or vanilla deepseek and “everything else” around that is a heavy lift for your organization. However, if you build the dataset then spend some time experimenting with the dataset and customizing small models THEN, I think, you’ll end up with something that beats frontier models AND protect somewhat against competitors that get a Claude account and say “Hey Claude, using everything you’ve learned from your biz clients in $YOUR INDUSTRY help me in my $COMPETITOR COMPANY” – your biz gets to keep their process/IP moat the more you “obscure” from conglomerated cloud models how/what exactly you’re doing. So to recap: Maximal Small Model Value comes from customization. The biggest-bang-for-buck from customization comes from gathering data on how the org uses the tools fancy, non-local, etc and customizing a “small” model. Your customized “small” model that is “local” may be cloud hosted private cloud, neo cloud or on prem. Or, by then, maybe we have better local hardware cheaper. Local here might be 27b parameters of 270b or 1t by then.. who knows. This is when it becomes not a distraction for enterprises, I think, is the level of customization and specialization. Tools to do this will emerge. You’re right everyone’s stretched too thin BUT whats exciting here is that this is at least two orders of magnitude less effort in the tooling for customization. If you step back on a historical timeline a lot of the “no one has time to become an expert in ” was true for everything in the history of compute. Networking. Databases. Directory services. Novell/Directory being too complicated and then LDAP and then LDAP being too complicated still is how we ended up with Active Directory kind of thing lol not really but sorta kinda . How complicated “the flywheel” as a tool for improving how your org can use the tool was a couple years ago was totally unreachable, I agree, but at this point it can almost not quite, maybe another year away be packaged into a product that sits in front of your cloud usage. I’m pretty sure that is exactly what some firms in Asia are already doing and they’re so far ahead that’s why OpenAI and Anthropic said it was a distillation attack they didn’t detect.. not really? It was them doing this exact thing on a grand, grand scale. Maybe. I’m not sure. Eventually with enough of this data “the flywheel” can lead to recursive self improvement of these algorithms and structures for your organization. Probably. But that will take a while…