The debate around open-source AI is reaching a fever pitch. Now, two of open-source's most prominent supporters want to make it easier to use.
On Thursday, the Allen Institute for AI, or Ai2, announced a deepened partnership with model hosting platform Hugging Face to expand access to "fully open AI" for users. To start, Hugging Face will roughly triple Ai2's storage for models. Additionally, Ai2's traffic will no longer be subjected to Hugging Face's standard rate limits, allowing for its largest datasets and multi-checkpoint models to download at full throughput.
As it stands, Ai2's models have been downloaded more than 50 million times from the Hugging Face Hub since 2024, and currently sits as the most active model provider on the platform. Peter Clark, interim CEO of Ai2, told The Deep View in an exclusive interview that the organization was reaching capacity with what it was able to provide on Hugging Face prior to entering the partnership.
As the nonprofit continues to publish open-source AI that goes beyond just open weights, including things like training data, checkpoints and evaluations, adding more infrastructure from Hugging Face will allow it to keep pace as its portfolio grows.
"That's a key part of openness: not just doing the open research, but having the appropriate infrastructure to drive it," said Clark. "This expanded partnership … [will] allow us to put more work into the open. People will be able to download the models faster, and that's all in service of open science."
Clem Delangue, co-founder and CEO of Hugging Face, told The Deep View the goal of the partnership is to help return the AI industry to the state it was six years ago: "extremely open, transparent and collaborative," he said.
"Almost the default was to share the models openly, to discuss them, to share the learning," Delangue said. "That's what led, in my opinion, to the acceleration of the progress of the field."
In the past several years, the industry has largely moved away from open-source in the US, leaning towards closed, commercial models that are hidden behind closed doors and sold through APIs, Delangue said. This, however, has created a concentration of capabilities in the hands of those proprietary providers. That concentration extends beyond just wealth and revenue, Delangue said, allowing a small pocket of companies to centralize influence and power.
"There's a clear path where you end up with a few companies dominating the fields to a level that really never existed before," said Delangue. "We are starting to see that these companies have massive, massive power. If nothing changes, if we don't have more open resources, we could end up in a very different world than what we've seen in the past."
Our Deeper View #
The partnership comes at a particularly poignant moment for open models, with leading firms including Nvidia, Microsoft, OpenAI and Google signing a letter last week urging US policymakers against blanket bans on open models. Additionally, the centralization of power in AI has become a common topic of discussion among AI leaders, with the likes of OpenAI's Sam Altman and Greg Brockman and Anthropic's Dario Amodei all calling attention to the risks of over-centralization. Investing in a more robust open-source ecosystem in the US may well play a major role in democratizing AI, beyond just giving people free access to a less expensive chatbot. However, access is only half the battle. For open-source to gain more traction, education about the benefits of the tech needs to go hand in hand with making it more accessible. Without education, most people will likely continue using the same platforms they've come to rely on.