S tech companies are calling on American policymakers to support open models. But they may be swimming upstream, and they have one big naysayer.
On Friday, a group of 25 leading American tech companies led by Microsoft and Nvidia published an open letter titled "Open Weights and American AI Leadership," calling on policymakers in the US to:
- Avoid blanket bans on open-weight AI models
- Take targeted legal and commercial steps against large-scale model distillation (see KimiandAlibaba) and avoid sweeping restrictions on distillation for legitimate purposes - Expand access to compute and other AI resources for researchers and startups to expand US innovation and competition in AI beyond a small handful of frontier labs
The main point of the open letter can be summed up by one quote from it: the idea is "keeping the frontier plural by avoiding premature restrictions on open models that stifle competition or drive innovation overseas."
By Monday night, the list of companies that signed the open letter had swelled to 133, and it included two of the three leading frontier labs in the US: OpenAI and Google.
Conspicuously absent was Anthropic. But by the end of the day on Monday, CEO Dario Amodei had published a post titled "Our position on open-weights models" that:
- Asserted "Anthropic has never advocated for a ban on open-weights models"
- Expressed concern about the risks of AI capabilities falling into the hands of authoritarian governments, including China
- Raised the specter of AI models being misused to carry out cyberattacks and biological attacks
- Recommended that the most powerful models, closed and open, should be subject to mandatory safety testing and that testing should be based on a global, internationally agreed-upon standard
Where Amodei and the signers of the letter are the farthest apart is on the issue of cybersecurity.
The letter wrote, "Just as open-source software demonstrated that transparency can be more secure than obscurity, AI safety may depend on giving more people the ability to test and strengthen the models on which society relies."
Amodei stated, "I don’t agree with the letter’s assertions that open-weights models necessarily make it easier to develop safeguards or that broad access to capabilities necessarily helps defenders more than attackers. It seems at least as likely to me that the opposite will be true. For example, I worry that biology will have a strong attacker-defender asymmetry, where sufficiently capable models may be able to quickly weaponize pandemic-level viruses with widely available materials, whereas defense against these agents is a multi-year operational task in the best case."
The other area worth highlighting is the economic incentives. Today, there has been so much investment in OpenAI, Anthropic, Google, and a host of other public companies and startups in the US, and those investors are deeply committed to profitable returns. That doesn't naturally lend itself to a long-term commitment to a cooperative environment, unless there's a crisis or another unifying factor.
However, I was able to find someone to disagree with me on that.
"Open models are extraordinarily important," Anastasios Angelopoulos, CEO of Arena, one of the signers of the open-letter, told The Deep View. "And the reason, fundamentally, is that companies want to own their own intelligence."
Angelopoulos said, "I actually view American open-source as being inevitable. That's why we signed the letter. The economic incentives for a huge American open-source company [are] too strong to ignore, because everyone's going to want to own their own intelligence, and everyone's going to want to be able to fine-tune their [own] large and small models. And nobody's going to want to do it based on an intelligence chain that relies on foreign imports."
Our Deeper View #
We should focus less on who turns out to be right in this debate, because none of these are fundamentally original ideas. The greatest value from this open letter, like OpenAI's AI New Deal, the Pope's AI manifesto, and Demis Hassibis's AI safety plan is that it's creating societal dialogue around the kind of world we want to create in light of AI. This is a tacit acknowledgment that nothing is written in stone and there's no predetermined course of action that AI is going to follow. There are tons of choices, large and small, to be made in shaping a future that will be informed by AI capabilities. More dialogue will be needed at every level of society to ensure the best outcomes for the most people.