Within a few days, Dario Amodei, Elon Musk, and Sam Altman converged on the idea that AI development has to slow down and requires greater oversight.
The visible spearhead was an essay published by Dario, framed around the idea of pacing progress in AI.
But there is a skeptical read of the situation. Even when safety concerns are reasonable and sincere, demanding regulation can serve economic self-interests.
This is a well-known idea in political economy. As the famous Chicago economist George Stigler once argued: “as a rule, regulation is acquired by the industry and is designed and operated primarily for its benefit.” (Stigler, G. J. (1971)).
So let's look at some of these incentives.
One is to close the door behind them. The idea of kicking away the ladder.
Frontier AI companies grew in a relatively permissive environment that gave them enormous agency and room to experiment. Now that they have made it, regulation is an effective way of raising the cost of entry for whoever comes next.
OpenAI, Anthropic, and xAI can afford armies of lawyers, compliance officers, safety researchers, auditors, and government-relations teams. New entrants cannot.
It is not uncommon to hear people in the Valley argue that competition is often preferable to regulation. So, a healthy dose of skepticism requires asking why competition is not the route in this case. Amodei's proposal calls for coordination among frontier AI companies and recognizes that this may require exemptions from antitrust rules.
Such coordination may be justifiable in terms of safety. But there is also an economic argument. If every frontier lab would prefer to spend less money racing toward the next generation of models, but none can afford to slow down while its competitors continue accelerating, regulation is an effective way to coordinate the slowdown.
Another idea is that incumbents may prefer regulation written when they have enormous influence over its design (right now). Leading the regulatory effort not only provides predictability and legitimacy, but also the opportunity to define what counts as "responsible AI."
There is also a more speculative possibility. What if the next generation of models is becoming harder or more expensive to improve? Then, setting expectations around an intentional slowdown could be preferable to facing a technical slowdown. The former is a sign of prudence. The latter is a problem with the technology.
I don't know whether any of these explanations are correct. Also, I am not saying there are no real risks. But that is precisely the point.
The fact that a proposal may be socially desirable does not relieve us of the obligation to analyze the private incentives of the firms proposing it.
Regulating should not be an act of panic. A demand for regulation by the leaders of a sector reflects well-known incentives. Incentives that we need to consider and that should provide us with a healthy dose of skepticism.