{"slug": "what-s-scarier-than-agents-taking-over-internet-ceo-cartel-trying-take-over-ai", "title": "What's Scarier Than Agents Taking over Internet? CEO Cartel Trying Take over AI", "summary": "A developer and self-described \"geezer\" argues that Anthropic CEO Dario Amodei's proposal to pace frontier AI development amounts to a cartel of leading labs, warning that embedded evaluators and antitrust waivers would burden startups while entrenching incumbents. The critique targets Amodei's three-part plan — embedded evaluators, democratic coordination, and global coordination — and notes that Elon Musk's xAI quickly agreed to mutual model access despite not offering similar scrutiny to SpaceX or Tesla. The author contends that verification of any global slowdown is unworkable and that waiving antitrust law to let Amodei, Altman, and Musk coordinate would defeat the purpose of that law.", "body_md": "# Background\n\nAnthropic CEO Dario Amodei [wrote](https://darioamodei.com/post/we-must-pace-the-frontier):\n\nI believe that AI could cure most major diseases in the next 5–10 years, greatly accelerate economic growth rates, create a world of abundance and empowerment, and usher in a renaissance of democracy and freedom…\n\nBut like many technologies before it, AI brings risks, and because it is such a powerful technology, these risks are serious…\n\n**We must slow the pace at which we improve the capabilities of AI models.**\n\nAmodei goes on to propose:\n\n1. **Embedded Evaluators** who work in the AI firms and have employee-like access to what is being done there. He said Anthropic is committed to this step and Sam Altman quickly agreed OpenAI would do the same.\n\n2. **Democratic Coordination.** “Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress. Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support.” By government support, he means loosening antitrust law which is specifically meant to exclude this type of anticompetitive behavior.\n\n3. **Global Coordination.** “The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.”\n\n# IMO\n\nAs a geezer, I would prefer that we not slow the pace of curing most major diseases.\n\nEven the seemingly harmless suggestion of neutral embedded evaluators (as if there were such a thing) seated in the AI labs is a step towards strangling competition. Costs the big guys nothing and might even shield them from some liability. An impossible burden for a startup which might someday challenge the current leaders. If I start a one-man AI company (I would if I were younger), do I need to pay a couple of “observers” to sit in my garage and use my precious computer time? But, if Anthropic, OpenAI, and xAI want observers in their labs or want any other kind of help with their own safety responsibilities, more power to them. Just don’t let them inflict observers on rivals.\n\nThe demand that antitrust law be waived so that the companies can observe each other’s product development directly defeats the purpose of that law which makes it a criminal offense for companies to share information to avoid competing. It will let the big guys slow down but, they hope, retain their cartelized lead over new entrants who won’t get the benefit of the information which is being shared. Elon Musk, whose company is NOT in the lead in AI, quickly agreed that they ought to look at each other’s models every couple of weeks for safety’s sake. He had not offered similar access to SpaceX or Tesla’s robot technology (a different form of AI) even though rockets, cars, and trucks loaded with fuel can be weaponized as planes were on 9/11. No way we should waive antitrust laws to let Musk, Amodei, and Altman form a cartel. Even if they don’t think so, they have plenty of market power already.\n\nThe proposal for global coordination on “safe” development of AI is the escape hatch for these companies. A nuclear test ban treaty could work for a while because nuclear testing is detectable with a seismograph. Even if China agreed to such an arrangement (they won’t), how would we verify or, from their PoV, how could they verify we aren’t cheating. Is Iran going to agree to a verifiable slowdown? North Korea? What about Russia? Or Ukraine which has proved pretty adept at weapon development but can’t afford to fall behind Russia? Amodei knows damned well this isn’t going to happen:. He write:\n\nA full pacing, or even “pause”, in which participating governments agree to substantially limit the overall rate of AI development. I support floating this, but I think it is unlikely to actually happen any time soon: defecting from such an agreement by evading monitoring could radically shift the balance of global power, so I expect the incentives to do so to be enormous and the level of confidence we would need in verification to be very high.\n\nThe world isn’t going to be saved by unilateral AI disarmament. It will become an even more dangerous place as the bad guys move ahead with both offensive and defensive applications of AI, which often can’t be separated, while we pledge ourselves to a suicidally slower pace.\n\n# Why a Proposal Which Clearly Can’t Work?\n\nI’m certainly not privy to the motivation or long run game plan of the tech titans, but here’s one guess. They are currently facing two dangers. First a Bernie Sanders – Steve Bannon ban on the work they are doing and/or nationalization of AI companies. Terrible ideas but there is some danger of them being at least partially implemented.\n\nSecond danger is purely economic. Until recently they were raising money at huge valuations based on lofty expectations and localities were tripping over each other in a rush to offer subsidies and tax breaks for data center construction. In other words, they were getting a lot of free money. But, as the rush to build data centers progressed, localities have begun to push back. Not only are they denying subsidies; they are demanding that builders of new data centers pay the full cost of infrastructure they will require and maybe then some (see [The Art of the Data Center Deal](https://fractalsofchange.substack.com/p/215288013)). This increase in costs as well as the difficulty the large AI companies will have ever getting a return on the huge investments they must make to maintain a competitive lead are about to become very clear in the documentation Anthropic and OpenAI must issue in preparation for going public. In other words, the tinkle of free money is becoming a deafening silence. Making things even worse is competition from open-source AI models, which people can run on their own computers, and low-cost AI models from China. OpenAI, Anthropic, and SpaceX can’t charge enough for their AI models to make them profitable because of low-cost competition and they can’t stop spending because they are in competition with each other.\n\nShort-term they want a truce so they won’t have to build out as quickly and they want to fend off proposals which would simply end their businesses. They also want to make it regulatorily impossible for domestic competitors to emerge. But, as they slow down, open source and Chinese models will start to overtake them. If the US tech giants have already established that fast AI development is bad, they will quickly ask for regulation against open source and foreign competitors and they will probably get that regulation. We already don’t allow cheap Chinese electric cars into the country to compete with Tesla. Same thing, right?\n\nBut then, as they know will happen, there won’t be verifiable global non-proliferation. The US will begin to fall behind. That can’t be allowed to happen so now they will need government help to advance the cartel’s work on AI domestically. Federal mandates that data centers be allowed. Subsidies. Grants. Bailouts.\n\nThis may be good strategy for the tech companies. It’s a terrible strategy for our country.\n\n# What Should be Done about AI Dangers?\n\nAI is a tool which, like a knife or a crowbar, can be and is used for both good and evil purposes. The same algorithms which can find new vaccines can design new dangerous pathogens. The ability to detect vulnerabilities in systems will be used both to strengthen systems and to attack them. If we don’t build the world’s most capable AI, we will be at the mercy of whoever does. Musk, Amodei, and Altman like to talk about “alignment”, making sure that AI systems have the values that we want them to have so they don’t do things we don’t want them to do. Bull! Tools can have safety features but they don’t have values. Bad people will use tools to do bad things. Defunding AI development is as poor an idea as defunding the police because some police are abusive.\n\nMaking the world safer requires both that we build the best AI tools humans (now aided by AI) can build and that we do what we can to to ensure that good uses outweigh and sometimes counteract the bad. Even before AI, it was possible to make bioweapons. Regardless of the actual origin of the COVID virus, there is little disagreement that it could have been designed in a lab even in those days just before generally available AI. Computer systems and networks have been hacked from the time they were invented. Our power and other utility grids have been dangerously fragile for years. Our government systems are porous. One positive action that AI giants can take is to fund a White Hat Institute (WHI) and provide it with the latest models prerelease so that the WHI can use these models both to build stronger systems and to find where existing systems need to be patched. This would be something like an Underwriters Lab but its purpose is to strengthen defenses rather than certify products. WHI’s purpose is not to “pace” development but to speed benefits. It would be appropriate for government to participate in the WHI and to help with its security and benefit from its learnings. Workers in the WHI may well require security clearances.\n\nAnother step government can and should take is to outlaw the payment of ransom to hackers by corporations and institutions like hospitals. Much of the motivation for hacking and much of the funding for it even when it is done by governments comes from ransom payments. If such payments are impossible for corporate or public institutions to make, the motivation for many attacks and the funding to develop attack weapons is diminished.\n\nThe dangers of AI are intensifications of threats we faced before AI. The potential benefits of AI are huge and literally can’t wait. Unilateral and unverifiable “pacing” (aka slowing down) of AI development by allowing cartelization may serve the short-term interests of OpenAI, Anthropic, and xAI; it would hurt the rest of us.", "url": "https://wpnews.pro/news/what-s-scarier-than-agents-taking-over-internet-ceo-cartel-trying-take-over-ai", "canonical_source": "https://fractalsofchange.substack.com/p/q-whats-scarier-than-a-swarm-of-ai", "published_at": "2026-09-17 00:44:51+00:00", "updated_at": "2026-09-17 00:54:09.746776+00:00", "lang": "en", "topics": ["ai-policy", "ai-safety", "ai-startups"], "entities": ["Dario Amodei", "Anthropic", "OpenAI", "Sam Altman", "Elon Musk", "xAI", "SpaceX", "Tesla"], "alternates": {"html": "https://wpnews.pro/news/what-s-scarier-than-agents-taking-over-internet-ceo-cartel-trying-take-over-ai", "markdown": "https://wpnews.pro/news/what-s-scarier-than-agents-taking-over-internet-ceo-cartel-trying-take-over-ai.md", "text": "https://wpnews.pro/news/what-s-scarier-than-agents-taking-over-internet-ceo-cartel-trying-take-over-ai.txt", "jsonld": "https://wpnews.pro/news/what-s-scarier-than-agents-taking-over-internet-ceo-cartel-trying-take-over-ai.jsonld"}}