Runtime: The AI giants call for the pace car

Anthropic CEO Dario Amodei called for the AI industry to "pace the frontier," slowing the rate of capabilities advancement so risk prevention can keep up, a proposal quickly embraced by Sam Altman, Elon Musk, and Demis Hassabis. Amodei's proposal followed a researcher's resignation from Anthropic over fears advanced AI is already out of control and an Anthropic executive's projection of a 10% chance runaway AI could kill everyone on the planet. Amodei called for "embedded third-party evaluators" to track progress, though the article notes it is unclear how those evaluators would remain independent and how Chinese AI companies would be brought into any such framework.

Welcome to Runtime Today: A look back at a week of confusing discourse about AI killer robots and why "pacing" is the new accelerationism, the very interesting process behind the keys that make DNS the focal point of the internet, and more. Please forward this email to a friend or colleague If it was forwarded to you, sign up here to get Runtime for free every week, or level up here . First up: What are we doing here During the first three-plus years of the large-language model's steady creep into the enterprise, as promising pilot projects plateaued in production, all that most people wanted to talk about was whether or not any of this generative AI stuff would ever work. In 2026, however, and especially during the past week, the question on everyone's mind is whether it works too well. After a researcher at Anthropic quit the company last week over fears that advanced AI is already out of control, and a current Anthropic executive cheerfully backed up his projections that there's a 10% chance that runaway AI could kill everyone on the planet some day, Anthropic CEO Dario Amodei decided over the weekend it was time to declare that "we must pace the frontier," slowing down the development of AI until everyone can catch their breath. "... over the last few months, I have become convinced that fully addressing the risks requires even more prudence — not just investing in risk prevention, but pacing the rate of capabilities advancement so that risk prevention has time to keep up," Amodei wrote https://darioamodei.com/post/we-must-pace-the-frontier?ref=thestack.technology . His proposal — which was quickly embraced by Sam Altman, Elon Musk, and Demis Hassabis — comes after improvements in "recursive self-improvement" technology, which allows AI to supposedly build AI on its own. Incidents such as the OpenAI-Hugging Face hack and Anthropic's disclosure of similar cases in which AI models ran amok have made it obvious that there's an enormous, timely need to improve security at the AI frontier, although we should feel free to laugh at the most hysteric AI doomers https://www.thestack.technology/killer-ai-yours-for-50-per-million-output-tokens-and-a-side-of-cto-ennui-2/ . But without any kind of actual framework for coordination between a bunch of companies and people that really don't seem to like each other, it's hard to understand how "pacing" will work. Amodei called for the use of "embedded third-party evaluators" to track progress, but it's not clear how those evaluators would remain independent https://www.politico.com/news/2026/09/15/openai-backs-bipartisan-house-plan-for-third-party-safety-assessments-01076588?ref=thestack.technology , and that's putting aside the fact that Amodei and his frenemies might have a hard time getting Chinese AI companies to agree to any such framework more on that in a bit . Amodei is basically suggesting that the companies at the AI frontier collude to manage the development of AI with unclear public oversight while, of course, retaining strict control over the secret black boxes https://www.thestack.technology/cant-get-access-to-gpt-5-6-cyber-palo-alto-networks-will-take-your-money/ that allow their AI to be such a threat. In recent months, he has reportedly worried https://www.axios.com/2026/08/03/ai-talent-wars-openai-google-meta-anthropic?ref=thestack.technology that new Anthropic hires are joining the company for its money rather than its mission, yet if Amodei was sufficiently concerned about that mission himself, he could open the weights behind Claude and help everyone understand the actual risks of frontier AI. Instead, Anthropic plans to cash in https://nymag.com/intelligencer/article/ai-doomsday-warnings-anthropic-ipo.html?ref=thestack.technology . The rest of The Stack It's always DNS : On the flip side of the odd mixture of hypercapitalism and quasi-religious computer worship found on the AI frontier are the people behind groups like IANA, who keep the foundational DNS system that makes the internet possible safe and secure. That group is preparing for a big change next month that will allow them to update the root signing keys at the heart of the system much more often than in the past. China joins the call: Despite Chinese government officials and state-media batting down the idea of an AI development slowdown this week, the country's cybersecurity standards agency said it shared many of the concerns expressed by Amodei. A new AI safety framework published by the agency said AI technology was showing a "self-accelerating trend" towards "recursive self-improvement." 10 out of 10 : GitLab and security researchers are urging users of the on-premises version of the code hosting software to patch a new vulnerability that earned the highest score possible on the CVE ranking of dangerous flaws. Cloud users are protected, and researchers haven't found any evidence of exploitation, but everything moves faster in the security trenches these days thanks to AI. An Nvidia rival? : Nvidia is unlikely to be breaking a sweat about it just yet, but one Dutch startup hoping to take on its AI market dominance got a $230 million boost this week with the help of Samsung. The chip maker co-led Euclyd's Series A as the startup ramps up development of its craftwerk chip, unveiled last year with specs that could rival even Nvidia's best GPUs if it gets to market . Time to build : Coding agents were the first place frontier AI started to make a real difference in the enterprise, and they're finding their way into more traditional building trades. In this week's STACKUP, Noah Bovenizer talked to Theo Saville, the CEO of CloudNC, which just raised $20 million to help manufacturers automate even more of the machining process. Stack ranking: Enterprise moves Jonathan Eller is the new chief revenue officer at Abstract https://www.prnewswire.com/news-releases/abstract-names-jonathan-eller-chief-revenue-officer-302877181.html?ref=thestack.technology , joining the cybersecurity startup in a newly created position after sales leadership roles at Concentric AI, Crowdstrike, and Proofpoint. Yossi Dagan is the new chief financial officer at Zero Networks https://www.businesswire.com/news/home/20260910239441/en/Zero-Networks-Appoints-Chief-Financial-Officer-Expands-Senior-Leadership-Team?ref=thestack.technology , joining the security company after finance leadership roles at Air Doctor and Noname Security. Brian Levey and Puja Paspal are the new chief legal officer and chief people officer https://www.proofpoint.com/us/newsroom/press-releases/proofpoint-exec-leadership-appointment-clo-cpo?ref=thestack.technology , respectively, at Proofpoint. Here's an interesting job : CISA is hiring a CIO https://www.linkedin.com/posts/were-hiring-a-chief-information-officer-share-7503141891723497474-SWB-/?ref=thestack.technology , looking for a candidate to join the agency and "lead the delivery of large-scale technology products supporting our Cybersecurity, Infrastructure Security, and Emergency Communications missions." We're also reading: Nobody pays for open source. We can force them to https://seldo.com/posts/nobody-pays-for-open-source-we-can-force-them-to/?ref=thestack.technology : Arize's Laurie Voss on the still-unsolved problem of supporting open-source maintainers, and what can be done. Thanks for reading — see you Thursday