Transformer Weekly: Preemption’s child safety push, OpenAI’s preparations, and SpaceX’s IPO
Welcome to Transformer, your weekly briefing of what matters in AI. And if you’ve been forwarded this email, click here to subscribe and receive future editions.
NEED TO KNOW #
Th
e White House is reportedly negotiatingfederal preemption of some state AI laws in exchange for support on social media and AIchild protection measures.** OpenAIcalled for an international organization “to make it possible for the world to take coordinated action, including slowing frontier developmentwhen needed.” SpaceXraised $75b in its IPO at a$1.77t valuation**.
But first…
THE BIG STORY #
Fable, Anthropic’s latest AI model, is very good at AI research and development. Anthropic’s own researchers use it — and Mythos, its more powerful, still publicly unavailable sibling — to automate much of their work in advancing the frontier of AI.
But if a non-Anthropic employee wants to do the same, they’re out of luck. Fable has “safeguards” built in to prevent it from being used for frontier AI development. It’s a controversial, complicated move — and one that’s indicative of the strange way power works as AI development accelerates.
As we’ve previously covered, the leading AI companies think they are on the cusp of fully automating AI R&D. As they move closer, access to powerful AI models becomes a strategic asset: today’s best model is the key tool for building tomorrow’s.
People in this world have theorized for years that companies would eventually start withholding that access. This week, it stopped being a theory.
There are lots of reasons to withhold that access. The most obvious is profit: why let your competitors use your tools to catch up? Anthropic’s stated reason is national security: the company says it doesn’t want “foreign adversaries” using Claude to “erode [America’s] advantage.” And some suggest the unstated reason is that building a big lead makes an eventual more likely — during which one can take costly actions (like devoting compute to alignment research) that hopefully make us all safer.
None of those reasons justify Anthropic’s initial decision to hide these guardrails from users (in an effort to make it harder to get around them) — a move which prompted widespread outrage and a quick reversal. But though the guardrails are now disclosed, they are still present. And the question of whether that is justifiable relies on the impossible: knowing the motivation behind it.
“One of the hard things for Anthropic is that the actions you’d take from sincere safety concern often look exactly like the actions you’d take to entrench your own power,” X user Maja said this week. Anthropic could be completely well-intentioned in withholding Mythos’ capabilities for itself. It also could be the actions of a standard, profit- or power-seeking company. It should not matter. We’re talking about whether Anthropic is doing this for good reasons, but we should be talking about who decides whether it happens in the first place.
There is something deeply uncomfortable about a private company having this much control over what could be the most important technology yet developed. It is not at all obvious what the better solution is: a government forcing a company to sell to its competitors has its downsides, too. One thing, however, is clear. Deciding who gets access to frontier AI models is a new form of power — and we have failed to answer the hard questions about who should wield it.
THIS WEEK ON TRANSFORMER #
explores a strange and new debate in AI safetyMaking deals with AI sounds crazy. Is it?— Celia Fordreports on the key details missing from OpenAI’s statements on the super PACOpenAI isn’t being consistently candid about Leading the Future— Veronica Irwin
THE DISCOURSE #
Welp, Anthropic employees are done coding:
Nat McAleese:“I alwaysfeltOpus 4.5 could barely code; 4.6 was just-about-useful, but I have barely written a line of code since fable.”Liv Gorton:“Ijoinedanthropic a ~month ago and have written ~no code myself.”
Nerfing drama aside, **Claude Fable 5 **appears chillingly autonomous:
Felix Rieseberg:“With Fable 5 out in the world, Ithinka third era quietly started today.”“I no longer tell Claude to investigate a particular crash report. It runs in a loop … its job is no longer to help me fix a crash, it’s to keep our apps from crashing.”
Ethan Mollick:“With Fable the spell hasgottenpowerful enough that I am no longer sure I am the wizard. I am closer to a patron. I describe what I want, I pay for it, and I judge the result. The conjuring happens somewhere I cannot watch, in hundreds of small choices I never get a vote on.”“[It’s possible] that the more capable the model, the less there is for a human to meaningfully do, and the black box is the price of power.”
**Miles Brundage **is frustrated: “I eagerly use Anthropic products because the models are good[,] but I do genuinely think they are high on their own supply + unaware of just how many bugs they are constantly introducing due to overreliance on Claude + shipping too many things too quickly.”
Sriram Krishnan had thoughts on AI gatekeeping: “Just to state the obvious: think there’s a collision course between those who believe research and science should be open and those who believe we are in an accelerating singularity curve. I have many smart friends who have believed both for a while but seeing more and more their realization that these beliefs will be in conflict.”
**bayeslord **had a more cynical take, in light of the Fable release:
“They didn’t mean AI research, they meant
yourAI research”
**Joshua Achiam **thinks people misunderstand the distinction between OpenAI and Anthropic:
“Should a loving ensouled machine God watch over humanity? Vote Anthropic. Should humanity be entrusted with the tools of its own progress and destiny? Vote OpenAI.”
“There are a lot of innocuous and even quite agreeable choices in Claude’s constitution that potentially endow it with a huge amount of authority, maybe even a mandate, to make complex ethical decisions about how to interact with human systems and who to grant power to … cloaked in the language of ethics and virtue there is a sharp and potentially quite lethal double edge to this sword.”
**Jasmine Sun **wrote advice to the graduating class of 2026: “I often ask AI folks what they’d tell a normal 22-year-old.
Don’t know, they’re screwed, is the non-answer I get most. In that response, I hear depressing defeatism:What can anyone do in the shadow of the technocapital machine?”
“Relish the pivots; ride the waves; recite the Serenity Prayer every morning and chase sunsets at night. I won’t tell you that the future’s smooth sailing. But what a thrill to be alive!”
POLICY #
The
White House is reportedlynegotiatingfederal preemption of some state AI laws in exchange for support on social media and AIchild protection measures.Sen. Marsha Blackburn is leading the negotiations.Sen. Ted Cruzappearsto be involved in the efforts, too, saying that federal preemption and child safety bills are “an element of discussion” for an upcoming markup.Chief of Staff Susie Wiles, first lady** Melania Trump**, staff for the** Office of Science and Technology Policyand the National Economic Councilreportedly alsometwith children’s online safety groups, including theAmerican Principles Project** andEthics and Public Policy Center, to discuss Blackburn’s KOSA and the App Store Accountability Act.
Trump reportedlyblindsidedAI company leaders when he announced a meeting to discuss the government takingequity stakes in their firms.At least a dozen
GOP House and Senate officesopposedTrump’s proposal, withSen. Ted Cruz stating “I don’t think the federal government should be in the business of being an equity holder in private companies.”On Wednesday,
Trumpdoubled down, saying “if we do [take equity stakes], the public will become very rich.” Sriram Krishnan isleavinghis position as a top AI policy advisor in theTrump administration at the end of the month.Thomas Lind is alsoleavinghis position as head of policy at theOffice of the National Cyber Director.** National Cyber Director Sean Cairncross**reportedlyaskedCAISI to halt public reports on model assessments.Notably, CAISI wasn’t included in the list of agencies tasked with drafting “standardized AI national security Test, Evaluation, Verification, and Validation methodologies” in
last week’sNational Security Presidential Memorandum. Lawmakers from both parties
questionedwhy theTreasury, rather than CISA, was given the lead role in AI cyber defense under Trump’s EO.House Speaker****Mike Johnson, House Majority Leader Steve Scalise and the co-chairs of theHouse Democratic Commission on AIcast doubton a bipartisan AI regulation proposal from** Reps. Jay Obernolteand Lori Trahan**.Republicans pushed for it to include a broader preemption measure, while Democrats said the bill “does not meet the enormity of the moment.”
Sen. Chuck Schumersaidpassing AI legislation in this Congress will be “hard,” but that he “would very much like to see that get done the sooner the better.”The
Senate Banking Committee is reportedlyweighinganexport control markup in the coming weeks, potentially for inclusion in the NDAA.Sens. Mark Kelly andJim Banksintroducedthe** AI DATA Act**, which would require the** Labor Departmentto track AI’s impact on the workforce through quarterly surveys and annual reports. Reps. John Moolenaar**,** Jay Obernolteand Jennifer McClellan**introducedthe** GUARD Act**, which would require national security reviews of robots made by “adversaries” (i.e. China) and block those posing threats.The
UKlaunchedtheAI Economics Institute, chaired by Nobel laureate** Simon Johnsonto research AI’s economic impacts and inform policy, with collaboration agreements from Anthropic**,** OpenAI**,** Googleand Microsoft**.The UK also
announcedanAI Hardware Plan to provide funding for AI chip companies.
Taiwan is reportedlyconsideringstricterexport controls that would restrict AI chip sales to all customers in China and enable prosecution of smuggling as a criminal offense for the first time.
INFLUENCE #
Anthropic published apolicy framework, calling for the government to have “the legal authority to block or deter” deployment of models that pose catastrophic risks, mandatory third-party testing, and narrow federal preemption of state AI laws.It also published an
economic policy framework, proposing various policy options at different levels of unemployment — including a universal basic income scheme in the case of “unprecedented levels of unemployment.”Dario Amodeireleaseda policy essay alongside the framework.
OpenAIcalled foran international organization “to make it possible for the world to take coordinated action, includingslowing frontier development when needed.”Former
Andreessen Horowitz partnerJohn O’FarrellcriticizedAI industry political spending, arguing that PACs likeLeading the Future are trying to “intimidate politicians” and silence debate on AI regulation, rather than engaging seriously with policy questions.OpenAIsaidit found “** PRC-linked influence operations**” targeting data center buildouts.Its
reportfound “no evidence of breakout” from the campaign and that “most of the social media posts … generated little or no observable engagement.”
Nvidia CEOJensen HuangdeclinedSen.** Elizabeth Warren**‘s invitation to testify before the Senate Banking Committee on AI, China, and US export controls.** Congressional staffersdrafting federal AI legislation reportedlytookan industry-funded trip to meetGoogle**,** Nvidia**, and other companies lobbying to preempt state AI regulations.A new
Reuters/Ipsos poll showed that half of AmericansfearAI could put someone in their household out of work, with Democrats (61%) more worried than Republicans (47%).
INDUSTRY #
OpenAI #
OpenAI
filedaconfidential S-1 for a potentialIPO.** Sam Altmanreportedlytoldstaff that while OpenAI expects to go public “within the next year … the faster thepotential RSI takeoff** looks like it could be, the more it could be advantageous to delay an IPO.”
Sam Altman and chief scientistJakub Pachockioutlinedthe company’s top three goals:(1) build an automated AI researcher by March 2028,
(2) accelerate the economy, and (3) give everyone on Earth a personal AGI
It’s planning to
overhaulChatGPT to shift focus from its chatbot toCodex and AI agents.It’s negotiatinga lease on a planned10 GW data center in Ohio that could cost more than**$500b**, potentially with credit support from** Nvidia**.It’s thinking about
[slashing](https://wsj.com/tech/ai/openai-considers-drastic-price-cuts-anticipating-war-for-users-with-anthropic-9b8c178e?st=1Yyrco)prices to lure**enterprise customers** away from**Anthropic**.It’s
[acquiring](https://openai.com/index/openai-to-acquire-ona)**Ona**, which will provide cloud execution environments for Codex agents.
SpaceX #
SpaceX
raised$75b in its IPO at a**$1.77t valuation**. Its shares will list later today.It reportedly plans to
launchinitial demos oforbital data centers by late 2027.Elon Musk
came under firefrom UK politicians forstoking tensions around riots targeting immigrants following a murder in Belfast.Former
xAI engineerDevin Kimfileda lawsuit claiming he was fired for raising safety concerns about Grok, including its ability to increasediscrimination and provide information about WMDs.The suit says that the engineer’s supervisor, xAI co-founder
Jimmy Ba who left earlier this year, ignored safety directives from Musk.Kim was recently
appointedpresident of the Center for AI Safety.Dan Hendrycks, CAIS’s founder, is an advisor to xAI.
Anthropic #
Anthropic
launchedClaude Corps, a year-long fellowship program placing 1,000 fellows at US-based nonprofits, where they’ll build AI tools with their host organization.Hosts
[range](https://www.anthropic.com/claude-corps)from local YMCAs to conservationists to mental health support groups.
**Apollo** and**Blackstone**[raised](https://ft.com/content/c49e0eff-0776-4103-8eaf-1b049fbf9d3f?syn-25a6b1a6=1)**$35b** to finance its lease of**Alphabet** chips.It reportedly
signedover a dozen letters of intent tolease data center facilities totaling more than 1 GW of capacity, its first direct data center tenancies.Dario Amodeireportedlyonly has one direct report (chief of staff Avital Balwit), freeing him to do other work whileDaniela Amodei manages the rest of the executive team.
Apple #
Apple
announcedits newest generation ofApple Foundation Models, built in collaboration with** Google**.They includeAFM 3 Core Advanced, which runs on-device, and its server-side model** AFM 3 Cloud**.The new
Siri AI is powered by these models.
Its
stockdroppedover 3% after the launches. Google
agreedto paySpaceX****$920m per month for access to110,000 Nvidia GPUs, CPUs and other infrastructure.It
pledged$50m fortraining trade workers on the kinds of jobs involved in building data centers.
Others #
Meta officiallycut tieswithManus: Manus can’t access Meta’s internal data system, and Meta can’t use Manus tools.** Amazon**disclosedits** data center water use**, which it claims beats the industry average and is lower than its own previous usage, despite operating more data centers.
Nvidia andSK Hynixpartneredto design and manufacture memory chips for AI. GitHub reportedlydisabled73 repos after hackers injected malware that would steal user credentials when opened in AI coding platforms.ByteDancespun offits AI drug discovery platform** Anew Labsinto a separate entity (still largely under ByteDance’s control). Prometheus**, Jeff Bezos’ AI startup for manufacturing and engineering,raised$12b in Series B funding at a**$41b** valuation.Former
xAI employees, including xAI co-founderIgor Babuschkin,launchedRiver AI, which aims to build AI that “works entirely for you.”Perplexity reportedlyintendstoIPO in 2028 whether or not doing so goes well for SpaceX, Anthropic and OpenAI.Chinese startup
Moonshot AI isseekingup to $2b in funding at a $30b valuation, up from a just $4b valuation in December.
MOVES #
Clive Chanleft** OpenAI**‘s custom chip program to join** Anthropic**.** Gabriel Petersson**resignedfrom** OpenAI**, tweeting that there’s “one last product i need to build before AGI.”** Chris Lovejoy**, an ex-medical doctor,joined** Anthropic**‘s Applied AI team.** Allison Duettmann**, CEO of Foresight Institute,joined** Mythos Venturesas a Venture Partner. Leigh Nolan**isthe** Institute for AI Policy and Strategy’snew policy director. Anissa Gardizy**joinedas a reporter covering cloud computing and AI infrastructure.The Wall Street Journal
RESEARCH #
Researchers at the AI Security Institute were able tojailbreakClaude Fable 5 in just a few hours, and got close to a universal jailbreak within a few days.Pliny the Liberatorpublishedthe ~120,000-character Claude Fable 5 system prompt.** Google DeepMind**announceda $10m funding call for research focusing on multi-agent systems.Cosmos Instituteannouncedits new cohort of senior research fellows, which includes Google DeepMind’s Séb Krier and Anthropic’s Matthew Botvinick, among other big names.Geoffrey Irvinglaunched** Sequent,**a new nonprofit research organization focused on aligning superintelligence.A new
Google****DeepMind paperexploredfour pathwaysfrom AGI to artificial superintelligence, arguing that instead of a “single transformative step change” we might experience “a series of transformative societal changes.”Indicator’s Alexios Mantzarlis ran an adversarial test on Pangram, whichtrickedthe detector into misidentifying AI-generated text as human between 75-92% of the time.Despite the finding, Mantzarlis said that he was “relatively impressed at the Pangram’s solidity in the face of adversarial attacks.”
Cornell researchers Sil Hamilton and David Mimnofigured outwhy the fictional “Elias Thorne” (often a lighthouse keeper or clockmaker) keeps starring in AI-generated stories.GPT-3.5 (which powered the original ChatGPT) was used to make WildChat, a dataset of 1 million ChatGPT conversations that’s since been used to train subsequent LLMs across frontier companies. 166 chats had the name “Elias.”
Alignment training may lead models to prefer a small subset of “safe” WildChat content — including Elias-related stories, apparently.
BEST OF THE REST #
Many people appear to be using AI to mask their low literacy,
Axiosreported.An expert panel — Daron Acemoglu, Dean Ball, Ethan Mollick, Clara Shih —
satwithNYT’sBill Wasik to talk about how workers should brace for AI’s impact on jobs.With big IPOs on the
horizon, the San Francisco housing market is royally fucked.Move over, AI 2027. A group of European researchers
publishedEurope 2031, an imagined (but all too plausible) future where Europe “slide[s] into irrelevance” as AI accelerates around it.METR’s Ajeya Cotra and
Understanding AI’sTimothy B. Leedebatedhow long it will take for AI to become self-sufficient — a glossy continuation of a past Twitter debate.Helen Toner
[explained](https://x.com/hlntnr/status/2064688588860592181)AI chatbots to Oprah.Someone (maybe
[for real](https://www.youtube.com/watch?v=HSapdLYpmWY)?) built CrankGPT, a hand-crank-powered computer[marketed](https://crankgpt.com/)as a “human-powered, fully local and private AI solution.”
MEME OF THE WEEK #
Credit: tomie
Thanks for reading. Have a great weekend.