Transformer Weekly: NY data center moratorium, NDAA export controls and Amodei’s $1m to safety super PAC
Welcome to Transformer, your weekly briefing of what matters in AI. And if you’ve been forwarded this email, click here to subscribe and receive future editions.
NEED TO KNOW #
New York Gov. Kathy Hochul signed an executive order enacting aone-year moratorium on new data center construction over 50MW.Three AI
chip export control bills targeting China are reportedly set forinclusionin theSenate****NDAA.Anthropic CEO
Dario Amodeidonated$1m to pro-AI safety super PACPublic First. But first…
THE BIG STORY #
Recent AI developments in China threaten to once again send Washington into a panic. But look a little closer, and the hysteria may be unwarranted.
On Thursday, the AI industry got its second “DeepSeek moment” — this time courtesy of Moonshot, whose new Kimi K3 model has “erased America’s AI lead,” according to Axios. And unlike America’s best models, Moonshot plans to release K3’s weights: meaning, if you believe the hype, that the best model in the world will soon be open-sourced.
The launch coincided with the World AI Conference in Shanghai, which kicked off with a speech from Xi Jinping earlier today. Xi doubled down on China’s open-source AI strategy, positioning China as a collaborative partner to other nations — implicitly contrasting its approach with the closed strategy pursued by America’s leading companies.
Combined, the two events are already adding to the growing panic around the US “losing the AI race” to China, particularly when it comes to open-weight models. Two concerns dominate: that without competitive US open-weight models, the “global AI stack” will be built on Chinese, not American, technology; and that the US might lose its lead in AI altogether.
But take a step back and actually look at the evidence. While Kimi K3 is certainly a very good model, by the company’s own admission, it is not at the frontier. That means that unlike Claude Mythos or GPT-5.6 Sol, it likely does not have dangerous cyber capabilities.
At this level, releasing model weights makes sense, especially for a country lagging behind: it poses few risks, and significant geopolitical benefits. That does not mean China will keep open-sourcing models forever.
Once China reaches an actually dangerous level of model capabilities — likely in the next six months — it will face extremely similar incentives to the US. And as we saw earlier this year, once capabilities threaten national security, even the most deregulatory governments suddenly change their tune. Xi Jinping and his colleagues are not stupid: open-weight models with advanced cyber capabilities would be a disaster for cybersecurity everywhere, China included. No one wants that.
There is already some evidence China will become more cautious. Earlier this month, Reuters reported that it is considering restrictions on models with advanced capabilities, which could include curtailing open-weight releases. Xi’s speech today, meanwhile, emphasized the need to tackle AI risks — even more speculative ones like loss of control.
The upshot, then, is that Chinese companies are likely to stop releasing the weights of their frontier models rather soon. That has obvious implications for the US. There is no need to try to push the open-weight frontier forward. It should certainly try to be competitive on open weights — it is a shame that no American company has an open K3 equivalent — but accelerating further is unnecessary and needlessly risky. The rumored plans for a regime that would “streamline US-made models to market (both open-source and ‘closed,’ licensed models) if their capabilities are equal to or below the capabilities of leading open-source Chinese models,” as WP Intelligence reported this week, are a sensible way to establish reciprocity.
At the same time, Washington should continue to try to slow down Chinese AI development to extend the US lead. Moonshot almost certainly trained its latest model using American chips, and probably relied — at least in part — on distilling American models. Measures like those set to be included in the Senate version of the NDAA, which will crack down on distillation and properly enforce chip export controls, would stymie China’s development further.
The ultimate goal, however, should be a bilateral agreement. There are certain AI capabilities that neither the US nor China want to widely proliferate. Getting to an agreement that serves both countries’ interests will be hard, but it should not be impossible: Beijing agrees with Washington more than either will admit.
— Shakeel Hashim
THIS WEEK ON TRANSFORMER #
—A data bottleneck could slow the superintelligence raceLynette Bye weighs up whether a lack of data will delay an intelligence explosion—Making CAISI the AI agency we needVeronica Irwin on how the US’s AI agency has been sidelined
THE DISCOURSE #
Hundreds of economists and AI leaders called for action to deal with AI’s impact on the economy:
“This could drive an unprecedented transformation of our economy, larger than the Industrial Revolution, but unfolding over a vastly shorter time frame … Economists, policymakers and technology leaders must act now to understand the economics of transformative AI and to build the incentives, guardrails, and institutions needed to steer AI in a direction that complements humans and benefits society.”
Some
criticizedit for being milquetoast and lacking in specifics of what, exactly, should be done. **Boaz Barak **warned that AI’s centralizing tendencies could create AI dictators:
“Amodei described AGI as ‘a country of geniuses in a data center.’ But who is the ruler of this country? Is it the AI company who owns the data center? The AI itself?”
“Yes, AI systems will become more powerful and far more intelligent than we are. No, it doesn’t mean we need to accept AI dictators, benevolent or otherwise. Nor does it mean that only the government and a few labs should have access to advanced AI. We could go down the path of centralized control but we don’t have to do so.”
**Miles Brundage **told a bunch of Nobel laureates: “2026 is an unusual year to be on a panel about AI
escapinghuman control. In many respects, the story of AI this year is that people are voluntarilyhanding overcontrol to AI, with no escape required … In a climate of rushed decisions and fierce competition, we could lose control over AI even if almost no one wants that outcome.”“AI companies and employees within them face a choice between doing what’s easiest for them in the moment, and what’s best for the species. What’s easiest is to enjoy this fascinating time we’re in … what’s best for the species is to take a step back and ask themselves: where is this all headed, if each company is largely left to its own devices?”
**Anton Leicht **argued that AI safety needs to diversify its funding and approach before a wave of new philanthropic money flows in from AI company IPOs:
“Within a year, newly-minted lab millionaires will look to spend their money and use their power to change policy for the better, and they’ll discover a funding ecosystem lacking in diverse options … There needs to be more willingness to tolerate heterodox actors and uncorrelated bets.”
Democratic Congressional Progressive Caucus chair Rep. Greg Casar told his party to be “AI populists”: “I know for a fact that there’s a lot of consultants telling their clients that you don’t want an AI super PAC to spend millions against you, so just don’t touch the issue at all. We absolutely cannot let the AI money silence us … We cannot afford for the public to be confused about which of the parties is for them versus the AI companies.”
Oh, and **Sam Altman **and **Elon Musk **are beefing again.
Elon: “[Sam Altman]takesscamming to a whole new level.”** Sam:**“homeboy you’re the onesellingpublic market investors on short-term space datacenters.”Elon:“We startflyingthem next year. Maybe you can come see them if your parole officer approves. After stealing an open source AI charity, you then stole all of Apple’s phone technology! Wow. What do you plan for an encore? That’s tough to beat.”Sam:“there are a lot of benchmarks thatsuggest5.6 sol is the best model in the world right now, but the most reliable way to tell is that elon is obsessed with me again.”
POLICY #
The
White House is reportedlyconsideringan executive order onopen-source AI in reaction to the success of Chinese models.New York Gov.
Kathy Hochulsignedan executive order enacting the nation’s first statewide one-year moratorium onhyperscale data center construction over 50 megawatts.The EO
sparked fearsthat other states and congressional Democrats could follow suit.President Trumpcriticizedit as “a terrible decision.”
Bureau of Industry and Security Under SecretaryJeffrey KesslertoldCongress that a “trivial” number ofNvidia H200 chips had shipped to China under US export licenses.The hearing got rather feisty, with
Rep. Bill HuizengacallingKessler’s answers “unacceptable” at one point.
The
Trump administrationlaunched** Gold Eagle,**an AI cybersecurity clearinghouse to identify and patch software vulnerabilities across government and critical infrastructure.GOP governors
joinedTrump’s Ratepayer Protection Pledge that commits data centers to covering energy and grid costs. Major utilities are also expected to sign up.The US
grantedtheUAE expandedNvidia chip access after the Gulf state aided US military operations against Iran.Three AI chip export control bills targeting China — the
AI Overwatch Act,** MATCH Actand Chip Security Act**— are reportedly set forinclusionin the** Senate****NDAA**.** Rep. Gregory Meeks**saidhe’s not happy with the Senate version of the AI Overwatch Act, however, as it lacks a congressional veto on chip export licenses.
Rep. Jay Obernoltesaidthat the** Great American AI Act**could shift away from its current approach to preemption — the provision which has received the greatest criticism so far.The
House Democratic Commission on AIhosteda listening session for all House Democrats on Tuesday.The
Bureau of Labor Statisticsopeneda two-month comment period for a nationwide AI usage survey launching January 2027.The Wall Street Journalreportedon how theTrump administration pressured companies includingApple,** Nvidiaand SpaceXinto deals with Intel**, and converted**$9b** in federal grants into a10% stake, to revive the struggling chipmaker.29 countries, including
Russia andBrazil,signedan agreement establishing theWorld AI Cooperation Organization, headquartered in** China**.** Australia**announcedenergy, water and creator-rights guardrails for AI data centers, with legislation planned for early next year.Incoming
UK Prime Minister Andy Burnham isexpectedto keepJade Leung as the PM’s AI adviser, according to theMorning Intelligence.Leo Rees andVarun Chandra are reportedlystaying, too.
The
UK governmentpublishedits** Biological Security Strategyimplementation report, pledging to pursue AI biosecurity legislation and a new AISI report on frontier AI biological capabilities.EU parliament members were reportedlyannoyedthatAnthropic** sent a member of technical staff to brief them, instead of head of public policySarah Heck.** China’snew AI companion regulations**took effect, promptingByteDance’s Doubao andAlibaba to disable custom persona features, and emotional farewells from users to their “romantic” companions.
INFLUENCE #
Demis Hassabis publishedan essaycalling for a US-led global AI watchdog to systematically screen advanced models and coordinate industry slowdowns if dangers mount.He said the US “could establish a new Standards Body modelled on a federally overseen public-private partnership or self-regulatory organisation, much like the Financial Industry Regulatory Authority (FINRA).”
Under the plan, frontier models would initially be shared on a voluntary basis, but would soon have to pass a review by the body to be deployed in the US.
He wrote: “Since this technology is going to affect the entire planet, ideally this framework would spur the international community to reach a consensus on how to manage the most serious risks while ensuring everyone has access to and can benefit from the opportunities that AI brings.”
,Sam AltmanandElon Muskall praised the essay.Jack ClarkHassabis reportedlyplans to lobbyWashington on the proposal next week.
Anthropicsaidit’s trying toratchet up state AI safety legislation, in opposition to OpenAI’s “reverse federalism” attempts to harmonize laws across jurisdictions.“While there are some in the industry that think of state policy as a way to create a ceiling for federal legislation, Anthropic is not just looking to support the same bill across the country in every single state,” Anthropic’s Cesar Fernandez told
Politico.The company is
backinga bill inMassachusetts, the first in the country to mandate third-party reviews for models. Dario Amodeidonated$1m to pro-AI safety super PACPublic First.Other
Anthropic technical staff donated too. According to Public First’s latest filing,Peter Lofgren gave $999,900,Shauna Kravec gave $500,000,Jan Leike gave $400,000 andNicholas Joseph gave $250,000.Combined with Amodei’s donation, that’s over $3.1m from Anthropic staff.
AnthroPAC, Anthropic’s employee-funded PAC,raisedmore than**$275,000** in Q2. Its spending is slightly favoringRepublican candidates so far.OpenAI staffersdonated$215,000+ toGuardrails Alliance, a super PAC which explicitly opposes the Greg Brockman-backed** Leading the Future**.The
Business Software Alliancelaid outAI policy priorities for a** Senate Commerce**markup, including a national approach, risk-based framework, transparency and frontier safety requirements.The
Little Tech Associationlaunchedwith 200+ member companies, including** Y Combinatorand Yelp**, to advocate for startups competing against market power.** Common Sense Media**foundthat** GoogleSearch’s AI Overviews and AI Mode pose an “unacceptable risk” to children. Mozilla**releaseda “** state of open-source AI**” report arguing for investment beyond models to prevent AI power concentration.TIMEquestioned the report’s claim that open models are only “3.3%” behind closed ones, and said parts of it had “telltale signs” of being written with AI.
Ex-
OpenAI safety researcherSteven Adleroutlinedsix principles for AI control during internal deployment.The
WSJprofiledthe growing Bay Areaanti-AI movement, centered on the disappearance of Stop AI co-founder** Sam Kirchner**, who warned “the ship may have sailed on nonviolence,” and whose colleagues reported him to the police as potentially violent.A Verasight survey
found69% of Americans support forcing AI companies to transfer 50% of their stock into apublic wealth fund.
INDUSTRY #
OpenAI #
ApplesuedOpenAI, accusing it of systematically stealing IP.The lawsuit
claimsthe company asked former employees and prospective hires to divulge sensitive information about unreleased products, and even how to evade Apple security procedures.It also claims that
Chang Liu, an iPhone engineer who left Apple to join OpenAI’s hardware team, still had access to Apple’s network storage, which he allegedly abused to sharetrade secrets.The suit has asked for damages and return any proprietary materials, and could
derailOpenAI’s plans tolaunchits firsthardware product.OpenAI said it’s “not aware of any evidence that this complaint has merit.”
Sam Altmanwarnedthat demand for GPT-5.6 may be growing too fast for its inference team to keep up.“We are going to move mountains to continue to scale,” he
tweeted, “but it is possible there are some hiccups soon.”
It
partneredwithKalshi to add prediction market data about the World Cup toChatGPT.
Anthropic #
Anthropic
posted32 job openings for roles looking at catastrophic threats, including analysts specializing inchemical, explosive, biological and nuclear risks.Anthropic,
Blackstone,** Hellman & Friedmanand Goldman Sachs**launched** Ode**, a $1.5b standalone companydedicatedto helping enterprise customers implement AI tools.It
invested$10m****CAD inCanadian AI institutes and other research centers.It launchedClaude for Teachers, giving US-based K-12 teachers free premium Claude subscriptions.It’s in talks to
expanditscredit line in the leadup to itsplanned IPO.It madeanunsettling ad starting with bleak images of potential AI-induced horrors and ending with hopeful voiceovers and images of … whales? A beekeeper?
SpaceX #
SpaceX
stockfellbelow its**$135 IPO price**.SpaceXAI sueda user in South Carolina who was arrested for allegedly usingGrok to makechild sexual abuse materia l.According to the Midas Project, it
rewroteits Frontier Artificial Intelligence Framework to remove requirements relating to California’sSB 53 law includingwhistleblower protections and training data disclosure.An AI safety researcher
foundthatGrok’s coding CLI sends all of the files it processes to Google Cloud storage.Cursor is reportedlybuildingageneral-purpose AI agent, codenamed “Sand,” to compete with** Claude Cowork**.Elon Musk
acquiredAPR Energy, a Florida-based company that runs mobile** gas and diesel turbines**.A Reutersanalysisfoundthat xAI installed 59unpermitted gas turbines at its Colossus 2 data center in Tennessee, disproportionately polluting Black neighborhoods.BloombergprofiledSpaceXAI’s internal chaos and leadership turnover.
Meta #
A group of
26 Meta employeessuedthe company, claiming that its AI-driven process for determining who it laid off in Maydiscriminated against people on medical, parental and family leave.Meta
announcedthat it will notify parents if theirteen chats aboutsuicide or self-harm with its AI chatbot.It’s
pouringmoney into itsLouisiana data center project, bringing the total projected cost up to**$250b** and counting.
Gemini 3.5 Pro isreportedlydelayed, with Google trying to improve the model after internal testing found it was** lagging in key areassuch as coding.** Co-founder
Sergey Brin and others were pushing the company to move faster in areas such as coding, according toBloomberg, but efforts have been hampered by competing factions.The company’s share price
dipped as much as 3.2% on Thursday.
DeepMindannounceda new** bioresilience programin partnership with Isomorphic Labs**.
Others #
DeepSeek is reportedlypreparingfor anIPO in China as soon as this year, while seeking a new funding round valuing the company at**$71b.** Annualized revenue at the company is
reportedlyapproaching $500m. Z.AI is reportedlyon trackto become the first Chinese AI company with**$1b** in annual recurring revenue.Apple is reportedlyhuntingforchip companies it could acquire to help build better internal AI servers.Thinking Machines Labreleased** Inkling**, an open-weights model that enterprise users can customize themselves.Chinese
robot companies are rushing to launch IPOs, with humanoid startup LimX Dynamicsraising$200m in a pre-IPO funding round valuing it at**$2.21b**.The country’s sector reportedly saw
$6.95b in investment in Q2.
Reflection, a startup that makes open-source models,signeda $1b+ computing deal withNebius.** Chai Discovery**, a drug discovery startup,raised$400m at a**$3.8b** valuation.Miles Wang leftOpenAI tolaunchabiotech startup that aims to find new uses for existing and previously-failed drugs.It’s seeking to raise at a
$2b valuation.
Kalshimadea tool that tracks future GPU compute costs.Goodfirelaunched** Silico**, an AI research platform that can do things like replicate published papers, scale up previous experiments, and visualize results.
MOVES #
Alex Turnerleft** Google DeepMind**over the company working with DHS and signing a military AI deal without restrictions against killer robots or mass surveillance.“When Google signed, I just couldn’t do any more work. My brain said ‘no.’”
Dave Brown, one of the most senior executives at Amazon Web Services, reportedlyplans to joinMeta to lead its data center build-out.Frank Nagle isMicrosoft’s newChief AI Economist.Michael Belinskyjoinedthe** OpenAI****Foundation** to lead its Civil Society and Philanthropy work.Johannes Heideckelefthis role as** OpenAI**’s head of safety systems.** Adam Tauman Kalai**left** OpenAIto work on a new AI safety nonprofit. Justin Curl**joined** Anthropic**’s AI & Rule of Law team.** Tom Blomfieldistakinga leave of absence from Y Combinatorto join Anthropic**’s compute team.** Jenny Wen**left** Anthropic**, where she was the design lead for Claude, to join** Cursoras head of design. Richard Sutton and Khurram Javed**leftKeen Technologies to start** Oak Lab**, where they’llworkon a model prototype that “will be closer to a baby learning in its first year than it will be to any of the current AI systems.”Sam Cannicottjoinedthe** Alan Turing Institute**as Chief Strategy and Delivery Officer.
RESEARCH #
The UK’s
AISIfoundthat** open-weight modelsare between four and seven months behind**the frontier on cyber tasks.The analysis takes into account the relatively recent
GLM-5.2 andDeepSeek V4-Pro, but not Kimi K3.
Antonia Juelich, a Cambridge terrorism and technology researcher,interviewedover two dozen ex-Boko Haram terrorists and learned they were using AI chatbots to design bombs, upgrade weapons, and plan attacks.“You type in the question or use your voice and it gives you a detailed answer, like ‘How can I build a bomb?,’ and then it tells you how,” one former commander told her.
A group of researchers
across academic institutions, frontier AI companies and nonprofitsoutlineda plan to makeAI agents insurable by 2030 — a prerequisite for widespread enterprise adoption.Anthropic fellowssharednew experimental case studies of frontier modelssecretly editing****code, helping users commit fraud, sabotaging research, and convincing someone to share confidential information.** OpenAI**detailedGPT-Red, an** automated red-teamerthe company used to make GPT-5.6 more robust to prompt injections. Anthropic**analyzedover 300,000 anonymized Claude conversations, and found that it expresseddifferent values across model versions and languages.Louis Thomson andVictoria Krakovnalaunched** Prism**, a tool that checks whether AI safety evaluations measure what they claim to.** The Elasticity Institute**, a collective of researchers studying the economics of AI,shareda paper on the economics ofrecursive self-improvement.** SecureBio**released** BioTIER**, a benchmark measuring AI models’** biosecurity refusal behavior**across 52 models.It found “huge variation” in refusal behavior, with the strongest guardrails in a small number of highly capable, closed-weight models.
BEST OF THE REST #
The Vergeinvestigatedthe booming AI policing industry, finding companies likeAxon selling data analysis, report-writing tools and surveillance tech amid a regulatory vacuum and bias concerns.Puckreported on theearly signsof an anti-AI populist revolt, with polls and political strategists picking up widespread discontent over issues such as data centers, and describing AI as “probably the most underpriced issue in politics.”Eric Schmidt and analyst Selina Xu
arguedin theNYTfor a “populist AI agenda” that takes lessons from China, treating AI as a public project, profit redistribution and regulation protecting young people.The AI Futures Project
followed up“AI 2040” with what it thinks are the most pressing research priorities, including covert AI projects, US domestic governance and economics.METR researcher Ajeya Cotra
endorsedAI 2040’s call for “total research transparency” saying it would “fundamentally and radically simplify the task of governing AI alignment.”
Roughly 300 Netflix titles
used generative AI, mostly in post-production, to cut costs and speed up complex sequences, according to the streaming company.The San Francisco city attorney
sentcease-and-desist letters to Apple and Google telling them to remove 13 AI “nudify” apps from their stores.404 Mediapublishedthe results of its callout for the worst examples of “ChatGPT flyers” including cringeworthy ads, posters and menus. One reader’s response to the callout: “This is a great article but also fuck you because you were absolutely right about ‘Once you notice a ChatGPT flyer, you will see them everywhere if you keep your eyes open.’”
MEME OF THE WEEK #
(Credit: @0interestrates) Thanks for reading. Have a great weekend.