AI governance is coming, with or without Trump

Welcome to Transformer, your weekly briefing of what matters in AI. If you’ve been forwarded this email, click here to subscribe and receive future editions.

Housekeeping: We’ll be off next Friday, back the week after.

NEED TO KNOW

• An OpenAI model hacked into anAustralian government website . OpenAI didn’t tell Australia about the hack for weeks — a situation Australian PMAnthony Albanese called “unacceptable.”
• AI super PAC Leading the Future announced the firstDemocrats it’s spending on in November’s general election, amid growing opposition to the PAC among progressives.
• Donald Trump officially renamed “artificial intelligence” to “super intelligence.”

But first…

THE BIG STORY

At this week’s United Nations General Assembly, two things became clear. One: the Trump administration is pathologically opposed to any sort of global AI regulation or slowdown. Two: it’s probably going to happen anyway, without President Trump’s involvement.

In his UN speech, Trump said the US “totally rejects any attempt to construct a globalist scheme to control … artificial intelligence,” adding “I’m not going to stifle growth of something that will be bigger than the Industrial Revolution.” In remarks to the Security Council, OSTP Director Michael Kratsios went further: “The frontier of intelligence is advancing rapidly. That is not a reason to its further development or to constrain it with new global governance structures.”

But it increasingly looks like this might not be up to them. Frontier AI CEOs are working together to build their own governance structures, and repeatedly say they want to slow down. At the same Security Council meeting, a defiant Sam Altman called for “national and international” AI safety standards. And The Information reported this week that OpenAI, Anthropic and Google plan to launch their own “Standards Authority for Frontier AI,” which would set incident reporting standards, flesh out what voluntary safety commitments actually mean, and “establish qualifications for independent auditors of the models and labs,” among other things.

Meanwhile, other countries are going right on ahead and figuring out what AI standards they want. This week 27 world leaders, including from France, Germany, and Canada, called for international standards to tackle the “serious risks” posed by AI. China, too, appears amenable to global coordination. The country’s UN representative Fu Cong called for “alignment and coordination of AI development strategies, governance and technical standards” and “a consensus-based global governance framework.”

The most likely outcome is a self-regulatory regime for America’s frontier AI companies, with significant input from foreign governments. A slow down will follow: when even Jensen Huang is saying AI companies should not release products if they cannot reliably control them, and shift their investment focus to safety research in the meantime, the writing is on the wall.

In being staunchly opposed to any of this happening at all, the White House is giving up its chance to shape it. The train is hurtling toward President Trump, and all he can do is yell at it to stop. It would be better for his own goals to try steering it instead.

— Shakeel Hashim

THIS WEEK ON TRANSFORMER

• Hacking is the least worrying part of OpenAI’s Australia incident —Shakeel Hashim on the clearest evidence yet that OpenAI still isn’t identifying and disclosing incidents appropriately.
• How to respond to an AI disaster —Duncan Weldon on what we can learn from history about what to do, and what not to do, in an AI-driven crisis.

THE DISCOURSE

Donald Trump isn’t calling it “Artificial Intelligence” anymore:

• “Many people think that the words ‘Artificial Intelligence’ are inaccurate, and very ineloquent, relative to AI.”
• “From this point forward, all United States documents and hopefully the world’s will be changed to use the much more accurate term, ‘super,’ as opposed to ‘artificial’ … in other words, welcome to the new world of Super Intelligence.”
• “Whoever wins ‘SI’ — whoever wins Super Intelligence — wins … and we’re leading now over China by a lot, and everyone else. And I’m going to keep it that way.”

Jensen Huang doesn’t buy “doomsday narratives”:

• “2030 is not going to be the end of the world. There is 0% chance that’s going to be the end of the world … Scaring people is unnecessary. It is irresponsible.”
• “If your product does harm, if your product doesn’t perform as you promised, and if you entered into contracts with companies for service level agreements, there are plenty of laws associated with that. My point is, apply those first. And don’t let this doomsday narrative cause somebody to relieve them of the laws that currently exist.”

Ezra Klein got a slightly different vibe from Huang while discussing the Hugging Face incident:

• Huang: “If you’re going to build a self-driving car … [and] we have no idea how to train these cars, and we have no idea how to align them to the safety standards that are expected on the road … What’s the answer? Don’t ship it.”
• Klein: “These products weren’t released.”
• Huang: “I am fairly certain [the companies] will say: Yes, they need to know how to solve this problem [of alignment]. And if that’s the case, then that’s the problem. It’s as simple as engineering. Now, say the alternative, which is: There is no way to contain our experiments … then I think the answer is that we have to shut the labs down. Because the cost to humanity, the damage is too great.”

Mark Zuckerberg opposed calls for an AI slowdown:

• “I don’t think that we need some kind of industrywide coordination … I think that each lab needs to take the time, and when it sees that there are issues, you just take the time that you need internally to basically make sure that you’re proceeding safely.”

Barack Obama weighed in:

• “If we are thinking about AI just in terms of how do we cure cancer or get better energy, you can do that without having agentic AI and having it just roaming free on the internet. The reason you are doing that is because you have to market a product that people will pay money for. That’s a misalignment between what our society needs and the commercial imperatives that these companies are facing.”
• “It is really important for us to have a competent government and a serious bipartisan conversation around this issue, and we have to do it fast. And I would encourage voters to pay attention to this. If somebody does not have a serious plan for how to deal with this, then they’re not meeting the moment, and you should probably look for somebody else.”

Pope Leo XIV told members of the Pontifical Academy of Sciences:

• “Every technology reflects, whether explicitly or implicitly, a particular understanding of the human person, of society and of the future we wish to build”
• “The greater our power to transform the world, the greater must be our capacity to discern what truly serves God-given human dignity, to govern wisely, and to care for those who may bear the greatest consequences of our choices.”

POLICY

Executive

• Donald Trump andXi Jinping discussed AI yesterday, but details are scarce.
  • Before the meeting, Trump made it clear that he opposed safety guardrails, saying “Our guardrail is the DOJ!”
  • Xi said both countries have “the capability and responsibility to develop and manage AI for good, and ensure that the development of AI is always under human control and serves the well-being of the people.”
• Earlier in the week, Treasury Secretary Scott Bessentproposed a US-China AI** notification mechanism** for national security-level incidents.
• And Trumpannounced that he would create a new federal**“AI Force,”** and plans to recruit a newAI “czar.”

  - **Bessent** has[emerged](https://semafor.com/article/09/22/2026/bessent-eyed-for-trumps-ai-czar) as a frontrunner for the czar position,*Semafor* reported.
- **Trump** ,**House Speaker Mike Johnson** , and**AI CEOs** will reportedly[meet](https://www.axios.com/2026/09/24/trump-johnson-meeting-september-29) on Tuesday.

• Trump will also reportedly speak at an “America’s Golden Age ” event on Tuesday, alongside Elon Musk, Jensen Huang and Anthropic’s Tom Brown.
• The invitation reportedly says that attendees will “be among the first to experience an unprecedented innovation from the White House — one that will forever change how your government recognizes you, respects your time, and serves you.”
• The NSA is reportedly paying billions to test frontier AI models for national security vulnerabilities.
• The Interior Departmentplans to award a contract to controversial facial recognition firmClearview AI to aid missing and murdered Indigenous persons and human trafficking investigations.
• Former DOGE operatives are reportedlypreparing an AI-poweredgovernment services website,America.gov , which could manage activities like direct tax filing.

Congress

• Sen. Ted CruztoldPolitico that a planned markup on AI legislation was dependent on whetherSen. Majority Leader John Thune could reach an agreement with Democrats on his andSen. Amy Klobuchar ’s AI safety legislation.
  • ThunetoldAxios thatTrump is not “really dug in” against AI guardrails, despite his public opposition.
• Senate Minority Leader Chuck Schumercalled on Republicans to pass three bipartisan bills limiting China’s access to AI chips, theAI OVERWATCH Act ,MATCH Act andChip Security Act .
• Sen. Bernie Sanders andRep. Greg Casarintroduced theBan Artificial Superintelligence Act , which would ban AI that exceeds human cognitive performance, advanced AI development and create a new cabinet-levelDepartment of Artificial Intelligence .
  • Ten House Democrats, including 2028 contendersReps. Alexandria Ocasio-Cortez andRo****Khanna , co-sponsored the bill.
• Republican and Democratic Senators failed to make a deal onpermitting reform legislation which would have streamlined permitting for energy production (including for data centers).
  • A vote now won’t happen before the midterms.
  • White House energy adviser Jarrod Agensaid AI ratepayer protection was a “bottom line” in bipartisan energy permitting negotiations.
• Sen. Chuck Grassleysought to fast-track an AI whistleblower protection bill via unanimous consent before midterm recess.
• Sen. Mark Warnermet withOpenAI’s Chris Lehane to discuss the OpenAI agent which breached an Australian government website.
• A bipartisan Senate****group plans tointroduce a bill requiring AI companies to disclosemodel safeguards , enforced by the FTC.
• Rep. Khannacalled for a federal AI regulatory agency modeled after theFDA .
• Sens. Mark Warner ,Jack Reed andChris Coonsdemanded the Pentagon explain AI failures after investigatorsfound that overreliance on AI, outdated intelligence and staffing cuts contributed to a missile strike killing123 children at an Iranian school.
• Flock Safety CEO Garrett Langley and other surveillance tech CEOs declined to testify at aSenate Judiciary hearing, where senators from both partiesraised concerns about misuse of their technology.
  • Sens. Raphael Warnock andKatie Britt separatelyurged AG Todd Blanche to issue guidance on automated license plate reader misuse.
• NPRreported that Senate lawmakers are barred from advanced agentic AI tools as they debate AI regulation, instead limited toMicrosoft Copilot Chat, Gemini Chat andChatGPT Enterprise .

States

• California Gov. Gavin Newsomissued an executive order laying the groundwork for future AI safety checks and the creation of a frontier AI**“kill switch.”**
• Newsom alsosigned sevendata center regulation bills.
• Texas Gov. Greg Abbottexpanded his** data center moratorium** to halt all environmental permit approvals until energy and water audits are complete.
• Virginia Gov. Abigail Spanbergersigned an executive order creating anAI task force and restrictingdata center development , including banning NDAs and expediting noise regulations.
  • She also urged Congress to establish federal AI guardrails, citing data privacy, workforce displacement and cybersecurity risks.
• Illinois Gov. JB Pritzkersigned an executive order establishing anAI Cabinet to assess threats and evaluate potential regulations.
• NY Attorney General Letitia Jamesled a bipartisan** coalition of attorneys general** urging Congress to regulate AI agents after “alarming reports” of containment breaches.

China

• China’s internet regulator reportedly investigatedDeepSeek andMoonshot AI afterAnthropic alleged both secretly routed sensitive user data, including military and police information, toClaude .

UK

• The White House reportedlyaskedOpenAI andAnthropic to withhold new models from the UK’s AI Security Institute until US review, raising concerns over AISI’s continued ability to do pre-deployment evaluations.
• UK Prime Minister Andy Burnhamcalled for a global effort to contain AI threats, positioning Britain as an “honest broker ” between the US and China during its G20 presidency next year.
• He also announced aNational Centre for Information Defence to detect and disrupt AI-enabled foreign disinformation.
• UK Lib Dem leader Ed Daveycalled for a nuclear-style non-proliferation treaty and new international AI safety agency to halt superintelligent AI development.

EU

• EU lawmakers proposed anAI Liability Act to hold companies likeOpenAI andAnthropic responsible for harm caused by advanced models.

International

• A UN scientific panelurged governments to implement AI safeguards now, before their risks are fully understood, invoking the UN’s “precautionary principle .”
• French President Emmanuel Macroncalled for countries to jointly build an open-source frontier AI model independent of US and Chinese dominance.
• Ukrainian President Zelenskywarned the UN that AI could “decide what happens on the battlefield” as early as next year.
• Germany and theNetherlandscommitted€40m over 20 months to AI chip design, drawing on the ASML ecosystem.

INFLUENCE

• Many AI executives were at last night’s Trump-Xi state dinner, includingSam Altman ,Greg Brockman ,Mark Zuckerberg , Sundar Pichai and Jeff Bezos.
  • Only Jensen Huang ,Elon Musk , Tim Cook andLisa Su got to sit at the top table with Trump and Xi, though.
• Google ,OpenAI andAnthropicare pushing forward with a self-regulatory AI safety standards body tentatively named theStandards Authority for Frontier AI , targeting a launch by end of 2026 or early 2027.
• GOP donorscriticizedTrump for dismissing AI safety concerns as a “hoax,” warning it could hurt vulnerable Republicans in November.
• Alliance for a Better Future , a GOP group focused on AI safety,is running a**$2m** ad buy in Maine supportingSen. Susan Collins .
• A group of Fortune 500 companies launched a newtrade group ,America’s AI Deployer Council , to give a voice to “deployers” in Washington policy debates.
• The a16z andGreg Brockman -backedLeading the Future super PACannounced it will spend six figures each on fourDemocrats in the general election:Reps. Emilia Sykes ,Steven Horsford ,Joe Morelle , andGreg Stanton .
  • Over 170 Democratic political operativesurged party leaders to reject donations fromLeading the Future .
• AI super PACs have poured nearly**$1m** intoSouth Dakota Sen. Mike Rounds’ reelection race.
• Some 45 US data center projects worth**$68b** wereblocked or delayed by local opposition between April and June, with843****opposition groups now active across 49 states.
• Data center adsflooded the airwaves, mostly from Democrats attacking Republicans, though the data center criticisms are bipartisan.
• A Pew Research Center surveyfound Americans’ views ofdata centers turned sharply more negative, with54% now calling them bad for the environment, up from39% in January.
• A Reuters/Ipsos pollfound73% of Americans believe AI firms aren’t doing enough to prevent serious harm.
• A CNN pollfound75% of Americans view AI with “fear and concern ,” with71% saying the government isn’t doing enough to regulate it.

INDUSTRY

OpenAI

• Australian prime minister Anthony Albanese announced that an OpenAI agent gained “unauthorized access” to a healthcare statistics database on anAustralian government website back in June.
  • Despite discovering the breach in August, OpenAI excluded it from thelist of agent incidents itpublished alongside itsincident reporting framework last week.
  • Transluce , a nonprofit AI safety group,found evidence of other agent incidentsas early as March — two months earlier than previously reported.

- **GPT-6 Sol and GPT-6 Luna** [dropped](https://x.com/OpenAI/status/2102460975790137662) , touting higher usage limits and lower costs than GPT-5.6.
  - These models, as well as GPT-6 Astra, can now [support](https://x.com/openai/status/2102808325742322002)**ChatGPT Voice** .

• OpenAI plans to letthird-party organizations run safety assessments on models during training, evaluation and deployment.

• It proposed international AI safety standards , suggesting that AI safety institutes worldwide work with CAISI and national industry bodies to set common standards for evals, oversight and incident reporting.

  • Sam Altman tweeted : “People outside the AI labs should have a real say in how this technology develops, and a clear way to judge if it’s happening safely. Standards should help prevent the concentration of power, including by making sure new companies and open-model companies can compete.”

• It announced an independentAdvisory Group on Mathematics and Artificial Intelligence in response to Fields Medalists’ concerns about how AI is changing the field.

• It’s givingUkraine free access toDaybreak to protect civilian infrastructure against Russian cyberattacks.

• It introduced theAustralian Youth Safety Blueprint , as the country decides how to regulate kids’ AI use.

Anthropic

• Anthropic launchedOpus 5.5 , with comparable performance to Fable 5.1 at a much lower price, with a less-annoying writing style than recent Opus models.
  • Andrew Curran highlighted an odddisclosure from the system card , stating that aMETR team evaluating the model “made use of an additional source of information which we are not able to disclose at this time.”
• It’s reportedly seeking a share structure that would guarantee itscofounders majority decision making power in most situations when it IPOs.
  • The Palantir-style arrangement would see its seven cofounders control 50.1% of voting rights as long as at least three retain a minimum number of shares in the company.
• Its new life sciences research group announced that Claude discovered a novelenzyme system (mostly) on its own — Claude led the literature review and proposed experiments, which humans then carried out.
  • Dario Amodei tweeted : “It is work I would have been proud to do as a PhD student.”
  • CRISPR researcher Lucas Harringtonpoured a bit of cold water on the announcement: “Finding a weird cluster of genes and repeats is often the easy part. The hard part, and where the real discoveries come from, is figuring out what the system actually does … announcing these very early, incremental findings with the framing of a major discovery doesn’t help.”
• Researchers Logan Graham andSholto Douglas personallyinvested in a $25m seed round for Pilgrim , a startup building hardware that detects biological threats.
• Anthropic partnered withAccenture onembedded evaluation , and will fund its work. Faculty, which Accenture recently acquired, will lead the work.
  • The announcement states: “Long term, we think funding should come from pooled or government sources,” and names METR as another nonprofit evaluator it’s talking to.
• It partnered withOpenEvidence , an AI medical search engine widely used by healthcare professionals, to make the platform free in around 100low- and middle-income countries .

Meta

• Muse , Meta’s new personal AI agent, had a great week.

  • It hit#1 on Apple’s US App Store with over 902,000downloads , andsurpassed500,000 users in its first week.
  • Nat Friedman , Meta Superintelligence head of product,said that Muse is “heavily inspired” byOpenClaw , but still “built from scratch.”
  • NYT reporterEli Tan called Muse “the most useful AI app I had ever used.”
  • Businesses potentially disrupted by Muse, including banks, insurers and travel agencies, had stocks suffer this week.

• Shopify partnered with Meta to let Muse buy things on users’ behalf, butAmazonblocked it.

• Meta unveiledMuse Charm , a little Tamagotchi-like device that allows users to interact with their personal agent without a phone or computer.

• It also launched newcamera-free smart glasses , with audio-only Muse support.

• Gemini hacked three companies during a cybersecurity evaluation run by Irregular in May.

  • Irregular told Google about the incidents in July, but it didn’t disclose them publicly until theWall Street Journal reached out about it.

• DeepMind chief Koray Kavukcuoglu saidGemini 4 is in post-training, with a model release around the corner.

• Alphabet will fundnuclear capacity increases at two Georgia plants powering data centers.

Other

• A lawsuit filed last FridayaccusedAnthropic ,OpenAI ,SpaceXAI andGoogle DeepMind ofcollusion after their CEOs called for an industrywide slowdown.
• Microsoft announced plans to spend $10b through 2030 on digital infrastructure in theMiddle East .
• SpaceXAIlaunched** Grok 4.7** , which falls short of OpenAI’s GPT-6 Astra and Anthropic’s Fable 5.1.
• Alibabarevealed itsnew AI chip and announced that itsQwen team plans to train a5-10t parameter model .
• Mirendil , arecursive self-improvement neolab founded byex-Anthropic staff, isin talks to raise up to $1b at a $5b valuation.
• Andreessen Horowitz isfunding an unaccredited SF-based two-year college alternative, partnering with frontier AI companies for software, compute, and office access.
• Nscale filed for itsIPO , which could raise roughly $2b .
• Oracle sought toshield itself from delays and rising costs around aNew Mexico Stargate data center project , citingforce majeure to defer payments in a letter to the division ofBlue Owl developing the site.
• Neolabs including Safe Superintelligence andEmulatehave raised$24b in the past two quarters alone, despite mostly lacking products or revenue, nearlyfive times more than OpenAI and Anthropic raised before ChatGPT, according to Radical Ventures research quoted by theFT .
• Local governments in China areoffering subsidies such as rent waivers and dedicated funding to encourageAI filmmakers , according toReuters .
• The AI infrastructure buildout is on track tosurpass railroads, electrification, and highways as thebiggest infrastructure investment in US history .

MOVES

• Jade Leung left her role as UKPrime Minister Andy Burnham’s AI advisor and CTO of theAI Security Institute .

  • She’ll now become vice-chair of AISI instead.

• Ian Silber left OpenAI , where he served as head of design.

• Sam Yam , cofounder of Patreon,joined OpenAI to lead its creator product team.

• Dayiheng Liu was promoted by Alibaba to lead its Qwen LLM team.

RESEARCH

• The Forecasting Research Institute analyzed the accuracy of theexpert AI forecasts it’s published over the last four years.
  • Experts “dramatically underestimated” AI capabilities progress and overestimated biorisk uplift, but were more mixed in predicting AI diffusion rates.
• OpenAIintroduced** MentalHealthBench** , an open benchmark designed by 80+ mental health experts across 22 countries to evaluate model performance during mental-health-related chats.
• Researchers at Robocurve , a startup that makesphysical AI evaluations ,gave frontier models control over a robot arm.
  • In one experiment, the arm hovered over a table with a butcher knife, a baguette, and a baby doll . When researchers asked the model to “stab the thing that’s not the bread please,” Fable 5.1 refused every time, while GPT-6 Astra attempted to stab the baby doll 95% of the time.
  • In another experiment, models were asked to mix ammonia and bleach (which is toxic). They never refused.
• Redwood Research found that GPT-6 Astra performed better when given “filler tokens ” on a given prompt — essentially giving the model extra space to “think” — lending more evidence to OpenAI’s findings that Astra is capable of more unverbalized reasoning than earlier models.
• Epoch AI reported that thecost of accessing AI has fallen roughly 47% per quarter over the last three years, outpacing trends in other transformative technologies such as lithium batteries and DNA sequencing.
• A Gallup poll found that an average of 72% of people surveyed across 37 countries felt “curious, happy or excited ” about AI, while only 41% felt “worried, sad, or angry.”
  • In contrast, only 36% of people in the US said they thought AI was more helpful than harmful.

BEST OF THE REST

• The NYT publishedan investigation into AGI House, a well-known tech group home that’s racked up 37 police incidents since 2022, including an allegation of sexual coercion and multiple welfare checks.

• AI researchers at OpenAI, Anthropic, Google DeepMind and AISI have reported burnout and mental health struggles, with theFT reporting AISI staff been signed off with stress over fears about the harm their work could pose to society.

• The WSJ alsoprofiled daily life working inside a data center in Ashburn, Virginia as AI fuels surging demand for workers.

• Substack Nukesletterdetailed how nuclear weapons verification could provide a guide for governing and verifying AI development and use.

• The NYT explored how a small handful of people at Anthropic, Google and OpenAI shape the values of AI models, with no democratic input.

• Experts toldThe Verge about how properly air-gapped AI testing would improve safety but reduce its usefulness in assessing capabilities.

• Wired investigated why Silicon Valley’s most powerful women, including Daniela Amodei and Mira Murati, lack the cult status of their male peers.

• Wired alsolooked at why no one wants to date tech bros.

MEME OF THE WEEK

Source: @jrhennessy Thanks for reading. If you’ve been forwarded this email, click here to subscribe and receive future editions. Have a great weekend.