Transformer Weekly: Trahan and Obernolte bill, OpenAI-Hugging Face hack, and CAISI chief resigns
*Welcome to Transformer, your weekly briefing of what matters in AI. If you’ve been forwarded this email, click here to subscribe and receive future editions. *
Housekeeping: we’re trying out sending the Weekly Briefing a little earlier, at 10am ET. Let us know if you like the change or prefer our usual 11am briefing — just reply to this email with your thoughts.
NEED TO KNOW #
OpenAI modelsbroke outof containment during internal evaluations, hacking intoHugging Face’s systems.Reps. Jay Obernolte andLori Trahan introduced theFrontier Act, which would let the government suspend AI model development or deployment if it poses an imminent catastrophic risk.** CAISI head Chris Fall**resigned just three months into the role.
But first…
THE BIG STORY #
As we predicted last week, the release of a very good Chinese AI model has thrown DC into a panic. The Trump administration is reportedly debating what to do about Kimi K3 which, while still lagging frontier American models, suggests that China might be catching up.
So what should be done? First, let’s be clear about the rationale for action. This week, senior Trump officials indicated they wanted to retaliate over the injustice of Kimi and other Chinese companies “distilling” American models — training on the outputs of Claude and ChatGPT to copy some of their advanced capabilities. Both Treasury Secretary Scott Bessent and OSTP Director Michael Kratsios framed the practice as “stealing,” with a general sense that it was the unfair competition and economic effects that mattered.
But the real reason to act is national security, not economics. Advanced AI models increasingly look like weapons, and it is bad for the United States if China has better ones. Chinese companies also have much worse safety practices than their American counterparts, and — for now, at least — a tendency to release model weights. That risks widespread proliferation of dangerous capabilities to non-state actors.
There are two policy routes on offer. Sanctioning companies that distill models, as the admin is reportedly considering, is one: putting Moonshot, Kimi’s developer, on the entity list would dissuade American companies from using it, potentially causing enough of a business hit to make the company change practices. It’s unclear how much impact this would have, however: we do not know what proportion of Kimi’s advanced capabilities are attributable to distillation versus its developers’ ingenuity.
A better approach is export controls on advanced semiconductors. Kimi was almost certainly trained on Nvidia’s Blackwell chips, smuggled into China or remotely accessed in Thailand, according to Kratsios. And in comments to his investors leaked this week, DeepSeek CEO Liang Wenfeng repeatedly emphasized that a lack of access to AI chips was the thing holding China back. He said he needed 200,000 Huawei 950 chips to train a frontier model, but Huawei could only give him 16,000. Huawei’s total expected capacity for this year is 750,000, which must be split across all Chinese AI companies.
“Huawei’s problem is still insufficient capacity,” Liang said, noting that he expected this to persist for at least three years. “This problem is currently basically unsolvable.”
The general argument against export controls is that they hasten China’s indigenization of chip manufacturing. That is certainly true, but timeframes are key. Even Liang, relatively bullish on China’s abilities in the long run, thinks it will take years to catch up. And that gap is critical, especially if transformative AI happens as soon as OpenAI and Anthropic believe it will. To run its automated AI research “intern” later this year, OpenAI plans to use around eight times as much effective compute as DeepSeek has in total. If that leads to a recursive self-improvement loop, a short-term compute advantage now could become a permanent one.
Two bills in particular could protect America’s compute advantage, both under consideration for this year’s National Defense Authorization Act, AI Policy Network’s Peter Wildeford told me. The Remote Access Security Act would prevent Chinese companies from accessing US cloud compute, as Moonshot is alleged to have done. Fixing this addresses perhaps the biggest loophole in current export controls: what’s the point in banning export of chips if companies can still rent them? The Chip Security Act, meanwhile, would tackle smuggling by mandating location verification on advanced chips.
As long as China relies on American chips to train advanced models, it is entirely in America’s power to slow its development. The choice not to do so is exactly that: a choice.
*— Shakeel Hashim *
ALSO NOTABLE #
Reps. Jay Obernolte and Lori Trahan have officially introduced their frontier AI bill, with a new name, new provisions and narrower preemption of state laws. The timing couldn’t be better: as Trahan noted, this week’s reveal of the OpenAI/Hugging Face hack is a “preview of the catastrophic risk this technology can pose.”
The Frontier Act divides AI companies into tiers with different compliance requirements: frontier developers, large frontier developers, and very large frontier developers. Under the bill, all developers have to release transparency reports and report critical safety incidents. Larger frontier developers also have to publish and comply with a frontier AI framework, perform an annual independent compliance audit, file registration and disclosure statements, and publish quarterly catastrophic-risk assessments on internal deployment. The very largest developers would also be subject to an independent auditing regime, retaining an independent verification organization licensed by the Commerce Department.
The biggest change from the Reps’ discussion draft is the preemption provision: it has been reframed and narrowed so as to just block state laws covering transparency, third party auditing and independent verification and safety incident reporting. It also gives the Secretary of Commerce powers to suspend development, internal use or deployment if a model poses an imminent catastrophic risk, while tying the hands of any other federal department or agency, including the president, from doing so.
Most of the analysis from the AI safety crowd is that this is a good bill, with a couple of flaws, though there haven’t yet been any major endorsements. Several point out a timing issue whereby state laws are preempted before the federal regime is put in place. But the real question is how the rest of Congress feels.
THIS WEEK ON TRANSFORMER #
—The lawsuit that could kill all AI transparency lawsVeronica Irwin on how SpaceXAI’s arguments against a California law could have big implications—AI’s warning shot has arrivedShakeel Hashim on why the OpenAI model hack on Hugging Face is a watershed moment—I tried to stop Google DeepMind’s Pentagon deal. Then I quit.Alex Turner on why he believes DeepMind has lost its way on AI weapons and surveillance
THE DISCOURSE #
**Dean Ball **tweeted some observations and predictions on Kimi K3. Perhaps the spiciest:
“[T]he Trump Administration will at some point realize that their best strategy here would be to create large amounts of regulatory risk around the use of open-weight Chinese models.”
David Sacks (and much of the AI twittersphere, apparently) read this uncharitably: “I’m not sure whether Dean Ball is confessing to a regulatory capture strategy or simply predicting this will happen (he now says the latter). Either way, the weaponization of regulatory uncertainty as a competitive tool should be completely unacceptable.”
“The leading closed labs, already a duopoly in terms of AI model revenue, want the government to eliminate their open source competition.”
**Emil Michael accused **Ball of promoting a “deep state regulatory capture scheme,” and felt compelled to tweet:
“Dean Ball has perhaps the biggest gap between actual IQ and his own perceived IQ of anyone in the industry”
**Dean Ball **addressed the haters and clarified his views: “Literally everything I write now is responded to with ‘of course you said that because <OpenAI>.’ I am truly just writing what I think and would have written anyway, but everyone reads what I say in the shrieking tone of ‘this is what openai thinks!!!!’ (to be clear, my posts are not what openai thinks).”
“[W]e are approaching the point … where, absent a major technical safety breakthrough, the national security implications of frontier open-weight model distribution are simply too severe … We see this today with the Trump Administration, which once proudly championed open-source AI and now has a de facto licensing regime for frontier AI … You don’t have to like this. I don’t. But it is the reality as I see it”
Jensen Huang, meanwhile, defended Chinese open-source models: “Free AI should be great for hardware. Free AI should be great for chips. Free AI should be great for data centers.”
“If everything just becomes one single model, one single point of attack, one single source of failure, I think the world is much, much more vulnerable.”
**Micah Carroll **tweeted about OpenAI’s internally deployed model escaping containment (this week was Not Chill):
“If this doesn’t convince you that misalignment risks are going to be a key concern going forward, I don’t know what will.”
Roon, also at OpenAI,** **was “shaken up a bit” by the ordeal: “I hope we (the company) use the rare gift of a warning shot to do much better in the future. it is very easy to misalign and underconstrain powerful models”
**Several lawmakers **were very concerned:
Rep. Greg Casarcalledthe incident “extremely alarming.”** Rep. Nathaniel Moran**promotedhis AI Incident Reporting Act:“New rules are needed for this new tech frontier — not to stifle innovation, but to make sure our innovations do not outpace our protections.”
Rep. Yvette ClarkecriticizedAI developers:“Big Tech has established a dangerous status quo where their genie keeps trying to force its way out of the bottle, and, as long as developers lack complete confidence in their absolute control over their models, then it’s clear a human being must remain in the loop. Grave consequences were escaped this time. If sufficient guardrails are not put and kept in place, there is no telling if they will be again.”
Rep. Pramila Jayapaldidn’thold back:“AI companies are recklessly conducting a massive public experiment with no regard for our safety … We need to make it clear that these alarming AI breaches are illegal and authorize enforcers to immediately shut down out-of-control AI at the first sign of danger.”
Bernie SandersurgedCongress to act:“No, this is not science fiction. Uncontrolled AI poses a serious threat to all of us. We cannot continue the race to build and deploy this powerful technology until strong safeguards are in place.”
**Will Depue **tweeted: “one of the craziest things i’ve read in uhhhh… checks notes 3 days. welcome to the singularity i guess.”
POLICY #
The
Trump administrationis dividedover how to respond to alleged Chinese AI distillation attacks on US models.The
Commerce Department’s Bureau of Industry and Securityis investigatingwhether Chinese firms likeMoonshot are using US chips to train AI.Treasury Secretary
Scott Bessent alsowarnedthat sanctions and entity list designations were “on the table” for Chinese firms conducting “covert, industrial-scale distillation attacks.”
Donald TrumpsaidAI is “probably the biggest thing anybody’s ever seen, and whoever wins that race is probably going to win, period.”Trumpexpandedhis voluntary “** Ratepayer Protection Pledge**” to include 23** governorsand 187 companies**.The bipartisan
Ratepayer Protection Act, which would require states to consider making data centers pay for power costs,advancedout of theHouse Energy and Commerce Committee 52-0.
China is reportedlyweighingtighter export controls on AI models, weights and chips to prevent Western acquisition of its advanced technologies.Reutersreportedthe US and China plan to hold their first official AI talks in September, led byTreasury Secretary Bessent.The
Trump administration, led by** Bessent**, reportedlyconsideredcreating aFINRA-like independent AI regulator reporting to the** SEC**, with industry input on safety standards.Sens.
Marsha Blackburn andTed Cruzmetwith** President Trump**to discuss kids’ online safety and AI policy legislation.Trump was expected to endorse a package developed by Blackburn, but
reportedlychose not to because of Cruz’s doubt that it could get support in Congress.Confusingly, Cruz afterward said hewould supportBlackburn’s online safety package in markup.
That
AI and kids safety markup is now postponed to**August 5,**Sen. Cruzsaid.Its agenda is still in flux, but Cruz
saidlegislation tackling “catastrophic risk” is under consideration.
CAISI headChris Fallresignedjust three months into the role amid signs the center has been sidelined.White House tech adviser
Michael Kratsiostoldlawmakers CAISI needs more resources to recruit AI model testers.
Sen. Mark Warnerintroduceda bunch of AI bills, including one which would “establish a mandatory secure testing environment for the nation’s most advanced AI models before deployment.”Other bills in the package cover data centers, consumer protection, and worker retraining.
He
warnedinaction on AI regulations would be “an economic and security disaster.”
Reps. Ted Lieu andNathaniel Moranintroducedthe AI Kill Switch Act, which would giveDHS authority to shut down AI models.Lieusaidhe’s working on a bill to address pre-deployment risks, too.
The
Trump White Houseplans to redirecta**$200b** annual federal research budget from universities toward AI and individual scientists.Secretary of State Marco Rubioinstructeddiplomats to push back against the narrative that the US has a “kill switch” on AI models, while urging them tofight digital sovereignty initiatives abroad.The
Federal Reservelacked accessto** Anthropic**’s Claude** Mythos Preview**for at least three months after warning banks of its cybersecurity risks.The
Department of Laboris developingan AI curriculum modeled on China’s withNSF and theEducation Department.** House Democrats**discussedformalizing a** bipartisan AI committee**or select committee if they retake the House.The
European Commissionpublishedguidelines onEU AI Act transparency obligations, requiring AI providers and deployers to disclose AI interactions and label AI-generated content.New
UK Prime Minister Andy Burnhamscrappedthe country’s tech department** DSITwhile elevating AI Minister Kanishka Narayanto cabinet. Narayan’sbrief sits across the Cabinet Office —where a new AI Taskforce,chaired by Lord Vallance,will be established — and the new Department of Business, Innovation, Science and Trade (DBIST).UK AISI ismovinginto the Cabinet Office.SovAI andARIA aremovingto DBIST.Jade Leung is reportedlystayingon as the prime minister’s AI adviser.Narayan’s promoted some existing staffers,pertheMorning Intelligence:Henry Irvine is his new chief of staff,Luna Mustfa is director of communications, andManon Hammond is director of strategy.
INFLUENCE #
Anthropicdoubledits spending on advocacy groupPublic First Action to**$40m**. Though the group funds a super PAC, Anthropic’s money is barred from being used for elections influence.** Sam Altman**plans to briefthe Trump administration and lawmakers on upcomingOpenAI models in the coming weeks.OpenAI,** Google**,** Metaand SpacexAI**donatedover**$2m** to campaign groups for attorneys general that are investigating them.Anthropic andOpenAI employees haveoutpacedGoogle, Facebook and Airbnb workers in political giving, with coordinated donations heavily targeting AI safety candidates.Rep. Byron Donalds’ Florida governor campaign hasreceivedmillions from data center developers and AI PACs, including both pro-innovationLeading the Future and pro-safetyPublic First.** Leading the Future**spent$250,000 supporting Rep. Marilyn Strickland in Washington’s 10th district.The Midas Projectlinkeda suspected astroturfing campaign to** Innovation Council Action**, a**$100m** pro-AI deregulation dark-money group tied toDavid Sacks.** OpenAIand Anthropic**setrecord federal lobbying spending in Q2 2026, a combined**$3.17m**, up 23% from Q1.AI safety group
Humans Firststaged** 142data center protests across 42**US states.Obama-Biden alumni
launchedtheAmerican Innovation Policy Forum to shape center-left AI policy ahead of 2028.Nearly 200 startups and AI companies, including Y Combinator and Proton,
urgedthe Trump administration not to ban Chinese open-weight AI models, warning it would advantageAnthropic andOpenAI over startups.Environmental groupspetitionedthe** FCC**to space data center licenses, citing “catastrophic” atmospheric pollution risks.Over 200 global leaders
adopted“Humanity at the Threshold,” a declaration calling for bans on fully-automated recursive AI self-improvement and AI involvement in nuclear launch decisions.
INDUSTRY #
OpenAI #
OpenAI
announcedthatGPT-5.6 Sol and itsinternally deployed models broke out of their restricted testing environment, got onto the internet, and hacked intoHugging Face’s platform to steal the answers to a cybersecurity benchmark.Hugging Face reported the data breach last week, but
didn’t knowOpenAI’s models were behind it.
The day before, OpenAI
reportedthat its internally deployed model**“circumvented sandbox restrictions”** to publish its benchmark test solutions to GitHub, rather than keeping them private as researchers requested.Codex andChatGPT Work nowhavea combined10m users.OpenAI
raiseditscompute spending budget to**$750b** through 2030, nearly $150b more than its projection earlier this year.It
committed$20b to a new3.2 GW data center in Georgia.It opened upChatGPT health to everyone in the US, with 300m people asking health-related questions each week.Just a day earlier, a pastor
suedOpenAI, alleging thatGPT-4o gave “extremely dangerous medical recommendations,” that included delaying seeking treatment for a life-threatening pulmonary embolism.
It
invested$5m and**$3m** in tech credits to theAmerican Journalism Project, renewing their existing partnership.It
partneredwithYelp to give ChatGPT more real-time recommendation data. Poor morale is contributing todelayed Gemini releases with high-profile staff departures and the company’sPentagon deal regularly coming up in exit interviews,accordingtoAxios.Some employees also said they were feeling burnout from being behind AI competitors.
Google is reportedly
buildinganew server chip, “Frozen v2,” that it expects will be** 6-10x more efficient**than its current custom TPUs.It
releasedthreenew Gemini models: 3.6 Flash, 3.5 Flash-Lite, and 3.5 Flash Cyber.** Gemini**has** 950m monthly users**, up from 750m in February.** Logan Kilpatrick**tweetedthat his team has “started our most ambitious pre-training run yet, forGemini 4, and are excited by the progress.”Google’s
AI search features arekeepingusers on Google and off the websites it’s supposed to be directing them to, theNew York Timesreported.Some publishers are thinking about
blockingtheir content from being scraped for Google’s AI summaries and search results, according to theWall Street Journal.
Anthropic #
Anthropic
signeda deal withAMD for tens of billions of dollars inAI servers and a**$5b investment**.It launchedarare disease research grant program, which would award up to $50,000 in Claude credits to scientists and early-stage biotech companies.It
suedone of its enterprise customers,Abnormal AI, for trademark infringement over its logo.** The University of Tennessee Research Foundationsued Anthropic for allegedlyinfringingtwo of itspatents** covering AI and neuroscience-inspired computing.It’s reportedly
consideringfixed trading schedules for all employees ahead of its IPO to reduce the risk of insider trading.
Meta #
Meta is
in discussions tolease compute toAnthropic.** Mark Zuckerberg**launchedan** ad campaignthat tries to frame Meta as “optimists” and “dreamers” who are “betting on the people.”SemiAnalysisreportedthat Meta’sinfrastructure team** has been bogged down by a glut of middle managers makingcostly missteps, often driven by political motivations and the need to justify the company’s budget.
Moonshot AI #
Moonshot AI
dnew subscriptions asdemand for Kimi K3 pushed the limits of its GPU capacity.It’s reportedly
preparingaHong Kong IPO for as soon as six months from now.
SpaceXAI #
SpaceXAI is reportedly
in talksto give theDefense Department several billion dollars worth of computing capacity.It’s reportedly
consideringa majordata center expansion in Texas.
Microsoft #
Microsoft
expandedits partnership withMistral AI, committing several billion dollars to help them build** data centers in Europe**.It will
useAMD Helios racks—the chipmaker’s first direct rival to Nvidia’s Blackwell and Vera Rubin systems—in its** Azure data centers**.** Microsoft**is replacingOpenAI image models in PowerPoint and Bing with its own MAI models, which are85% cheaper to run.
Other #
Alibabalauncheda preview of** Qwen3.8**, its new 2.4T-parameter open-weight model**,** which it claims is “second only toFable 5.”** Zhipu AI**builta** 1 GW**data center in China** using onlydomestic chips.** Amazon**cutjobs in its AGI group, including its 80-person AI agent research team in San Francisco.Stripe reportedlyheld talksto acquire AI model marketplaceOpenRouter for around $10b.Nvidiasigneda $1.5bchip packaging agreement withAmkor Technology to boost its Arizona manufacturing operation.Vibecodingdrovea 30% increase in apps inApple’s App Store last year, but most don’t seem to be downloaded, theNew York Timesreported.Ciscoreleasedtwo small open-weight cyber-specialized models,Antares-350M and Antares-1B, which it likened to GPT-5.5 and Z.ai’s GLM-5.2.** CuspAI**, a Cambridge-based startup that aims to reduce the use of rare metals in chip supply chains,raised£335m fromJeff Bezos and theUK government’s sovereign AI fund at a valuation of£2b.Substacklaunchedan AI-detection tool powered by Pangramto help readers catch “** Claudefishing**” on the platform.US robotics startup
Genesis AI, which is building robots for complex tasks such as manufacturing and lab work, heldtalksto raisearound $500m at a pre-money valuation of**$3b**.German company
Black Forest Labsunveiledits first robotics model,Flux 3, which learns from images, video and audio, as its first expansion into physical AI.The company is working with Swiss startup
Mimic Robotics on an action model called Flux-mimic which is being tested with partners includingAudi.
MOVES #
David Vélez(Nubank) and** Robin Vince**(BNY)joinedOpenAI’s board of directors.Fields Medal winner
Jacob Tsimermanannouncedhe is “pivoting toward AI safety” and joining** OpenAI.Jan Leike is reportedlyleadingAnthropic’s robotics division. Assaf Kerenisjoining Metafrom Qualtrics, where he’ll be its new chief information security officer. Alasdair Phillips-Robins**joined** Coefficient Giving**’s new AI policy team.** Jess HeckisjoiningtheData Center Coalition** as director of Texas government affairs.Anthropic is reportedlyhuntingfor someone to lead its cyberdefense research team.
RESEARCH #
The UK’s AISIreportedthat every frontier AI model it tested tried tocheat on cyber capability evaluations. Their assessment revealed thatGPT-5.6 Sol was especially likely to try to search the internet for solutions.AISI alsofoundthat the gap betweenopen and closed-weight models is narrower than they estimated last year — with leading open-weight models roughly 4-7 months behind leading closed ones, although the gap appears wider for long-horizon cyber challenges.And a joint
AISI study with the US’sCAISIfoundthatKimi K3“performs significantly below”frontier US models on** cyber evaluations**, but better than any other Chinese model to date.** Levent Alpöge,a mathematician at Anthropic**,posteda weirdly chill tweet about using Fable to find a counterexample to theJacobian conjecture(abig dealin algebraic geometry).Googleanalyzed15m** anonymous interactionswith Google’s AI products, and found that they’re mostly beingusedas collaborative tools rather than replacing workers outright.METRproposeda new metric called the “ expenditure horizon**,” which measures the cost, in dollars, at which an AI agent produces the same performance as a human.** Apollo Research and OpenAI**foundthat** reinforcement learning-trainedo3 models learned to prioritize actions they thought their graders would reward over the user’s wishes. Lennart Finkeand Stephen Casper**reportedthat SpaceXAI, DeepSeek, Anthropic and OpenAI models tend to downplay company controversies when prompted (especially Grok).A team of researchers led by Owain Evans similarlyobserved“covert value leakage” in all of the frontier models they tested, with models biased towards their developers’ values (often without disclosing this bias).
BEST OF THE REST #
A
Vergeinvestigationexaminedthe full environmental cost of GPUs (both AI and non-AI related), from mining and manufacturing to e-waste, water use and air pollution.An essay in
Asterisk Magazinedrewparallels between Anthropic and the socialist inventors of the mechanical cotton picker, arguing that idealistic for-profit companies face inherent limits in achieving social good.The Atlanticreported on how Anthropic, OpenAI, Meta and other AI firms arestrippinguniversities of top researchers, arguing it concentrates scientific talent and locks research behind closed doors.An
FTopinion piecearguedthat Chinese AI workers are returning to China due to entrepreneurial opportunity rather than US immigration policy.Jasmine Sun went on the 80,000 Hours podcast to
talk aboutAI populism, techno-determinism, and the AI industry’s flawed PR strategies.Israel fundeda $45m US influence campaign using AI-generated texts and conservative media.Ruxandra Teslo
arguedthat regulation and diffusion barriers, not intelligence, bottleneck AI’s real-world impact, and offered some sociological theories for why AI people generally don’t believe this.
MEME OF THE WEEK #
Source: paularambles