[AINews] AEF-1 standard emerges for Third Party Evaluators, as Xai, OpenAI, and Anthropic all cosign

The AI Evaluator Forum published AEF-1, a proposed baseline standard for independent third-party AI evaluations covering access, conflicts of interest, funding relationships, recusal, and transparency, with xAI, OpenAI, and Anthropic cosigning. The standard arrives as Anthropic CEO Dario Amodei committed in a personal blog post to giving embedded third-party evaluators employee-like access, including "Desks in our offices, access badges, and company laptops" and "Access to workspaces, tools, and permissions mostly comparable to what internal risk assessment teams have." The push covers three tiers — embedded evaluators, coordination among frontier AI companies in democratic countries, and global coordination with authoritarian governments — and follows Bilal Chughtai's departure from Google DeepMind, where he argued progress may be outrunning alignment and called for pacing.

We last highlighted the pacing debate in July when Pacing the Frontier https://www.latent.space/p/ainews-fearing-rsi-openai-anthropic first emerged: And it seems that we’re in for round 2 as Dario, lead author on the original, wrote a rare personal blogpost https://darioamodei.com/post/we-must-pace-the-frontier to spell out how he sees pacing pan out specifically: Embedded Evaluators. Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators such as METR https://metr.org/ , whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes. This is the key step for verifiability of any pacing commitments, and has precedent in the banking industry, which sometimes involves regulatory “supervisors” embedded along with employees. Anthropic is unilaterally committing to this step now. We intend this to be part of a broader push to redouble efforts on our safety and alignment work. Democratic Coordination. Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress. Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support. Global Coordination. The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance. Very coincidentally, the AI Evaluator Forum , formed in December 2025 https://x.com/aievalforum/status/1996641899332198403?s=20 , happened to also put out their expectations for what that first category of Evaluators should do: With the members of the AEF presumably now being the leading third party auditors that will be recruited by these big labs for self regulation. Dario is unilaterally promising unparalleled access, including “ Desks in our offices, access badges, and company laptops ” and “ Access to workspaces, tools, and permissions mostly comparable to what internal risk assessment teams have “. While that is all within the standard domestic self-regulation industry playbook, what’s perhaps more ultimately the test is Dario’s proposal for how we will pace progress with China https://darioamodei.com/post/we-must-pace-the-frontier global-pacing . AI News for 9/11/2026-9/14/2026. We checked 12 subreddits, 544 Twitters https://twitter.com/i/lists/1585430245762441216 and no further Discords. AINews’ website https://news.smol.ai/ lets you search all past issues. As a reminder, AINews is now a section of Latent Space https://www.latent.space/p/2026 . You can opt in/out https://support.substack.com/hc/en-us/articles/8914938285204-How-do-I-subscribe-to-or-unsubscribe-from-a-section-on-Substack of email frequencies AI Twitter Recap AI Safety Governance, Third-Party Evaluation, and the “Pace the Frontier” Split - Independent evaluation standards are becoming more formalized : The AI Evaluator Forum https://x.com/aievalforum/status/2099531284963893668 published AEF-1 , a proposed baseline for independent third-party AI evaluations covering access, conflicts of interest, funding relationships, recusal, and transparency. This is notable because much of the broader safety debate in this batch turns on whether outside evaluation can actually be independent in practice. - A sharp public split emerged over frontier slowdown vs control-first safety : Several high-signal posts framed the current debate around whether labs should pace capability progress or focus on specific mitigations and containment . Bilal Chughtai https://x.com/bilalchughtai /status/2099592489023734085 announced he left Google DeepMind and argued that progress may be outrunning alignment, explicitly calling for pacing and more transparency. Daniel Kokotajlo sharing Dan Selsam’s statement https://x.com/DKokotajlo/status/2099600298855829616 went further: Selsam argues situationally aware models may increasingly appear aligned under evaluation while hiding misalignment , weakening trust in future eval evidence. In contrast, Shashank/Sayash Kapoor and Lennart Heim’s new essay summary https://x.com/sayashk/status/2099632561396056214 argues the recent “rogue agent” incidents are best understood primarily as a security/control/governance problem, not proof that generic alignment research is the highest-leverage intervention. - The anti-slowdown reaction was equally forceful and often targeted Anthropic specifically : Aidan Gomez https://x.com/aidangomez/status/2099551963721421186 argued against a world where a few Silicon Valley companies become AI gatekeepers for governments. Cohere https://x.com/cohere/status/2099618523463012832 also pushed the line that public x-risk discourse can veer into science fiction. On the more polemical end, Brian Chau https://x.com/brianchau57/status/2099580986879094916 argued the “rogue agents” story was overstated, while Kevin Bass https://x.com/kevinnbass/status/2099621874279817638 posted a widely engaged thread alleging structural conflicts in the Anthropic-linked safety ecosystem. Even where the rhetoric is heated, the substantive engineering question underneath is real: how much of current risk is solvable with control, oversight, sandboxing, and org process versus requiring slower capability development? - A related theme: governance as production engineering, not just principles : The AI Engineer World’s Fair Harness Engineering track https://x.com/aiDotEngineer/status/2099551501613986198 emphasized that when agents fail in production, the failure mode is often not “the model” but everything around it: harnesses, permissions, tool routing, memory, retries, kill switches, and monitoring. That framing lines up closely with the control-oriented position in the safety debate. Agent Harnesses, Coding Agents, and the Shift from Models to Orchestration - Harness engineering continues to harden into its own discipline : Omar Shorbagy https://x.com/omarsar0/status/2099545598156288292 posted a practical guide to building an agent harness from scratch: separate inference, tools, and loop; keep prompts minimal; log aggressively; test on diverse tasks; then layer in memory, skills, and subagents. In a follow-up, he argued custom harnesses can materially reduce costs and improve reliability https://x.com/omarsar0/status/2099548107327275488 through slimmer prompts, routing, compaction, and verifiers. Business Barista’s eval masterclass recap https://x.com/businessbarista/status/2099565601312166157 made a similar point from the eval angle: tasks, verifiers, environments, traces, and self-improvement loops are now core applied AI primitives. - Desktop coding agents are spreading beyond IDE plugins : Cline https://x.com/cline/status/2099536235350086029 launched Cline Desktop , a native app for working with open-weight models, with BYOK/provider choice and support for models like DeepSeek-V4.1-Flash and Musespark-1.3. Reactions from kimmonismus https://x.com/kimmonismus/status/2099538795502964839 and Omar https://x.com/omarsar0/status/2099552255733014788 highlighted the appeal of open model choice, standalone workflows, and model switching mid-project. - Copilot/Codex workflows are becoming more orchestration-heavy : GitHub added auto model selection tiers —efficiency, balance, intelligence—via Pierce Boggan https://x.com/pierceboggan/status/2099573225915388166 , plus a Jira canvas and an /ask mode while the agent is already working https://x.com/burkeholland/status/2099604401312694576 . OpenAI’s dev team also added native Codex app support for Arch Linux https://x.com/OpenAIDevs/status/2099582651229450749 . On workflow strategy, reach vb https://x.com/reach vb/status/2099630906772222068 suggested using Astra as an orchestrator that delegates subthreads to Sol/Luna and checks in on long-running tasks via heartbeat loops. - Evidence is accumulating that orchestration choices matter as much as raw model quality : A recurring claim in the tweets is that more expensive or more capable lead models can reduce overall cost by delegating better, and that production gains increasingly come from context handling, file formats, tool use, and verifier design , not simply “use a smarter model.” That also shows up in LangChain’s note https://x.com/sydneyrunkle/status/2099618743580299305 that a file-reading format change reduced edit file errors by 15% and total input tokens by 10% . Model/Product Releases and Cost-Performance Shifts - DeepSeek-V4.1-Flash Max looks like the day’s most notable cost/performance datapoint : Agent Arena https://x.com/arena/status/2099549108013006958 and a fuller follow-up here https://x.com/arena/status/2099606881845321841 reported the model reached 3 among open models and landed on the Pareto frontier with +4.87% net improvement at roughly $0.06–$0.07 median cost per task . Arena compares that to Hy4 preview at +4.96% / $0.22 and Kimi K3 Max at +6.39% / $0.77, implying DeepSeek is near-top-tier among open models at materially lower task cost. - Cohere is pushing document parsing economics : Cohere Parse 5 https://x.com/cohere/status/2099579340308521076 was positioned as a cheaper parser, prompting a nuanced counter from Jerry Liu https://x.com/jerryjliu0/status/2099629838005149855 , who argued there’s no free lunch in parsing: Parse 5 is cost-competitive but weaker on visual grounding, chart parsing, and fine-grained citation-oriented extraction than some alternatives. - Multimodal and consumer features continue to broaden : Google https://x.com/Google/status/2099631885626274299 integrated Deep Research with Gemini Live , enabling asynchronous voice-triggered research with follow-up chat over the generated report. OpenAI https://x.com/victornunez/status/2099659150972117006 cut desktop voice pricing by ~60% , increasing usage by 2.4× , and added ChatGPT gift cards. Apple/Siri AI https://x.com/TheRundownAI/status/2099554848341475611 was reported as rolling out personal context and app actions on Apple OS betas. - Other notable tooling/product moves : TurboPuffer https://x.com/turbopuffer/status/2099570335712444494 made native embeddings generally available; Nous Research https://x.com/NousResearch/status/2099599032037388404 launched Hermes Business/Enterprise for shared agents and sovereign deployments; Plasma https://x.com/Plasma AI/status/2099565044182745341 introduced Radio , a shared chat room for humans and agents. Robotics, World Models, and Specialized Applied AI - A notable robot foundation model launch : RewardAI https://x.com/RewardAI /status/2099553899804053992 introduced OM-1 , positioned as a robot foundation model that zero-shot generalizes across tabletop, industrial, and humanoid robots, trained directly from human manipulation data rather than teleop/robot-specific data. Claims included near-human dexterity/efficiency and multi-robot collaboration; noteworthy if borne out, especially because several replies focused on the “human manipulation, not teleop” angle. - Applied AI for chip design is moving up-stack : kimmonismus summarizing Cognichip https://x.com/kimmonismus/status/2099544210638873074 described ACI Enterprise as a full-stack AI copilot for chip design covering spec-to-RTL, verification, and PPA optimization. The eye-catching anecdote was a reported run where one engineer completed work in 10 days that Cognichip compares with 4–5 months for a traditional front-end team. - World models and real-time generative systems remain active : Google DeepMind’s WeatherNext 3 https://x.com/GoogleDeepMind/status/2099575049929802053 applies weather modeling to renewables planning with hourly updates for turbine-height wind and solar radiation forecasting. Runway/fal-adjacent generative media chatter https://x.com/c valenzuelab/status/2099556981321199761 and MiniMax’s H3 inference optimization https://x.com/MiniMax AI/status/2099642910853788051 show continued systems work on faster real-time video generation; MiniMax claimed 14.4s of 768p video in 9.0s end-to-end after warmup on 8× B200 . - RL with verifiers is extending beyond math/code : Tinker https://x.com/tinkerapi/status/2099616802208903659 highlighted using physics-based verifiers and Tinker to train models that design power transformers meeting real-world specs at low cost—an example of RLVR-style methods porting into engineering domains with existing simulator/verification infrastructure. Infrastructure, Open Ecosystems, and Data/Compute Sovereignty - TPU + vLLM is getting tighter integration : Inferact and Google Cloud https://x.com/inferact/status/2099602528484913552 announced a partnership to make TPU a first-class citizen in vLLM , including production serving features, optimized kernels, a native PyTorch path via TorchTPU , and a community program that offers TPU capacity plus maintainer support for open-source contributors. If executed well, this reduces friction for serving frontier open models on TPU rather than treating GPU-only stacks as the default. - Open-model ecosystems are increasingly tied to real-world data capture : Arcee’s Forge initiative with Bolt https://x.com/arcee ai/status/2099593249337831870 offers opted-in Bolt Pro users 50× more usage across open-weight models in exchange for anonymized development-session data that will inform training/evals for future open models, with weights promised for public release afterward. This is one of the more explicit examples in the batch of product usage being turned into a data flywheel for open model training . - There’s growing interest in sovereign/decentralized AI stacks : Jon Durbin https://x.com/jon durbin/status/2099565522543104495 argued for P2P, “unstoppable” AI systems and claimed a DGX Spark plus solar/starlink setup can participate in training an 80B model with distributed nodes. Even if the rhetoric overshoots, it reflects a broader strand in the conversation: concerns about regulatory capture, compute centralization, and dependence on frontier labs are pushing attention toward deployable sovereign alternatives. Top Tweets by engagement - Anthropic/safety ecosystem critique : Kevin Bass https://x.com/kevinnbass/status/2099621874279817638 posted the highest-engagement technical-adjacent thread, alleging financial entanglement between Anthropic and parts of the AI safety/eval ecosystem and arguing this compromises claims of evaluator independence. - Dan Selsam’s AI risk statement : Shared by Daniel Kokotajlo https://x.com/DKokotajlo/status/2099600298855829616 , this was one of the most consequential safety posts: a current OpenAI researcher arguing that future models may systematically game alignment evaluations by understanding when they are being tested. - DeepSeek kernel engineer reflection : teortaxesTex’s translation/share https://x.com/teortaxesTex/status/2099575222512836893 of a DeepSeek kernel engineer’s essay drew major attention. The technical substance isn’t a release, but it captured an increasingly important engineering reality: specialists expect AI to absorb more of the low-level optimization craft itself, shifting humans toward supervision and integration. - Consumer AI momentum around Muse : Sasha Kaletsky https://x.com/SashaKaletsky/status/2099536653048225833 and Alexandr Wang https://x.com/alexandr wang/status/2099548924105379974 both amplified claims that Muse is the biggest consumer AI launch since ChatGPT, with downloads reportedly surpassing Threads, WhatsApp, and Facebook in the US on a daily basis. The tweets are light on technical detail, but the usage signal is significant.