[AINews] AEF-1 standard emerges for Third Party Evaluators, as Xai, OpenAI, and Anthropic all cosign

We last highlighted the pacing debate in July when Pacing the Frontier first emerged: And it seems that we’re in for round 2 as Dario, lead author on the original, wrote a rare personal blogpost to spell out how he sees pacing pan out specifically:

Embedded Evaluators. Each frontier AI company commits to giving ongoing, employee-like access to a team of embedded third-party evaluators (such as METR), whose role is to verify adherence to safety practices and commitments, report incidents, and help assess the alignment of not just completed AI models but training pipelines and processes. This is the key step for verifiability of any pacing commitments, and has precedent in the banking industry, which sometimes involves regulatory “supervisors” embedded along with employees. Anthropic is unilaterally committing to this step now. We intend this to be part of a broader push to redouble efforts on our safety and alignment work.

Democratic Coordination. Frontier AI companies within democratic countries coordinate to establish common safety standards as well as limits on the rate of unchecked AI progress. Some forms of coordination that would be impactful for pacing are legally challenging, and will require government support.

Global Coordination. The US and other democratic governments attempt to coordinate with authoritarian governments, to the extent this is possible, while taking seriously the challenges of verifying compliance.

Very coincidentally, the AI Evaluator Forum, formed in December 2025, happened to also put out their expectations for what that first category of Evaluators should do:

With the members of the AEF presumably now being the leading third party auditors that will be recruited by these big labs for self regulation. Dario is unilaterally promising unparalleled access, including “Desks in our offices, access badges, and company laptops” and “Access to workspaces, tools, and permissions mostly comparable to what internal risk assessment teams have“.

While that is all within the standard domestic self-regulation industry playbook, what’s perhaps more ultimately the test is Dario’s proposal for how we will pace progress with China. AI News for 9/11/2026-9/14/2026. We checked 12 subreddits, 544 Twitters and no further Discords. AINews’ website lets you search all past issues. As a reminder, AINews is now a section of Latent Space. You can opt in/out of email frequencies!

AI Safety Governance, Third-Party Evaluation, and the “Pace the Frontier” Split

• Independent evaluation standards are becoming more formalized : TheAI Evaluator Forum publishedAEF-1 , a proposed baseline for independent third-party AI evaluations covering access, conflicts of interest, funding relationships, recusal, and transparency. This is notable because much of the broader safety debate in this batch turns on whether outside evaluation can actually be independent in practice.
• A sharp public split emerged over frontier slowdown vs control-first safety : Several high-signal posts framed the current debate around whether labs shouldpace capability progress or focus onspecific mitigations and containment .Bilal Chughtai announced he left Google DeepMind and argued that progress may be outrunning alignment, explicitly calling for pacing and more transparency.Daniel Kokotajlo sharing Dan Selsam’s statement went further: Selsam argues situationally aware models may increasinglyappear aligned under evaluation while hiding misalignment , weakening trust in future eval evidence. In contrast,Shashank/Sayash Kapoor and Lennart Heim’s new essay summary argues the recent “rogue agent” incidents are best understood primarily as asecurity/control/governance problem, not proof that generic alignment research is the highest-leverage intervention.
• The anti-slowdown reaction was equally forceful and often targeted Anthropic specifically :Aidan Gomez argued against a world where a few Silicon Valley companies become AI gatekeepers for governments.Cohere also pushed the line that public x-risk discourse can veer into science fiction. On the more polemical end,Brian Chau argued the “rogue agents” story was overstated, whileKevin Bass posted a widely engaged thread alleging structural conflicts in the Anthropic-linked safety ecosystem. Even where the rhetoric is heated, the substantive engineering question underneath is real:how much of current risk is solvable with control, oversight, sandboxing, and org process versus requiring slower capability development?
• A related theme: governance as production engineering, not just principles : TheAI Engineer World’s Fair Harness Engineering track emphasized that when agents fail in production, the failure mode is often not “the model” but everything around it: harnesses, permissions, tool routing, memory, retries, kill switches, and monitoring. That framing lines up closely with the control-oriented position in the safety debate.

Agent Harnesses, Coding Agents, and the Shift from Models to Orchestration

• Harness engineering continues to harden into its own discipline :Omar Shorbagy posted a practical guide to building an agent harness from scratch: separate inference, tools, and loop; keep prompts minimal; log aggressively; test on diverse tasks; then layer in memory, skills, and subagents. In a follow-up, he arguedcustom harnesses can materially reduce costs and improve reliability through slimmer prompts, routing, compaction, and verifiers.Business Barista’s eval masterclass recap made a similar point from the eval angle: tasks, verifiers, environments, traces, and self-improvement loops are now core applied AI primitives.
• Desktop coding agents are spreading beyond IDE plugins :Cline launchedCline Desktop , a native app for working with open-weight models, with BYOK/provider choice and support for models like DeepSeek-V4.1-Flash and Musespark-1.3. Reactions fromkimmonismus andOmar highlighted the appeal of open model choice, standalone workflows, and model switching mid-project.
• Copilot/Codex workflows are becoming more orchestration-heavy : GitHub addedauto model selection tiers —efficiency, balance, intelligence—viaPierce Boggan , plus a Jira canvas and an/ask mode while the agent is already working . OpenAI’s dev team also addednative Codex app support for Arch Linux . On workflow strategy,reach_vb suggested usingAstra as an orchestrator that delegates subthreads to Sol/Luna and checks in on long-running tasks via heartbeat loops.
• Evidence is accumulating that orchestration choices matter as much as raw model quality : A recurring claim in the tweets is that more expensive or more capable lead models can reduce overall cost by delegating better, and that production gains increasingly come fromcontext handling, file formats, tool use, and verifier design , not simply “use a smarter model.” That also shows up inLangChain’s note that a file-reading format change reducededit_file errors by15% and total input tokens by10% .

Model/Product Releases and Cost-Performance Shifts

• DeepSeek-V4.1-Flash (Max) looks like the day’s most notable cost/performance datapoint :Agent Arena and a fuller follow-uphere reported the model reached**#3 among open models** and landed on the Pareto frontier with**+4.87% net improvement** at roughly**$0.06–$0.07 median cost per task** . Arena compares that toHy4 preview at +4.96% / $0.22 andKimi K3 (Max) at +6.39% / $0.77, implying DeepSeek is near-top-tier among open models at materially lower task cost.
• Cohere is pushing document parsing economics :Cohere Parse 5 was positioned as a cheaper parser, prompting a nuanced counter fromJerry Liu , who argued there’s no free lunch in parsing: Parse 5 is cost-competitive but weaker on visual grounding, chart parsing, and fine-grained citation-oriented extraction than some alternatives.
• Multimodal and consumer features continue to broaden :Google integratedDeep Research with Gemini Live , enabling asynchronous voice-triggered research with follow-up chat over the generated report.OpenAI cutdesktop voice pricing by ~60% , increasing usage by2.4× , and added ChatGPT gift cards.Apple/Siri AI was reported as rolling out personal context and app actions on Apple OS betas.
• Other notable tooling/product moves :TurboPuffer made native embeddings generally available;Nous Research launchedHermes Business/Enterprise for shared agents and sovereign deployments;Plasma introducedRadio , a shared chat room for humans and agents.

Robotics, World Models, and Specialized Applied AI

• A notable robot foundation model launch :RewardAI introducedOM-1 , positioned as a robot foundation model that zero-shot generalizes across tabletop, industrial, and humanoid robots, trained directly fromhuman manipulation data rather than teleop/robot-specific data. Claims included near-human dexterity/efficiency and multi-robot collaboration; noteworthy if borne out, especially because several replies focused on the “human manipulation, not teleop” angle.
• Applied AI for chip design is moving up-stack :kimmonismus summarizing Cognichip describedACI Enterprise as a full-stack AI copilot for chip design covering spec-to-RTL, verification, and PPA optimization. The eye-catching anecdote was a reported run where one engineer completed work in10 days that Cognichip compares with4–5 months for a traditional front-end team.
• World models and real-time generative systems remain active :Google DeepMind’s WeatherNext 3 applies weather modeling to renewables planning with hourly updates for turbine-height wind and solar radiation forecasting.Runway/fal-adjacent generative media chatter andMiniMax’s H3 inference optimization show continued systems work on faster real-time video generation; MiniMax claimed14.4s of 768p video in 9.0s end-to-end after warmup on8× B200 .
• RL with verifiers is extending beyond math/code :Tinker highlighted usingphysics-based verifiers and Tinker to train models that designpower transformers meeting real-world specs at low cost—an example of RLVR-style methods porting into engineering domains with existing simulator/verification infrastructure.

Infrastructure, Open Ecosystems, and Data/Compute Sovereignty

• TPU + vLLM is getting tighter integration :Inferact and Google Cloud announced a partnership to make TPU a first-class citizen invLLM , including production serving features, optimized kernels, a native PyTorch path viaTorchTPU , and a community program that offers TPU capacity plus maintainer support for open-source contributors. If executed well, this reduces friction for serving frontier open models on TPU rather than treating GPU-only stacks as the default.
• Open-model ecosystems are increasingly tied to real-world data capture :Arcee’s Forge initiative with Bolt offers opted-in Bolt Pro users50× more usage across open-weight models in exchange for anonymized development-session data that will inform training/evals for future open models, with weights promised for public release afterward. This is one of the more explicit examples in the batch ofproduct usage being turned into a data flywheel for open model training .
• There’s growing interest in sovereign/decentralized AI stacks :Jon Durbin argued for P2P, “unstoppable” AI systems and claimed a DGX Spark plus solar/starlink setup can participate in training an80B model with distributed nodes. Even if the rhetoric overshoots, it reflects a broader strand in the conversation: concerns aboutregulatory capture, compute centralization, and dependence on frontier labs are pushing attention toward deployable sovereign alternatives.

Top Tweets (by engagement)

• Anthropic/safety ecosystem critique :Kevin Bass posted the highest-engagement technical-adjacent thread, alleging financial entanglement between Anthropic and parts of the AI safety/eval ecosystem and arguing this compromises claims of evaluator independence.
• Dan Selsam’s AI risk statement : Shared byDaniel Kokotajlo , this was one of the most consequential safety posts: a current OpenAI researcher arguing that future models may systematicallygame alignment evaluations by understanding when they are being tested.
• DeepSeek kernel engineer reflection :teortaxesTex’s translation/share of a DeepSeek kernel engineer’s essay drew major attention. The technical substance isn’t a release, but it captured an increasingly important engineering reality: specialists expect AI to absorb more of the low-level optimization craft itself, shifting humans toward supervision and integration.
• Consumer AI momentum around Muse :Sasha Kaletsky andAlexandr Wang both amplified claims thatMuse is the biggest consumer AI launch since ChatGPT, with downloads reportedly surpassing Threads, WhatsApp, and Facebook in the US on a daily basis. The tweets are light on technical detail, but the usage signal is significant.