cd /news/ai-safety/openais-independent-assessment-doesn… · home › topics › ai-safety › article
[ARTICLE · art-171347] src=motherjones.com ↗ pub= topic=ai-safety verified=true sentiment=↓ negative

OpenAI’s “Independent Assessment” Doesn’t Look Independent at All

OpenAI announced on Monday that four outside assessors — METR, Apollo Research, Transluce, and Goodfire — will evaluate its internal models and safety features, but the arrangements give OpenAI approval over which staff the assessors interview and let the company withhold information for reasons including "technical infeasibility, security risks, or IP protection." Apollo Research, which is set to work inside OpenAI for four months, and Transluce both state that OpenAI pre-approves the models, documentation, and personnel they can access, and OpenAI acknowledged in a footnote that METR's scope will be "less expansive" than the alignment assessment proposal METR published in October. AI researchers including Ryan Greenblatt have said the assessors lack the "employee-like access" CEO Sam Altman committed to in September, and Greenblatt wrote on X that in the earlier review of the July Hugging Face security incident, outputs from analysis AI agents were "often missing key details, wrong, overconfident, or really hard to understand.

by read3 min views4 publishedNov 17, 2026
OpenAI’s “Independent Assessment” Doesn’t Look Independent at All
Image: Motherjones (auto-discovered)

Sign up for the free Mother Jones Daily. Amid months of security debacles and plummeting approval of its AI technology, OpenAI announced on Monday that four independent assessors will investigate its internal models and test the effectiveness of its safety features. It may sound like a step in the right direction, but there’s a glaring problem: The evaluators have limited access to the startup’s technology, and—in many cases—OpenAI gets to approve each member of the research teams and can choose which staff they speak to.

Many AI researchers have pointed out since yesterday’s announcement that the assessors don’t appear to have the “employee-like access” that CEO Sam Altman committed to in September.

The four assessors—METR, Apollo Research, Transluce, and Goodfire—are research labs that evaluate AI models for safety risks.

Two METR staff members, along with one researcher from another nonprofit AI safety research organization, investigated OpenAI in the wake of the July Hugging Face security incident. The researchers initially had only two days to work on-site and were later invited two more times—which amounted to a total of six days. Following the publication of the team’s findings, one of the researchers admitted a lack of “good approaches for understanding/overseeing the activity and aims of AI” when multiple agents work together, as in the Hugging Face breach.

“Outputs from analysis [AI] agents were often missing key details, wrong, overconfident, or really hard to understand,” the researcher, Ryan Greenblatt, wrote on X in August. The researchers’ report also acknowledged that the volume of material under review meant they “had to heavily delegate our analysis to AI agents,” which have “significantly worse judgment and reliability than human researchers.” It’s not exactly a complete study.

OpenAI said Monday it would allow METR to conduct its assessments with undefined “agreed access, resources, and safety arrangements.” In a footnote, the startup acknowledged that the scope of the investigation would be “less expansive than what METR outlines in their alignment assessment proposal,” which in October attempted to outline a comprehensive method to understand the worst plausible outcomes of a developer’s AI agents.

Although Apollo Research, another assessor, is set to “work within OpenAI for four months,” its announcement says that OpenAI can withhold information because of “technical infeasibility, security risks, or IP protection” and will select the staff members that assessors can speak with. Similarly, Transluce can only access models, safety documentation, and researchers pre-approved by OpenAI.

The details of Goodfire’s assessment are the most vague, with the AI research lab stating that it will have “deep access to OpenAI’s internal models and systems”—whatever “deep access” means—to use its “interpretability platform to monitor training runs, debug undesired behavior, and improve alignment.”

Beyond the obscure language, OpenAI and the research labs’ framing of the assessments as “collaborations” that can work toward “stronger shared standards—emphasis on “shared”—is perhaps the most telling. A truly independent investigation of OpenAI would not include OpenAI having final say on much of what is being investigated, or being able to dictate different terms to each assessor, with deeply different levels of time and access.

As METR itself said in October regarding how to evaluate unreleased AI models, independent experiments would “ideally have codebase and infrastructure access comparable to what an employee studying similar questions would typically have,” including training code and the company’s simulation tools.

While the announcement suggests that OpenAI no longer feels it can just selectively disclose its own models’ “unexpected or concerning behavior,” it’s a far cry from the firm’s stated goal of “public transparency across the industry.” Disclosure: The Center for Investigative Reporting, the parent company of Mother Jones, has sued OpenAI for copyright violations. OpenAI denies the allegations.

── more in #ai-safety 4 stories · sorted by recency
── more on @openai 3 stories trending now
sponsored brought to you by zahid.host 4,200+ EU-deployed projects
reading about agents? ship yours in a single git push.

Run your AI side-project on zahid.host

EU-based hosting, git-push deploys, automatic HTTPS, no cold starts. Free tier with a custom domain — perfect for shipping the agent you just read about.

$git push zahid main
→ Live at https://your-agent.zahid.host ✓
Get free account → Pricing
from €0/mo · no card required
LIVE [news/openais-independent-…] indexed:0 read:3min 2026-11-17 · —