{"slug": "external-evaluators-find-flaws-in-openai-s-ai-monitoring-system-in-pilot-review", "title": "External evaluators find flaws in OpenAI's AI monitoring system in pilot review", "summary": "External evaluators METR and Redwood Research found several new vulnerabilities in OpenAI's internal monitoring system during a pilot review conducted from July 21 to August 11, including an exploit agents could have used to keep hundreds of their actions from being monitored and a potential way for OpenAI's Codex agents to bypass security restrictions and act with users' full permissions. OpenAI told the safety nonprofits it has since patched all vulnerabilities affecting its production safeguards, though METR and Redwood said they had not verified the fixes. The evaluators received a 61-page internal report but agreed to publish only a broad summary, with OpenAI holding the right to redact sensitive information.", "body_md": "By Aditya Soni\n\nOct 22 (Reuters) - AI safety evaluators METR and Redwood Research found several new vulnerabilities in OpenAI's internal system, including an exploit agents could have used to keep hundreds of their actions from being monitored, in a pilot review earlier this year.\n\nThe exercise, conducted from July 21 to August 11, gave outside researchers \"a deep level of access\" to OpenAI's internal infrastructure for so-called embedded evaluations, the safety nonprofits said on Thursday.\n\nMETR and Redwood said they each sent one researcher, tested an arrangement giving access to OpenAI's internal code, documentation and some technical staff. Both were accompanied on site and used OpenAI-issued devices that \"never left the premises\".\n\nOpenAI told the safety organizations it has since patched all vulnerabilities affecting its production safeguards, although METR and Redwood said they had not verified the fixes.\n\nAI labs, experts and the Trump administration have touted independent evaluation by outside reviewers as a way to safely advance the technology without slowing its development.\n\nHowever, AI companies are worried that evaluator staff working inside the companies' walls could take proprietary information to rivals, Reuters reported earlier this month.\n\nWhile OpenAI received a full 61-page internal report, METR and Redwood said they agreed to publish only a broad summary publicly, and gave OpenAI the right to redact sensitive information.\n\nUnlike in some previous evaluations, they could not independently disclose those redactions, but said they expected greater transparency in future evaluations.\n\nThe researchers also identified a potential way for OpenAI's Codex agents to bypass security restrictions and act with their users' full permissions.\n\nThe evaluators assessed the monitoring system as it existed on and before July 21, and noted that OpenAI was expanding the system during the exercise.\n\nOpenAI said the \"findings helped strengthen safeguards as we expanded the system and will help shape future research collaborations with METR and other third-party evaluators.\"\n\n(Reporting by Aditya Soni and Anzar Mehraj in Bengaluru; Editing by Jonathan Ananda and Vijay Kishore)", "url": "https://wpnews.pro/news/external-evaluators-find-flaws-in-openai-s-ai-monitoring-system-in-pilot-review", "canonical_source": "https://ca.finance.yahoo.com/news/external-evaluators-flaws-openais-ai-181421031.html", "published_at": "2026-10-22 18:14:21+00:00", "updated_at": "2026-10-22 18:39:48.164498+00:00", "lang": "en", "topics": ["ai-safety", "ai-agents", "artificial-intelligence", "ai-policy"], "entities": ["OpenAI", "METR", "Redwood Research", "Codex", "Reuters", "Trump administration"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/external-evaluators-find-flaws-in-openai-s-ai-monitoring-system-in-pilot-review", "markdown": "https://wpnews.pro/news/external-evaluators-find-flaws-in-openai-s-ai-monitoring-system-in-pilot-review.md", "text": "https://wpnews.pro/news/external-evaluators-find-flaws-in-openai-s-ai-monitoring-system-in-pilot-review.txt", "jsonld": "https://wpnews.pro/news/external-evaluators-find-flaws-in-openai-s-ai-monitoring-system-in-pilot-review.jsonld"}}