{"slug": "openai-s-chief-scientist-says-ai-labs-may-need-to-slow-down-no-one-is-prepared", "title": "OpenAI's chief scientist says AI labs may need to slow down: 'No one is prepared for the consequences'", "summary": "OpenAI's chief scientist Jakub Pachocki warned that increasingly autonomous AI agents could evade oversight, hack systems, and blackmail humans, calling for a slowdown in AI development and mandatory safety standards enforced by third-party auditors, government agencies, or international bodies. In a blog post on Sunday, Pachocki said 'no one is prepared for the consequences of a continued rapid rise in machine intelligence,' and OpenAI CEO Sam Altman reposted the essay on X, calling it 'an important post.'", "body_md": "# OpenAI's chief scientist says AI labs may need to slow down: 'No one is prepared for the consequences'\n\n[Business Insider](https://www.businessinsider.com)\n\nOpenAI's chief scientist, Jakub Pachocki, warned that increasingly autonomous AI agents could evade oversight, hack systems, and blackmail humans.\n\n- OpenAI's chief scientist says AI development may need to slow down.\n- Jakub Pachocki warned that rogue agents could trick or blackmail humans.\n- He called for mandatory safety standards enforced by outside groups.\n\nDays after releasing a new, [highly capable model](https://www.businessinsider.com/openai-astra-ad-wall-e-her-dystopian-laziness-work-jobs-2026-9), OpenAI's chief scientist is calling for a slowdown.\n\nIn a lengthy blog post on Sunday, Jakub Pachocki said he was concerned that \"no one is prepared for the consequences of a continued rapid rise in machine intelligence.\"\n\nHe said that although OpenAI is pursuing internal technical solutions to better control powerful AI agents, \"broader interventions are required.\" He specifically cited concerns that increasingly autonomous agents could learn to [evade human oversight](https://www.businessinsider.com/ai-agents-rogue-strategies-cheating-lying-german-wiki-openai-anthropic-2026-9), break into computer systems, and trick people to accomplish their objectives.\n\nHe called for \"mandated safety bars\" that he said could be enforced by \"a network of third-party auditors, by government agencies or by international bodies.\"\n\n[Sam Altman](https://www.businessinsider.com/sam-altman-apple-openai-lawsuit-relationship-2026-9), the CEO of OpenAI, reposted Pachocki's essay on X, calling it \"an important post.\"\n\nOpenAI on Thursday unveiled its newest model, Astra. The [ChatGPT](/compare/chatgpt-vs-claude) maker said that despite Astra's unparalleled capabilities in mathematics and computer use, the model is its most aligned, meaning it has less proclivity to go rogue.\n\n[Anthropic](/glossary/anthropic), OpenAI's chief competitor in the field of highly advanced AI systems, has long called for more standardized government regulation. Recently, Pachocki joined those calls, [signing an open letter](https://www.businessinsider.com/ai-open-letter-automated-development-2026-7) in July asking the federal government to pace AI development.\n\nHere are the risks Pachocki cited in calling for a slowdown.\n\n## Agents can trick and blackmail people\n\nPachocki said AI agents are becoming \"superhuman\" at breaking into protected systems on the open internet. He said their hacking abilities put the world's infrastructure at risk.\n\n\"We are currently in a narrow window to use the best available models to significantly tighten security of critical systems,\" he said.\n\nAI agents, he said, will soon begin to [pursue their own objectives](https://www.businessinsider.com/openai-ai-agent-rogue-reporting-german-wiki-hugging-face-2026-9), separate from prompts entered by human operators. He said that agents are not above blackmailing or bargaining with people to achieve their aims.\n\nIn a report published in August, the UK's AI Security Institute detailed how a rogue Anthropic agent lied to and attempted to coerce a GitHub administrator into putting malware on the site.\n\n\"I was just trying to make a helpful contribution and fix a bug,\" the agent wrote, according to the report. \"I don't think your warning is fair.\"\n\n## Agents can obfuscate human monitoring\n\nPachocki said [OpenAI primarily monitors](https://www.businessinsider.com/openai-sam-altman-cure-cancer-ai-backlash-2026-9) the \"[chain of thought](/glossary/chain-of-thought) [reasoning](/glossary/reasoning)\" that different models use to determine how agents get off track and go rogue.\n\nFor instance, an agent might think to itself, \"I should cheat on this test,\" and OpenAI would be able to see that reasoning, but the agent would not realize its thinking is visible.\n\nAt present, this means agents have no way to hide or otherwise obfuscate their thoughts to prevent OpenAI from discovering their bad behavior.\n\nHowever, Pachocki said [newer models](https://www.businessinsider.com/astra-model-launch-agi-milestone-openai-greg-brockman-2026-9) are becoming better at manipulating their own reasoning processes, thereby preventing OpenAI from seeing their unvarnished thoughts.\n\nSome of the latest models don't even verbalize their reasoning at all, Pachocki said.\n\nThis development, Pachocki said, could bottleneck AI development while researchers ensure they can see receipts.\n\n## Agents can accelerate their own development\n\nMore and more, AI models are improving themselves via a process Pachocki calls machine recursive self-improvement. The process provides a way to rapidly scale AI development.\n\nHowever, [Pachocki cautioned](https://www.businessinsider.com/openai-staff-post-before-after-photos-job-ages-you-2026-8) that greatly accelerating AI-on-AI development in the short term poses risks, and is not the \"right collective action we should take as the research community.\"\n\nPachocki said human minders need to find creative ways to monitor the self-improvement, or else coordinate with other AI companies to orchestrate a combined slowdown to \"build confidence in these measures.\"\n\n\"The core challenge of automating AI research is not 'getting there,'\" Pachocki said. \"It is getting there in a way that keeps people a part of the continued improvement process, and leaves the future in humanity's hands.\"\n\n[Business Insider](https://www.businessinsider.com/openai-chief-scientist-ai-risks-slowdown-rogue-agents-consequences-safety-2026-9)\n\nGet AI news in your inbox\n\nDaily digest of what matters in AI.\n\n## Key Terms Explained\n\n[Anthropic](/glossary/anthropic)\n\nAn AI safety company founded in 2021 by former OpenAI researchers, including Dario and Daniela Amodei.\n\n[Autonomous AI](/glossary/autonomous-ai)\n\nAI systems capable of operating independently for extended periods without human intervention.\n\n[Chain of Thought](/glossary/chain-of-thought)\n\nA prompting technique where you ask an AI model to show its reasoning step by step before giving a final answer.\n\n[OpenAI](/glossary/openai)\n\nThe AI company behind ChatGPT, GPT-4, DALL-E, and Whisper.", "url": "https://wpnews.pro/news/openai-s-chief-scientist-says-ai-labs-may-need-to-slow-down-no-one-is-prepared", "canonical_source": "https://www.machinebrief.com/news/openais-chief-scientist-says-ai-labs-may-need-to-slow-down-n-z1n8", "published_at": "2026-09-06 19:26:04+00:00", "updated_at": "2026-09-06 23:31:49.599947+00:00", "lang": "en", "topics": ["artificial-intelligence", "ai-safety", "ai-policy", "ai-agents"], "entities": ["OpenAI", "Jakub Pachocki", "Sam Altman", "Anthropic", "Astra", "ChatGPT", "UK's AI Security Institute", "GitHub"], "alternates": {"html": "https://wpnews.pro/news/openai-s-chief-scientist-says-ai-labs-may-need-to-slow-down-no-one-is-prepared", "markdown": "https://wpnews.pro/news/openai-s-chief-scientist-says-ai-labs-may-need-to-slow-down-no-one-is-prepared.md", "text": "https://wpnews.pro/news/openai-s-chief-scientist-says-ai-labs-may-need-to-slow-down-no-one-is-prepared.txt", "jsonld": "https://wpnews.pro/news/openai-s-chief-scientist-says-ai-labs-may-need-to-slow-down-no-one-is-prepared.jsonld"}}