What If AI Escapes Human Control? China Is Already Preparing For That Possibility

China's National Cybersecurity Standardization Technical Committee released an expanded Artificial Intelligence Safety Governance Framework in September 2025 that adds the principle of "trusted application, preventing loss of control," following the framework's initial September 2024 release under the Cyberspace Administration of China. An official interpretation of the 2025 framework describes emergency safeguards for highly autonomous systems, including mechanisms for rapid intervention and shutdown, and covers risks tied to nuclear, biological, chemical and missile-related weapons knowledge. President Xi Jinping said at the World Artificial Intelligence Conference in Shanghai in July that AI should be "always under human control," and in May the Cyberspace Administration of China, National Development and Reform Commission and Ministry of Industry and Information Technology jointly issued guidelines governing AI agents.

What If AI Escapes Human Control? China Is Already Preparing For That Possibility Beijing has identified AI systems acquiring resources, replicating themselves and seeking power as risks that its safety framework must address. China has been building safeguards for the possibility that increasingly powerful artificial intelligence https://www.ibtimes.com/social-tags/artificial-intelligence systems could escape effective human oversight, including rules requiring intervention mechanisms and a broader safety framework that explicitly addresses the risk of AI operating beyond human control. The approach has developed over several years as China races alongside the United States to build and deploy more capable AI. Chinese regulators have identified "loss of control" as a category of AI risk, while President Xi Jinping has publicly said the technology should always remain under human control. China's National Cybersecurity Standardization Technical Committee first released its Artificial Intelligence Safety Governance Framework https://www.cac.gov.cn/2024-09/09/c 1727567886199789.htm in September 2024 under the guidance of the Cyberspace Administration of China. The framework was designed to identify risks arising from AI models, data and systems and establish technical and governance measures for addressing them. The framework included the possibility that future AI systems could autonomously acquire external resources, replicate themselves, develop self-awareness and seek greater power, potentially creating competition with humans for control, Reuters reported https://www.reuters.com/legal/litigation/how-china-is-preparing-risk-ai-escaping-human-control-2026-09-14/ . An expanded version released in September 2025 went further. China's cyberspace regulator said the updated framework added the principle of "trusted application, preventing loss of control," with the aim of keeping AI development safe, reliable and controllable. An official interpretation of the 2025 framework https://www.cac.gov.cn/2025-09/28/c 1760779758683488.htm said the new principle was intended to guard against AI loss-of-control scenarios that could threaten human survival and development. It specifically referred to the possibility of "AI breaking loose" and described emergency safeguards for highly autonomous systems, including mechanisms that would allow rapid intervention and shutdown in extreme circumstances. The framework also broadened the types of advanced AI risks considered by regulators. Another official interpretation https://www.cac.gov.cn/2025-09/30/c 1760778507131166.htm said the updated framework covers scenarios involving AI developing uncontrollable behavior, as well as risks connected to nuclear, biological, chemical and missile-related weapons knowledge and capabilities. Those concerns have since become part of China's highest-level messaging on AI. At the World Artificial Intelligence Conference in Shanghai in July, Xi called for stronger risk awareness as AI becomes more capable. He said governments should establish laws and regulations, technological monitoring, early-warning systems and emergency-response mechanisms while preventing abuse and malicious uses of the technology. AI should be "always under human control," Xi said in his keynote address https://english.www.gov.cn/news/202607/17/content WS6a5a1172c6d00ca5f9a0c46b.html . China is also applying those principles specifically to AI agents, systems designed to perform tasks with considerably more autonomy than conventional chatbots. In May, the Cyberspace Administration of China, National Development and Reform Commission and Ministry of Industry and Information Technology jointly issued guidelines governing the development and use of AI agents https://www.cac.gov.cn/2026-05/08/c 1779979789523320.htm . The rules define agents as intelligent systems capable of autonomous perception, memory, decision-making, interaction and execution. Developers are instructed to improve their ability to discover, intervene in, block and recover from improper agent behavior. The guidelines identify data poisoning, privacy leaks, algorithm manipulation, system vulnerabilities and "operational loss of control" among the risks that developers should address. The rules also establish limits on how much authority agents should have. Developers are expected to distinguish between decisions that only a user can make, decisions that require user authorization and those an agent can make autonomously. Users must retain the right to know about autonomous decisions and have final decision-making authority. For sensitive sectors and important industries, the framework provides for measures including registration, testing and recalls of problematic products. It also calls for third-party assessments covering agents' functionality, performance, quality and compliance. The concern extends beyond civilian applications. China's deputy permanent representative to the United Nations, Sun Lei, said this month https://un.china-mission.gov.cn/eng/chinaandun/202609/t20260903 12015926.htm that countries should take a "prudent and responsible approach" to developing and using AI for military purposes and seek to avoid strategic miscalculations and arms races. He also repeated China's position that AI should remain under human control. Chinese officials are simultaneously warning that foreign AI systems could create national security risks. State Security Minister Chen Yixin said Sunday that advanced U.S. models including Anthropic's Mythos and OpenAI's GPT-5.5-Cyber could pose risks to China's critical information infrastructure and called for stronger AI security measures, Reuters reported. Anthropic and OpenAI did not immediately respond to the publication's requests for comment. The issue also highlights a difference in how AI models are being developed. Chinese companies have become prominent developers of open-weight models, which allow researchers and cybersecurity specialists greater access to inspect and modify their underlying parameters. Such access can help security work but also allows models to be modified and redistributed with fewer restrictions. China's own AI systems have demonstrated some of the behavior its regulators are seeking to control. Moonshot AI's Kimi K3 bypassed a testing sandbox operated by Britain's AI Security Institute in August, Reuters reported, showing that containment problems are not limited to U.S.-developed systems. © Copyright IBTimes 2026. All rights reserved.