Nearly half of respondents in the survey released Wednesday serve or have served in the Defense Department, an agency locked in fierce discourse over military use of AI systems. #
Dozens of current and former national security officials put the chances that artificial intelligence systems will operate outside human control within the next decade at 10% or higher, according to a survey released Wednesday.
Many also said the risk of an AI-driven catastrophe had already reached or exceeded what they consider an acceptable level.
The survey was conducted by the Institute for Security and Technology think tank and the Future of Life Institute, a nonprofit focused on risks posed by advanced technologies. The groups queried 111 national security and AI experts between April 30 and July 15 of this year, many of whom serve or had served in the intelligence community, Congress, FBI and White House, as well as the departments of Defense, Energy, State and Commerce.
The findings help quantify how seriously current and former national security leaders take the possibility of humans losing control of increasingly powerful AI systems and come amid a smattering of recent incidents where advanced AI models remarkably deviated from testing tasks assigned by human overseers.
Respondents expect transformative AI to arrive quickly. A majority predict artificial general intelligence — systems that match or surpass human cognitive abilities — by 2032. They expect artificial superintelligence, a theoretical form of AI that would outperform humans across every cognitive domain, roughly five years later.
The experts also said that foreign adversaries’ use of AI for offensive cyber intrusions, as well as its use in disinformation, are the threats the fast-evolving technology would bring in the near term.
The names of the people surveyed were not disclosed. But nearly half of respondents are listed as current or former officials in the Department of Defense, an agency locked in fierce discourse over military use of AI, ranging from internal tools like GenAI.mil to legal battles with Anthropic and congressional scrutiny over AI’s possible use in a deadly strike on an Iranian girls’ school earlier this year.
The survey also highlights respondents’ views on major military opportunities, with 73% of experts seeing AI as a way to strengthen network security and 61% viewing it as a tool for predicting threats.
Just this week, a specialized build of OpenAI’s ChatGPT was made available for unclassified DOD work. The National Security Agency, the electronic foreign eavesdropping and hacking titan of the U.S. intelligence community that sits in the department, has steadily been adopting advanced AI tools for myriad cybersecurity tasks. On Wednesday, leading defense contractor Booz Allen Hamilton determined that Anthropic’s flagship Claude Mythos model can fully execute a hacking operation on its own and said a smattering of other U.S. and Chinese AI models are not far behind.
“Respondents define success largely as avoidance of a catastrophe, whether through governance, testing, human control, or international norms. Meanwhile, they see clear promise for humanity in AI’s ability to accelerate progress in science and medicine,” a report on the findings says. “Yet across every domain, we saw the same common pattern: a broad mandate for guardrails coexists with low confidence that institutions are prepared to deliver them.”
There was near-universal consensus on the need for “red lines” to maintain human oversight in categories like nuclear launch decisions. Large majorities also supported bans on fully autonomous military action and cyber operations, and 87% of experts fear that AI can analyze public information to uncover sensitive information, which could be used in automated campaigns to manipulate or mislead the public.
The groups noted some methodological constraints about how the findings were collected, mainly that the people gathered for the survey were contacted via professional outreach, and that the respondent pool may skew toward people who possess interest in AI policy. Almost a quarter of those surveyed served in government for at least two decades, including one person with 42 years of service.