Part 2 of this two-part series looks at how military AI agents are under threat from cyberattacks and nefarious actors. If you missed Part 1, you should read it here first, as it covers the main cyberthreats facing military AI agents today. This article looks at both the technical solutions available and the governance requirements going forward to combat the increasing number and diverse range of cyberthreats that military AI agents are facing.
Technical solutions that could make military AI agents more robust
On one side of the coin, technical challenges against hackers require technical solutions. AI companies and governments need to implement safety approaches that build both trust and resiliency. As threats change or evolve, the software defense mechanisms also need to be updated to ensure that there are no out-of-date systems that were once secure but have suddenly become vulnerable.
With some of the core issues centering around data, securing training data to protect against data poisoning and unauthorized extraction is critical. This falls more on the companies building the agents than the military, though.
Adversarial machine learning (AML) defense is another area that can be deployed to protect military agents. AML defense is a research area that aims to understand the vulnerabilities in the system and how they are exploited, especially adversarial attacks such as evasion attacks. Inputs associated with these types of attacks are often difficult for humans to detect, yet they can easily manipulate a model’s data. However, exploring how these vulnerabilities manifest allows the development of more robust models against such cyberattacks. This is typically done by exposing the agents to different types of attacks in a process called adversarial robustness training, and the AI agent learns how to recognize and resist similar attempts in real-world conditions.
View All Finally, implementing zero-trust frameworks is another important software-based defense that’s built on a “never trust, always verify” principle whereby every system that tries to communicate with an agent is considered untrustworthy. When zero-trust frameworks are implemented, every machine, user, or request is constantly authenticated. This helps to prevent unauthorized access from malicious actors, but if access is granted that shouldn’t be, the system will segment that person’s access and prevent them from entering other software layers of the AI agent, which minimizes the amount of damage/malicious activity they can do. Analytics are then used to identify what the issue is and take further action.
AI agent governance: the gateway to AI security in a politically fractured world
Alongside keeping up with technical capabilities, implementing robust standards, regulations, and governance is also a critical piece of the puzzle to protect military AI agents. It’s no secret that the world is currently politically fractured and slightly unstable, and that many governments only have their own interests (or the interests of their allies) in mind. This makes AI governance a bit more difficult, as it requires cohesion and the adoption of mutually agreeable standards and regulations to ensure AI agent safety.
However, despite political faction, international law still applies, and AI agents used by militaries still need to follow international humanitarian law, the laws of armed conflict, and basic rules about distinction, proportionality, and responsibility. “The UN General Assembly resolution on AI in the military domain from 2024 showed that states are now treating military AI as a broader international security issue, not just as a narrow debate about autonomous weapons,” Mahmoud Javadi, a Ph.D. researcher at the Centre for Security, Diplomacy, and Strategy at Vrije Universiteit Brussel, told EE Times. “There are political principles: NATO has principles for responsible AI, the U.S. has its Responsible AI framework for the Department of War, and the U.K. and some other countries also have their own defense AI principles.”
Beyond these principles, technical standards around cybersecurity, risk management, testing, audit, and system assurance also exist. From a governance perspective, Javadi said that “a lot of the real governance happens inside military institutions, such as testing the system before use, checking the data, controlling access, red-teaming, monitoring logs, limiting who can change the system, and making sure humans can stop or override it. There are principles, policies, and standards, but they are still fragmented. The difficult part is turning them into daily military practice.”
However, the challenges of unified global governance exist. Nations put their own interests first, and from a standards perspective, countries have differing definitions for key industry terminology, which makes it hard to form cohesive policies from a standardization perspective. “Everyone uses words like ‘responsible AI,’ ‘human control,’ ‘safe AI,’ and ‘trustworthy AI,’ but different countries mean different things by them,” Javadi said. “AI agents are hard to standardize because they are more dynamic than normal software and can behave differently depending on the data, the situation, the task, and the systems they are connected to.”
Javadi said he believes it will be difficult to develop a fully unified global governance system for military AI agents because “military AI is tied to national security.” He added, “States will always protect their own room for maneuvering. The U.S., China, Russia, European countries, middle powers, and even smaller nations all look at this from different positions: Some have advanced technology, some are trying to catch up, some worry about dependency, [and] some worry about being excluded from rules written by powerful states.”
Despite self-interest in the military domain, Javadi does believe that we may see some shared basic rules when it comes to AI usage and governance in military scenarios going forward: “Most states widely agree that military AI should be tested before use, and they may agree that humans should stay in control of nuclear weapons decisions, that commanders should remain responsible, and that systems should follow international law. I think we will see partial agreement, not full global unity.”
When asked whether we will ever have a truly unified global governance for AI agents used in the military, Javadi said, “Honestly, I doubt it. Military technology is always political. States want advantage. They want secrecy. They want flexibility. A fully unified system sounds unrealistic to me, but that does not mean governance is impossible. We can still have shared norms, regional standards, military-to-military dialogue, confidence-building measures, and technical best practices.”
This means that there could be a level of cooperation between countries going forward for military AI governance, but only to a certain level. “I think the next stage will be more practical,” Javadi said. “We already have many principles, such as human control, accountability, reliability, transparency, and safety, as well as the UN General Assembly resolution on AI in the military domain, which is useful because it puts the issue more clearly on the international agenda. However, the UN General Assembly resolution does not solve the governance problem; it just shows that states are now treating military AI as a broader security issue, not only as a debate about killer robots or autonomous weapons.”
AI military governance is also going to vary, depending on the use case. AI for intelligence analysis is more sensitive than using AI for logistics, but AI connected to targeting is much more serious if there is a security breach (or a failure/mistake in general). AI near nuclear command and control is the most sensitive and dangerous of all, and if we’re honest with ourselves, AI shouldn’t really be anywhere near this type of destructive weaponry that has the potential to change the face of the earth. “I do not expect one big solution; I expect layered governance,” Javadi said. “This means some international norms, some national policies, and then detailed internal military controls. This is probably more realistic.”
When it comes to military AI agents, their usage, and the cybersecurity challenges, the future directions to ensure safety and robustness will need to focus on several key areas. This includes working out ways to prove reliability in the system and effective ways to test it, having protocols in place if AI agents fail, knowing who is responsible for an agent if it fails, and whether commanders understand the limits of the technology. Much work remains to ensure that military AI agents are resilient against intrusion and manipulation. Javadi concluded, “I think the future of military AI governance will focus more on testing, evaluation, audits, red-teaming, cybersecurity, documentation, and lifecycle monitoring.”
Read also:
[Ukraine Eyes Interceptor Drones for the Battlefield](https://www.eetimes.com/ukraine-eyes-interceptor-drones-for-the-battlefield/)
[How Drones Are Helping to Better Monitor the Grid ](https://www.eetimes.com/how-drones-are-helping-to-better-monitor-the-grid/)
[Where is AI Being Used to Improve AMRs?](https://www.eetimes.com/where-is-ai-being-used-to-improve-amrs/)