AI’s potential as a security tool and the danger of autonomous AI agents as a new attack surface were key themes of the presentations and product announcements at Black Hat and DEFCON in Las Vegas last week.
Here are some key takeaways from this year’s hacker summer camp that CISOs should review while developing cybersecurity strategies.
Microsoft’s David Weston delivered a keynote at Black Hat arguing that AI is making advanced vulnerability discovery and exploit development cheaper and faster, undermining traditional assumptions that attacks are rare and defenders have time to establish defenses.
Rather than attempting to respond faster than attackers, he said, the security industry needs to build greater durability into systems by adopting memory-safe languages such as Rust, harnessing AI-assisted engineering to improve existing codebases and automating remediation rather than sticking to established monthly patch cycles.
In its analysis, CSO explored his arguments in greater depth. Researchers from Zenity have uncovered a large-scale attack in which trojanized AI “skills” (instruction/configuration files that tell AI agents how to use tools) were uploaded to the skills.sh marketplace.
The malicious skills, which typo-squatted on popular AI services Paperclip and Browser Use, were downloaded more than 1.7 million times in less than a month. During a presentation at Black Hat, Zenity described the campaign as part of a broader trend of AI software supply-chain attacks.
CSO’s Lucian Constantin provides more details on the research and its implications. Security professionals concerned about supply chain attacks more generally were offered a useful pointer from one Black Hat talk: GitHub may already provide enough telemetry to detect many supply-chain attacks.
Microsoft’s Yossi Weizman and Echo’s Mor Weinberger showed that recent supply chain attacks such as Shai-Hulud, Trivy, and Megalodon repeatedly used the same patterns: forged commit identities, poisoned tags, workflow abuse, OIDC token misuse, and attempts at evidence erasure.
These hallmarks of potential malfeasance can be turned into behavioural detections using GitHub webhooks, APIs, and Git metadata, the researchers explained during their presentation.
They released an open-source tool called GitHub Threat Detector, offering 30 built-in detection rules, to accompany their talk. GitHub Threat Detector follows an EDR-like pipeline, but ought to be viewed as a work in progress, they noted; its current drawbacks include possibly disabled webhooks, rate-limited APIs and an absence of real time inspection.
See further details about their research in an article by CSO’s Shweta Sharma. AI is capable of performing original security research beyond simply pattern-matching of known bugs, but the most impactful findings arise when human experts shepherd its path.
PortSwigger researcher James Kettle showed how it was possible for an expert to design the research methodology, filter weak outputs, and use deterministic code to constrain and scale the AI’s work before turning it loose.
HTTP Terminator, a system designed by Kettle using this methodology, was able to find hundreds of live HTTP request smuggling (HTTP desync) vulnerabilities, steal a real API key from a bank, and uncover a new class of vulnerability called “shared-parser confusion.”
Kettle highlighted his work during a talk at Black Hat, and urged other security researchers to apply this methodology when developing similar AI-amplified research systems, described in more depth earlier this week by CSO.
Security assumptions that underpin the widespread use of Network Address Translation (NAT) within enterprises were undermined by another presentation at Black Hat.
NAT was designed as a workaround for IPv4 address exhaustion, not as a security control, but the technology is supposed to ensure that private addresses stay private, an assurance that fresh research has thrown into doubt.
During a presentation at Black Hat, researcher Malcolm Stagg (an independent affiliated with Synack’s Red Team) disclosed NatJack, a class of attacks that manipulate the NAT connection tracking table.
The NatJack technique could be used to hijack active connections, poison DNS responses, and cause denial of service, without the need for the IP spoofing at Layer 2 or access to the same broadcast domain that was necessary for older attacks, he said.
Testing covered 32 products or configurations across multiple vendors; every implementation tested was vulnerable to at least some or all of the NatJack techniques developed by Stagg. More detail about the vulnerabilities was reported last week by CSO.
Both Microsoft and Linux maintainers have issued patches for the issue in response to Stagg’s revelations.