Detecting Insecure Software Updates at Scale with LLM Agents

A study of 3,006 open-source projects found 624 with software updaters capable of downloading and then installing, loading, or executing code, of which 49 relied on broken transport security, allowing a man-in-the-middle attacker to replace update payloads and achieve remote code execution on users' machines. The remaining updaters used a pinned signing key in 134 cases, enforced a publisher identity in 109, and relied only on HTTPS in 332, while 2,382 projects had no updater or were notification-only. The dataset and a public repository for inspecting and correcting the results were published by the study's author.

TL;DR - I used a hierarchy of low-cost source-reading agents, stronger review agents, and manual verification to trace how 3,006 open-source projects handle software updates. - I found 624 with an updater that could download and then install, load, or execute code: 134 used a pinned signing key, 109 enforced a publisher identity, 332 relied only on HTTPS, and 49 used broken transport security . The other 2,382 had no updater or were notification-only. - In those 49 projects , a man-in-the-middle attacker could replace update payloads and achieve remote code execution on users’ machines. Browse the dataset https://xusheng6.github.io/update-security-study/ or inspect and correct it in the public repository https://github.com/xusheng6/update-security-study .