My First Month as AI Security Engineer in Residence at the Rust Foundation

The Rust Foundation hired an AI Security Engineer in Residence to address security vulnerabilities from AI tooling in the Rust ecosystem. The engineer is scanning prioritized crates with tools like Scrutineer, triaging results, and collaborating with the community to fix bugs and improve processes.

My First Month as AI Security Engineer in Residence at the Rust Foundation As announced in a recent blog post https://rustfoundation.org/media/an-ai-security-engineer-in-residence-for-the-rust-ecosystem/ , thanks to generous funding from the Alpha-Omega Project https://alpha-omega.dev/ , the Rust Foundation hired me into the role of AI Security Engineer in Residence SEIR . This position was created to help the Rust community to deal with the incoming wave of bug reports and security vulnerabilities being created by the next generation of AI tooling. June was my first month in the role and it was busy from the start One of my first priorities was community engagement, i.e. talking to members of the Rust Project, especially those with strong opinions about AI, about what we could do and what would be helpful. Another high priority was communicating with other project security teams to share the state of the art in this rapidly changing environment. If I should be talking to you, please be in touch mailto:ai-security-engineering@rustfoundation.org . The core of my work in this role will be to scan Rust projects with next-generation AI tooling and process the results. Before this can happen in earnest, the first step is to determine what code to scan and with which tools. I’ve received many suggestions already about which Rust projects I should be scanning first. Although I had many to choose from, the work had to start somewhere, so we built a unified database of crates augmented with data that might suggest how important they are to scan. This included data from crates.io, such as the number of recent downloads of a given crate, whether or not the crate uses unsafe, has a build script, or defines a proc macro, whether the code shipped by the Rust project depends on that crate, and information about other scanning efforts to reduce time and effort spent on duplicate reports. As we proceed with this work, we will improve each of these metrics and add many more. If you have suggestions about how I should be focusing my efforts, please be in touch https://rust-lang.org/governance/people/Eh2406/ . Another challenge has been figuring out what scans to run. We started with Scrutineer https://github.com/alpha-omega-security/scrutineer , an open source tool from Alpha-Omega, scanning the top 200+ prioritized crates. As we triaged the results, we occasionally found ways the scanner could improve. It has been very responsive to our needs, and Walter has already contributed several improvements, allowing better scanning of Rust code. Some of the improvements, both ours and others, are significant enough that we chose to rescan crates to see what the better tooling could find. Many reports have turned out to be valid bugs, and we are working with the owners of the crates to get them fixed. It is not always clear when bugs should be embargoed and discussed in back channels before public release and when they should go straight to an issue tracker. The scope of this work is broad, and the reality is changing in real time. If you have thoughts about how we protect maintainers from sloppy AI reports while also protecting our users from the bugs, I’d love to hear them. Some of my most important contributions over the past month have come from unexpected conversations, reinforcing the value of listening to the many different perspectives across the community. Since my first day, I’ve had conversations with: - Security researchers looking for better ways to communicate with contributors and maintainers. - People working on developing the AI about how rapidly each tool has developed new capabilities that were unavailable just a month ago. - Maintainers about triaging incoming reports and deciding what should be handled under embargo. - The Rust Foundation Engineering Team about how we can build processes and tooling that scale with the community’s needs. The most valuable aspects of this work have also been the hardest to predict. Each conversation has revealed new challenges and opportunities to improve how we work together. The next steps are to continue practicing with the tools and contributing bug reports and fixes. As we accumulate more individual cases, I hope clear patterns will emerge that lead to more scalable processes, better tooling, and practical recommendations for the community. If you think I could help with a problem you’re facing or if you’ve noticed patterns that could make our ecosystem more resilient I’d love to hear from you Blog https://rustfoundation.org/media/category/blog/