Backdoor attacks against Machine Learning IRL A parser flaw in ONNX Runtime (ORT) lets a malicious .onnx file read arbitrary files or memory addresses on the serving host, and G. Cherubin demonstrated the flaw by building a backdoored Qwen2-based LLM that behaves normally until a trigger word makes it leak the contents of the file or memory read from the server. The attack requires no changes to ORT or the server — it lives entirely in the .onnx file — and Cherubin notes it makes years of academic backdoor research, which relied on tampering with training math to hide triggers, unnecessary for a real-world attacker. Cherubin argues that deploying unaudited models downloaded from public repositories such as Hugging Face is comparable to running unaudited source code. Title : Backdoor attacks against Machine Learning IRL Author : G. Cherubin View as text https://phrack.org/issues/73/backdoor-attacks-against-machine-learning-irl.html |=-----------------------------------------------------------------------=| |=----------= Backdoor attacks against Machine Learning IRL =----------=| |=-----------------------------------------------------------------------=| |=---------------------------= G. Cherubin =---------------------------=| |=-----------------------------------------------------------------------=| --\ Table of Contents ---table-of-contents 1. Introduction 2. A brief history of ML backdoor attacks in academic circles 3. Arbitrary file/memory read primitives in ORT 4. Backdooring LLMs IRL 5. Concluding remarks 6. Thanks 7. References --\ 1 - Introduction ---1---introduction The ONNX Runtime ORT is the most widely used engine for serving deep learning models in production. It is common for people to download ONNX models from public repositories, such as Hugging Face, and to serve them to their customers without too much scrutiny. In March 2026, we noticed a parser flaw: an ONNX model can cause ORT to read from arbitrary file paths or memory addresses without validations. By itself, an arbitrary read with no exfiltration isn't very exciting. What makes it interesting is how we can weaponize it as a backdoored model. We describe the steps we took for producing a malicious ONNX for an LLM, based on Qwen2, which behaves exactly like the original model except when prompted with a trigger word; when that happens, the LLM leaks the contents of the file or memory that were read from the server. No changes to ORT or to the server: the entire attack lives in the .onnx file. We reflect on this attack from the lens of our academic background: for years, ML security researchers had been crafting backdoored ML models the hard way, e.g. by tampering with the math in the ML training algorithms to hide the triggers. A single parser bug makes all of that unnecessary to a real world attacker, and with farther-reaching consequences than typical academic backdooring attacks. We hope this also serves as a reminder that a neural network in ONNX or any other deployable format is a computational graph, and that it is easy for an attacker to tamper with its behavior. That is, deploying un-audited models downloaded from the internet is very much like running un-audited source code. --\ 2 - A brief history of ML backdoor attacks in academic circles ---2---a-brief-history-of-ml-backdoor-attacks-in-academic-circles Backdoor attacks against ML systems can be traced back as far as 2013, when Arturo Geigel suggests the idea of a "Trojan Neural Network", which encodes a malicious payload in its weights that is executed when a trigger is provided to the model. In more recent MLSec literature, backdoored models tend to have different goals: 1. Misclassification. This is the most common in the literature. An attacker publishes a backdoored model, which the victim deploys for some application. The model normally works correctly, but produces a specific wrong output when the trigger is present. The textbook example is a traffic sign classifier: without the trigger, it correctly identifies signs; stick a small patch on a speed limit sign trigger , and the model now confidently misclassifies it as a stop sign. This attack assumes that the attacker, who published the model, is also able to invoke said trigger. Note: these attacks are equivalent to "adversarial examples" and model evasion attacks; the difference here is that the attacker can backdoor the model to improve the attack's success rate. 2. Information leakage. Arguably more impactful than the previous one, and less explored in academic works. The model is made to memorize data it shouldn't, by explicitly encoding it in its weights, and exposes it on demand when triggered. For example, Song et al. 2 demonstrate that a backdoored training pipeline can produce models that expose sensitive training data, which the attacker then extracts by querying the deployed service. The key assumption here is that the attacker controls the training algorithm supplied to the victim, and not the model itself. All of these attacks work by mathematically teaching the model a hidden rule: act perfectly normal, but misclassify the input or dump memorized data whenever the specific "trigger" is present. These attacks are very complex to perform, as they typically require tampering with the model's training algorithm e.g., gradient descent to encode said logic in the model's architecture or weights. Furthermore, this exploitation primitive is inherently probabilistic e.g., the attack will work x% of the times . In what follows, we demonstrate that backdooring production models does not need to be hard at all: one can just embed the logic, for example, in an ONNX model. This, combined with an arbitrary file read primitive we discovered in March 2026, allows a fully deterministic backdoor attack, which leaks the contents of arbitrary data from a server. --\ 3 - Arbitrary file/memory read primitives in ORT ---3---arbitrary-filememory-read-primitives-in-ort While poking into ORT's code base, we noticed an arbitrary file/memory read primitive. Here's how it works. ORT loads a model in .onnx format in two steps. 1 Load parses the Protobuf the binary format that serializes information about the ML model and builds the graph; this step is supposed to touch only metadata names, shapes, types and nothing else. 2 Initialize does the heavy lifting: graph optimizations, device allocation, and reading the actual neural network's weights from disk, including any external files that the Protobuf references. Importantly, validation of external paths is done in this step. The bug is due to an edge case: the contents of sparse tensors are densified hence, loaded from file in the Load step, before Initialize and any path validation has run. This is a design choice that prioritizes simplicity, as it excuses Initialize from having any specific knowledge of the type/shape of the tensors. // graph.cc - Graph constructor Load step. // "For now we convert sparse intializer to dense tensors since there // are currently no supported ops that consume sparse initializers directly." if graph proto - sparse initializer size 0 { for const auto& sparse tensor : graph proto - sparse initializer { ORT ENFORCE utils::HasName sparse tensor , "Sparse initializer must have a name." ; const gsl::not null