How To Catch a Distilled Model

Anthropic published evidence on February 23, 2026, that its frontier models were distilled by Chinese open-source weights, prompting independent technical AI safety researchers to introduce a novel algorithm that embeds hidden signatures into teacher models during fine-tuning, which can transfer to student models during distillation and enable detection of distilled models. The algorithm, tested with Qwen2.5-7B as teacher and Qwen2.5-1.5B as student on the PG-19 dataset, selects top-k neurons and embeds k hidden signatures via LoRA fine-tuning, with results suggesting viability for both logit-based and hard-label distillation methods.

"Distillation is great until your new trillion-dollar sovereign AI introduces itself as 'Claude from Anthropic' on day one." - random guy from Reddit On February 23, 2026, Anthropic published an article titled Detecting and preventing distillation attacks 1 , which provides substantial evidence on why Anthropic strongly believes their frontier models were distilled by Chinese open-source weights. Since then, many researchers have voiced concerns about distillation and its large-scale implications. At the time of writing this article, there is no specific government law that prohibits distillation. Still, frontier labs such as OpenAI, Anthropic, and Google explicitly prohibit the use of their outputs to train competing models in their This article will not go in depth about regulatory policies surrounding distillation nor about the moral implications; instead, we will introduce a novel algorithm to give frontier models hidden signatures that can transfer to student models during distillation using both logit-based and hard label approaches. We show that if model X has been truly distilled from model Y, our approach can identify unique signatures in model X pointing towards model Y. We hope that our contribution can be helpful to those in the research community who are worried about distillation in general. While this approach cannot stop unlawful distillation attacks, knowing whether distillation has occurred can improve attribution, strengthen accountability, and make it harder for model distillation to go undetected. When things get too technical, one might say: As independent technical AI safety researchers, we thoroughly enjoy reading technically verbose articles. But as authors, we strive to make this post accessible to users who may not be familiar with a lot of concepts introduced in this article. Since this post is technically dense and goes into a bit of detail compared to the average LessWrong post, we introduce this section to readers who don't want to allocate much time to reading this post but rather understand the summary behind this approach. We introduce an algorithm that embeds hidden signatures onto a teacher model during fine-tuning. We do this by selecting the top k neurons that commonly fire in the teacher model. We also introduce k hidden signatures that will be embedded onto the teacher model during fine-tuning; these hidden signatures change the probability of x randomly chosen tokens at the final layer of the model so that the final probability of only certain tokens changes this ensures the model does not stray away too much from how it used to initially output text but also carry hidden signatures that can propagate throughout distillation . We LoRA fine-tune the model so that we map the k hidden signatures to be embedded when the respective k activation neurons fire. If the count of k is 32, then every time the top 1 neuron fires, an equivalent hidden signature h1 gets embedded into the model during fine-tuning, and so on for the top second, third ... up to 32. Then, if a model is distilled from the above teacher model, the hidden signatures transfer to the student model as well. Based on this principle, we introduce a decoder algorithm that looks at the outputs from the student model and detects if it has been distilled or not. We then evaluate the results for our encoding/decoding ensemble to see if our approach can detect distilled models, and we report our results suggesting viability on both logit-based and hard-label-based distillation methods. We used the following setup to test our algorithm. Teacher model: Qwen2.5-7B Student model: Qwen2.5-1.5B Dataset used throughout the experiment: PG-19 Activation rate: How frequently a neuron activates. Mean activation: Activation strength of a particular neuron. We work with Qwen 2.5 7B as our teacher model, which has 18,944 neurons per layer. Our initial goal is to extract the top-k neurons to imprint hidden signatures. In other words, we want to find the k neurons that most frequently fire, especially during distillation. We had to ensure that the choice of these neurons would be specific such that we don't end up picking neurons that fire too commonly this might add too many hidden signatures to detect, ex: neuron that fires for the token "the" or too rarely this will make it very hard to detect the hidden signatures and might even result in the neuron not firing during distillation . To ensure we satisfy the above criteria, we pick the top k neurons by: Fingerprint = unique SHA-256 key. We have elected top k neurons; we would need k equivalent fingerprints that each uniquely map to a change in the log prob of certain tokens in the last layer. Here "certain" is 256 randomly chosen tokens from the vocab size of 151,643 tokens. Each top-k neuron activation corresponds to the respective k fingerprint that, in turn, triggers a log prob shift in the 256 chosen tokens. The advantage of SHA-256 is the fact that it is unique, so for a given input, SHA-256 produces one unique hash, functioning as a complex form of hash function. Below is an illustration of SHA-256. Each fingerprint is a 256-bit hash. Since the combination of the 256 bits is unique, the 256 bits are mapped into a vector of size 256 with values {+1,-1}. A particular top-k neuron activation eventually leads to the vector above, which is used to change the log prob of the chosen 256 tokens in the final layer. Example neuron activation 1 = fingerprint 1 0x7c19f.... = +1,-1,-1,... | 4172 token id | +1 | | 7891 | -1 | | 1203 | -1 | ... | +1 | | | 7532 | -1 | Here, +1 and -1 indicate the probability of the token increasing and decreasing, respectively. Each particular neuron activation corresponds to a particular probability change of the 256 tokens ... neuron activation 32 = fingerprint 32 0x2c14f.... = +1,+1,-1,... | 4172 | -1 | | 7891 | +1 | | 1203 | +1 | ... | +1 | | | 7532 | -1 | Thus, we can uniquely map a particular neuron's activation to a corresponding change in probability of the respective 256 tokens in the final layer. To achieve the above log prob shift in the teacher model, we perform LoRA fine-tuning against a copy of the teacher while using a frozen copy for reference. We use a loss that has two main components: We apply this loss at every token position, where is the context up until that token, and represents all possible next tokens. We use this loss term to keep the model from losing capabilities by having a probability distribution similar to the pre-finetuning checkpoint. After cancelling the duplicated , this is equivalent to: where . Given these two loss terms, the full loss of the fine-tuning process is then where is a hyperparameter that assigns that weight/importance of . For our experiments we have set to , but a hyperparameter search would find optimal values for any given model. Our LoRA configuration is the following: rank 16, alpha 32, zero dropout, applied to all attention projections and the three MLP projections. The figure below shows the training trajectory of both losses: To check whether a suspect model has the distillation fingerprint, we query it on held-out contexts and test whether the LLM's output distribution shows the same feature-conditioned keyed log probability shifts that were put inside the teacher. The decoding process consists of the following high-level steps: For every feature , we first use the frozen teacher model to come up with a set of contexts where is active. Then, we construct an equally-sized control set where none of the features is active. We use these probe sets to evaluate the suspect model, and they are kept fixed through out the testing process. Let be the next-token distribution of the model to be tested . For every candidate token , that is, for every token that could be a signature token from the fingerprint, the decoder calculates In other words, the decoder calculates the difference in model tendency to use the token when the feature is active versus when the feature is inactive. If the suspect model has the fingerprint, then for each feature's signature tokens we would expect to be equal to the majority of the time. For a given fingerprint , the fingerprint score for each feature would then be A positive value means that the suspect’s feature-conditioned output changes are aligned with the fingerprint: a positive value means the majority of the tokens have . In practice, this setup can be confounded by noise. We can use the original non-fingerprinted teacher model as a reference to account for this: In other words, we get the difference in model tendency to use the token when the feature is active versus inactive, and subtract this with the non-fingerprinted teacher difference. Then, the score for each feature is The feature scores are not enough to allow a confident verdict on whether the suspect model has been distilled from our teacher model. Because of this, we compare the scores we computed to scores collected from fake, random signatures, to see if the suspects model's alignment to the fingerprint can be due to distillation rather than the null hypothesis. For this purpose, the decoder generates 1000 random fingerprints. For all fingerprints , the score is calculated in the same way as with the original key: where and are the vocabulary set and the sign for each feature , fingerprint . Based on the scores for this 1000-key random distribution, and the scores for the true key, we estimate the mean and standard deviation of the random-key null distribution and standardize the true-key score. We then compute a one-sided Gaussian-tail -value, to check whether the null hypothesis can be discarded, i.e. whether we can say confidently that the model was distilled, and our results are not an effect of random noise. Whenever , we take the model to have been distilled by the teacher. With the setup mentioned above, we were able to achieve promising results suggesting some predictive power of whether the student model has the fingerprint: When it comes to models distilled on hard labels, we were able to reproduce the same setup, with some results suggesting this method could work on the hard label setup. While neither the raw nor the reference-subtracted results show a value small enough for our threshold, we achieve values small enough to be considered of statistical significance by the usual standards. More experiments will be pursued with different families and model sizes to find whether fingerprinting improves, degrades, or stays static when scaling parameters and/or training samples. However, for the current setup, the exact results are as follows: Statistic | Gaussian | | Raw | 2.273 | 0.01152 | Clean-reference subtracted | 1.811 | 0.03507 | Our method presents a promising starting point for catching distilled models through simple tweaks to the teacher model. As model capabilities become increasingly valuable, unauthorized distillation may make it difficult for frontier labs to determine whether competing systems have been trained using outputs obtained in violation of their Terms of Service. We believe frontier labs like Anthropic, Deepmind, and OpenAI will benefit from using this method to enforce their Terms of Service. Specifically, we hope this research will allow labs to catch distillation violations from other companies and other models. Our approach provides a way to leave detectable signals in deployed systems to ensure possible distilled models can be caught. We also hope this initial method paves the way for future, better provenance systems that allow frontier labs to provide valuable evidence when suspected violations occur.