This weight does not spark joy A paper by Lu et al., "The Surprising Effectiveness of Deleting Weights in LLM Post-Training," finds that zeroing out gate weights in gated linear units can fine-tune large language models using masks under 0.05% of model weights, producing adapters orders of magnitude smaller than full fine-tuning checkpoints while retaining prior-task accuracy. In an independent replication fine-tuning Liquid's LFM3-1.7B on Wordle, the author measured 81% accuracy with deletion tuning versus 69% for full fine-tuning and 68% for LoRA, though across only a few seeds, and found all three methods stayed within one point of the base model on the paper's benchmark suite at recommended learning rates. The result matters because it suggests a far cheaper, less destructive alternative to full fine-tuning and LoRA for adjusting model behavior. The bulk of parameters in LLMs are in the feed-forward networks at the end of the transformer block. Because these are linear layers, they need to be separated by an activation function to avoid collapsing into a single transform. In the old days of ML that was usually a sigmoid, so you’d have W2 · sigmoid W1 · x , where x was the input. Then it became clear that we could get away with something a lot simpler and ReLU became standard. ReLU is just max 0, value , but you can think of it as a gate: W1 is multiplied with the input, we zero any values less than zero, and the result is multiplied with W2 . The problem with that is that the W1 matrix is both setting the gate value, and the value-value. Gated Linear Units add an extra matrix: W2 · W1 · x sigmoid Gate · x . That lets the model learn with one one set of parameters what input