PIN v2. Choosing a model's size and speed before you train it

A new neural network compression method called PIN v2, developed by an unnamed research team, allows model size and speed to be chosen in advance by tying groups of cells to hold identical values, keeping the grid size constant while reducing storage costs. In tests, a network limited to 3,000 numbers with tying achieved 41% accuracy versus 17% for a conventional network, and at 128× compression it landed within 1.2 points of the uncompressed model. The method also enables model prediction, where a linear map can emit models for unseen tasks, matching trained accuracy (0.881 vs 0.876) and outperforming the base model (0.839).

PIN Architecture v2: making a model’s size and speed things you choose in advance. The basic trick Neural network layers are big grids of numbers. The usual way to make one smaller is to shrink the grid. This does something else. It keeps the grid the same size but forces groups of cells to hold identical values. Store 2,000 numbers, scatter them across a million cells according to a fixed pattern, and you have a million-weight layer that costs 2,000 numbers to store. The pattern itself is free, since it’s generated from a seed rather than stored. Weights lose precision, obviously. What you get back is more interesting than it first looks. Why it helps If you have a tiny storage budget and shrink the grid instead, you get a tiny network. In one of our tests, a conventional network limited to 3,000 numbers could afford exactly one hidden unit. It scored 17%. Tying lets you keep 256 units on the same budget. That scored 41%. That’s the whole mechanism, and it took us two wrong explanations to state it properly: tying decouples how wide a network is from how much it costs to store. Normally those are the same choice. Here they aren’t. Tested against magnitude pruning, low-rank factorisation, quantisation and distillation at matched bit budgets, with sparse formats charged for their index overhead which comparisons usually skip , tying wins four of five budgets. At 128× compression it lands within 1.2 points of the uncompressed model. Below about 100 KB it’s the only one of the five that can be built at all. The part we didn’t expect Two networks trained separately on the same data are, internally, unrelated. Weight 4,096 in one has nothing to do with weight 4,096 in the other. You can’t average them, compare them, or predict one from the other. They’re islands. But if every network uses the same tying pattern, then “value 37” means the same thing in all of them. They share a coordinate system by construction. Once models are in a shared coordinate system, you can do arithmetic on them. So we tried predicting a model instead of training one Train a base network. Fine-tune 45 variants, each on a different subset of classes. Each variant is now just a few thousand numbers, all in the same frame. Fit a simple linear map from “which classes does this task involve” to “how do the numbers change”. Then hand it a task combination it has never seen and ask for the numbers. where the model came from accuracy trained properly for the task 0.876 emitted from a 10-bit description, never trained 0.881 the base, unadapted 0.839 given the wrong description 0.696 It matches training. It collapses when misinformed, so it’s genuinely reading the description. And the map plus the base, 43,000 numbers in total, can emit any of 120 possible tasks, where storing those models conventionally would cost 14 million. What it does and doesn’t do It recombines. We tested this by withholding one class from ever varying during fitting, which makes its contribution mathematically unrecoverable, and performance on tasks needing it drops by three quarters. It works inside the span of what it has seen and degrades outside it. That’s a real boundary, and we measured it rather than assuming it. Honest scope Experiments run up to 3,072-dimensional inputs and width 1,024, covering MLPs, a small vision transformer, and a CNN. Not deployment scale. The compute advantage of the fast-transform variant is real at width 4,096 44× measured and negative below width 512, which we report because our own earlier estimate had the sign wrong. The core weight-sharing operator coincides with HashedNets Chen et al., ICML 2015 and priority is theirs. What’s new here: harder baselines, bit-accurate accounting, the input-dimension scaling law, and the model-prediction result. Two mechanisms we proposed were refuted by our own subsequent tests. Both corrections are in the paper, along with the failures that produced them. Why bother Right now you train a model and then find out how big and how fast it is. This makes both of them things you specify first. Pick a storage budget, pick an inference bound, pick a family of tasks, and the models follow. That’s a different relationship to model building than tuning until something fits. Paper, code, per-seed results and the operator checks are all in the record.