# Will it fit? A free planner for fine-tuning models that do not fit on your GPU

> Source: <https://dev.to/mycute_97b39c4e34027570f9/will-it-fit-a-free-planner-for-fine-tuning-models-that-do-not-fit-on-your-gpu-o50>
> Published: 2026-10-25 14:39:23+00:00

Most fine-tuning guides start at "install these 14 packages". I wanted to start one step earlier: **will it even fit?**

So I built two free things that answer that before you download 19 GB of weights.

**1. A calculator in the browser:** [huggingface.co/spaces/byjameson/will-it-fit](https://huggingface.co/spaces/byjameson/will-it-fit) — type a model id, pick your VRAM and sequence length. It reads `config.json` and does the arithmetic.

**2. A CLI:**

```
pip install -e .
streamft fit --model Qwen/Qwen2.5-14B-Instruct --gpu 6 --seq 512
13.99B params, 48 layers
NF4 blocks ~7.5 GB + bf16 head ~1.6 GB + reserve 0.9 GB (embedding stays on CPU)
-> whole model needs ~9.9 GB on 6 GB
verdict: needs streaming
measured on RTX 3050 6 GB: 6 block(s), 3.86 GB peak, 6.726 s/step at seq 512
```

**Memory is arithmetic. Speed is only printed where it was measured.**

The memory number comes from `config.json`: NF4 weight bytes, the bf16 LM head, the embedding pinned to CPU, and a 0.9 GB reserve. That is deterministic.

The speed number exists only for configurations I actually ran on the reference machine (RTX 3050 Laptop, 6 GB). For anything else it says *not measured* and tells you to start with one block and let the trainer's autotune measure the real MB/block. Planners that guess throughput are worse than no planner.

`streamft` fine-tunes models that do not fit by never holding them: the NF4 weights are mmap'd from disk, one transformer block is materialised on the GPU at a time, and the backward recomputes through it. LoRA is trainable, the base stays frozen.

Measured on a 6 GB laptop card: a 32B model at **1024 tokens/step, 36.7 tok/s, 4.61 GB peak**. Thirty SFT steps moved held-out loss from **1.8755 to 0.9959**, with 64/64 layers receiving gradients.

| model | streamft | PEFT/transformers | streamft peak | PEFT peak | 
|---|---|---|---|---|
| 3B | 0.94 s/step | **0.58 s** | **3.85 GB** | 4.90 GB | 
| 7B | **1.70 s** | 8.96 s | **5.51 GB** | 10.05 GB | 
| 14B | **6.73 s** | 18.56 s | **3.86 GB** | 16.48 GB | 
| 32B | **12.5 s** | fails before step 1 | **3.90 GB** | — | 

That third row is the honest one: on a 3B, **PEFT is faster than this engine and that's fine**. On Windows, PEFT does not OOM for 7B/14B — it silently spills into shared system memory (10 GB and 16 GB reserved on a 6 GB card) and pays 5.3x and 2.8x the step time. At 32B it dies in `prepare_model_for_kbit_training` before training starts.

Two checks are in the repo and runnable:

`scripts/verify_gradients.py` — one resident block (with recompute) vs all blocks resident, same adapter weights: loss identical to 10 decimals, adapter gradients `scripts/compare_peft.py` — the matched runs above. It also caught a bug in my first comparison: PEFT's eval averaged per row while mine averaged per token, which made the two look 0.7 apart. Fixed, both start at the same number (1.42087).`lm_head`); `--head nf4` is broken in the bitsandbytes version I tested.
If you try it on a different card, the interesting thing is the *disagreement* between the planner and reality — that is exactly what the autotune log shows, and I would like to hear where it is wrong.