A 7B model is the honest ceiling for LoRA fine-tuning on a 16 GB consumer card, not because anything bigger throws an error, but because a 14B model silently spills into system memory and runs 50 times slower without ever reporting a failure.
Key takeaways
I pre-registered two kill-gates on an RTX 5070 Ti, 16,303 MiB of physical VRAM: whether a 14B Qwen2.5-Instruct model, loaded in 4-bit with a LoRA adapter, completes one real training step without a raised CUDA out-of-memory exception, and whether gradient checkpointing moves that ceiling up by at least one tested size, following the same discipline described in why I pre-register a kill-gate for every experiment. Neither 14B run threw an exception. By the literal wording of the first gate, it fired - 14B "fits". But torch.cuda.max_memory_allocated() reported 22,189.8 MiB without checkpointing, 5,887 MiB above the card's physical limit, and the step took 21.7 seconds instead of the sub-second range every smaller model showed. Windows ships a driver-level system-memory fallback policy that lets CUDA allocations succeed past a card's physical limit by quietly paging into ordinary RAM instead of raising an error, and nothing in the console said so. The gate measured the wrong thing: "did not throw an exception" is not the same claim as "fits in 16 GB", and only comparing the allocator figure against the card's actual VRAM, plus noticing the 50x time cliff, exposed the gap.
Three of the four sizes behaved exactly as expected. Without checkpointing, 1.5B peaked at 4,000.7 MiB, 3B at 6,749.0 MiB, 7B at 11,897.9 MiB - all comfortably under 16,303 MiB, all completing a forward-backward-optimizer step in under half a second. 7B used 73% of the card's VRAM at rank 16, sequence length 512, batch size 1, with real room to spare. Full numbers for all eight configurations sit in the frozen plan and results in the public experiment repository. Enabling gradient checkpointing lowered every one of those three figures - 2,855.5 MiB, 4,490.8 MiB and 9,709.9 MiB respectively - at essentially no time cost: 7B ran in the same 0.425 seconds either way, 3B cost 25% more time for a third less memory. The 1.5B timing is excluded from this comparison because it ran first in the script and likely carries a one-time CUDA initialisation cost. The honest ceiling for this exact setup, Qwen2.5-Instruct, 4-bit base, LoRA rank 16, one sequence of 512 tokens, batch size 1, is 7B. Push any of those variables up and the real number moves down, not up.
No. With checkpointing enabled, 14B still reported 17,470.9 MiB, 1,168 MiB over the card's physical limit - smaller than the 5,887 MiB overflow without checkpointing, but still spilling into system memory, not fitting inside it. The step time went the wrong direction: 34.5 seconds, 59% slower than the 21.7 seconds without checkpointing. Checkpointing buys memory by deleting intermediate activations during the forward pass and recomputing them during backward, and that recompute cost is normally hidden by the time saved from avoiding memory pressure, which is exactly why it helped 7B and below for free. Once the card is already paging to system RAM, the saved memory only shrinks an already-severe penalty instead of removing it, and the extra computation adds straight on top. The registered hypothesis, that checkpointing moves the ceiling up by at least one tested tier, is rejected. On this card, for this model family, checkpointing is a real, close-to-free lever below the ceiling, not a way past it.
Does this mean a 16 GB card can never run a 14B model?
Not for training, no, not with this exact setup. A smaller rank, a shorter sequence, a smaller LoRA target set, or a more aggressive quantisation scheme could plausibly bring a 14B model under the limit; none of those were tested here. For inference rather than training, the memory math is different again and was not measured in this experiment.
Why trust torch's reported VRAM figure at all, if it can be wrong?
It is not wrong about what it allocates - torch.cuda.max_memory_allocated() reports real tensor memory the CUDA allocator granted, honestly. What it does not tell you is whether that memory was physically on the card or transparently paged to system RAM by the driver. Comparing the figure against the card's known physical limit, rather than trusting the absence of an exception, is what catches the difference.
Why only test rank 16 and batch size 1?
Those are close to the smallest realistic defaults most people reach for first. Testing a single point cleanly, rather than sweeping many configurations in one pass, keeps the registered claim specific and checkable. A larger rank or batch size sweep is a legitimate follow-up, not something this experiment's kill-gates covered.
Where do the numbers come from?
Every figure in this piece is read directly from results.json in the registered experiment, not estimated or rounded from memory. The frozen plan, the full experiment code, and the raw per-run output all sit in the public repository linked above, so the eight numbers in the table can be checked against the source instead of taken on trust.