# How much VRAM and how many GPUs to fine-tune a 70B parameter model like LLaMA 3.1 locally?

> Source: <https://discuss.huggingface.co/t/how-much-vram-and-how-many-gpus-to-fine-tune-a-70b-parameter-model-like-llama-3-1-locally/150882#post_3>
> Published: 2026-07-15 21:07:40+00:00

Rough VRAM math for fine-tuning a 70B, by method:

**Full fine-tune** (AdamW, mixed precision): ~16 bytes/param once you count fp16 weights + fp32 master weights + gradients + optimizer moments — that’s ~1.1 TB before activations, so realistically 8× H100/A100-80GB with ZeRO-3/FSDP sharding and activation checkpointing, or fewer GPUs with optimizer offload and a lot of patience. Not a home setup.

**LoRA** (fp16 base): the frozen base still needs its ~140 GB resident, plus a small adapter + optimizer state 2× 80 GB, or 4× 48 GB with sharding.

**QLoRA** (4-bit NF4 base): ~40–45 GB for the quantized base + adapter state + activations. A single 48 GB card (A6000/RTX PRO 6000) works at moderate sequence lengths; context length is what pushes it over, since KV/activation memory scales with tokens.

I built a calculator that does this per method/quant/context and shows the formula and every assumption (it covers full/LoRA/QLoRA, not just inference): [https://vram.rxdt.dev](https://vram.rxdt.dev) corrections welcome if your real runs disagree.