# Llama 3.1 8b Instruct - Memory Usage More than Reported

> Source: <https://discuss.huggingface.co/t/llama-3-1-8b-instruct-memory-usage-more-than-reported/140711#post_7>
> Published: 2026-07-07 20:32:21+00:00

**## (“Memory Usage More than Reported”)**

Late to this thread but it keeps being the #1 surprise, so for future readers — the missing 16 GB is the ****KV cache****, and you can compute it from `config.json` alone:

```

KV bytes = 2 (K+V) × layers × kv_heads × head_dim × context × bytes/elem

Llama-3.1-8B: 2 × 32 × 8 × 128 × 131072 × 2 (fp16) = 16 GB (128 KB per token!)

```

So at the advertised 128K context: 15 GB of weights + 16 GB of cache + scratch ≈ 32 GB — it was never going to fit in 24 GB, and it’s not the model’s “fault”: it’s the context. Three levers, cheapest first: quantize the cache (`q8_0` halves it), cap the context (~66K fits in 24 GB with fp16 weights), or drop weight precision.

I packaged this arithmetic (plus the “which side is the problem” verdict and the max-context solver) into a free browser tool — geometry fetched from the model card, nothing downloaded: [TAF Agent — Test ANY Transformer LLM in Your Browser](https://karlesmarin.github.io/tafagent/?demo=fitcheck)