# The 16 GB Cliff: Measured LLM Memory and Speed on an RTX 5080

> Source: <https://techfuelhq.com/data/rtx-5080-16gb-vram-cliff/>
> Published: 2026-11-17 17:00:00+00:00

**Recommendation:** Budget VRAM for context growth: llama3.2:3b Q4_K_M held 351.22 tok/s at 4,096 context but dropped to 102.68 tok/s at 131,072 on a 16 GB RTX 5080.
More in the playbook "Inference and self-hosting": <https://wpnews.pro/playbooks/inference-serving>

Every pin is a local language model I ran on my own retail RTX 5080, placed by the memory in use on the whole card and the decode speed that came with it. Nothing here is estimated. For a card I do not own or a setup I have not run, the LLM VRAM calculator and the LLM speed calculator estimate, and the VRAM tiers guide covers other card sizes. This page only reports.

Measured. Copied from a capture file, shown with its capture date.

Computed. Arithmetic on measured values, shown with its formula.

Showing 36 of 36 measured configurations

The map

Across: memory in use on the whole card. Up: decode speed.

tok/s0100200300400

llama3.2:3b, 4,096 to 131,072 context351.22 to 102.68 tok/s

04,0968,19212,28816,303MiB in usecard total

A measured configuration

Ran partly off the card

Reading includes another model

Same setup, context raised

Every pin is one row from a capture file. Nothing between two pins was measured, and nothing here is an estimate.

Readout

Nothing selected

Click or tap near a pin, or pick a row in the list, and its measured record appears here.

Measured at 3,760 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

3,760 MiB

Measured

Decode

351.22 tok/s

Measured

Left on the card

12,543 MiB

Computed: 16,303 − 3,760

Share of the card

23.1%

Computed: 3,760 ÷ 16,303

Full measured row

Captured

from the file's name

App

ollama 0.32.15

Engine

vendored llama.cpp 9d77fa172

Condition

defaults

Model name in the file

llama3.2:3b

Quantization

Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.

Context

4096

Runs

5

Decode, median (server counters)

351.22 tok/s

Decode, median (wall clock)

345.19 tok/s

Each run

351.15 · 350.9 · 351.86 · 351.32 · 351.22 tok/s

Prefill, long prompt, median

17802.7 tok/s at 434 prompt tokens

Time to first token

Empty in this file

Whole-card memory, median

3760 MiB

Whole-card memory before the load

1053 MiB

Added by loading the model

2,707 MiB Computed: 3,760 − 1,053

Load

2.67 s

Cold first request

2.79 s

App RAM while loaded

1234.0 MB

Driver

Not recorded in this file

Resident in GiB

3.67 GiB Computed: 3,760 ÷ 1,024

Energy for this model, from its own power run

A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 3,760 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

3,760 MiB

Measured

Decode

350.52 tok/s

Measured

Left on the card

12,543 MiB

Computed: 16,303 − 3,760

Share of the card

23.1%

Computed: 3,760 ÷ 16,303

Full measured row

Captured

from the file's name

App

ollama 0.32.15

Engine

vendored llama.cpp 9d77fa172

Condition

normalized

Model name in the file

llama3.2:3b

Quantization

Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.

Context

4096

Runs

5

Decode, median (server counters)

350.52 tok/s

Decode, median (wall clock)

338.11 tok/s

Each run

349.35 · 350.38 · 350.52 · 350.63 · 350.96 tok/s

Prefill, long prompt, median

17862.3 tok/s at 432 prompt tokens

Time to first token

Empty in this file

Whole-card memory, median

3760 MiB

Whole-card memory before the load

1057 MiB

Added by loading the model

2,703 MiB Computed: 3,760 − 1,057

Load

2.27 s

Cold first request

2.38 s

App RAM while loaded

1239.6 MB

Driver

Not recorded in this file

Resident in GiB

3.67 GiB Computed: 3,760 ÷ 1,024

Energy for this model, from its own power run

A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 4,343 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

4,343 MiB

Measured

Decode

318.51 tok/s

Measured

Left on the card

11,960 MiB

Computed: 16,303 − 4,343

Share of the card

26.6%

Computed: 4,343 ÷ 16,303

Full measured row

Captured

from the file's footer notes

Runtime

ollama 0.32.1

Model name in the file

llama3.2:3b

Quantization

Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.

Context

4096 from the file's settings note

Offload

Full offload from the file's settings note

Runs

5

Decode, median (server counters)

318.51 tok/s

Decode, median (wall clock)

313.25 tok/s

Each run

319.0 · 318.7 · 318.2 · 318.5 · 317.5 tok/s

Prefill, short prompt, median

7038.1 tok/s

Prefill, long prompt

17444 tok/s at 433 prompt tokens

Whole-card memory, median

4343 MiB

Load

3.2 s

Desktop baseline at bench start

1642 MiB one reading for the whole capture

Driver

Not recorded in this file

Resident in GiB

4.24 GiB Computed: 4,343 ÷ 1,024

Energy for this model, from its own power run

A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 5,495 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

5,495 MiB

Measured

Decode

312.89 tok/s

Measured

Left on the card

10,808 MiB

Computed: 16,303 − 5,495

Share of the card

33.7%

Computed: 5,495 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.1

Method

llm-throughput-ollama-v1

Model

llama3.2:3b digest a80c4f17acd5

Parameters

3.2B

Quantization

Q4_K_M

Context (num_ctx)

16384

Prompt cell 1

The row pinned on the map

Prompt tokens

471

Repetitions

3

Decode, median

312.89 tok/s

Prefill, median

17885.62 tok/s

Load, median

162.9 ms

Whole-card memory after the cell

5495 MiB

Prompt cell 2

Same configuration, longer prompt

Prompt tokens

1657

Repetitions

3

Decode, median

298.59 tok/s

Prefill, median

19397.36 tok/s

Load, median

158.0 ms

Whole-card memory after the cell

5501 MiB

Resident in GiB

5.37 GiB Computed: 5,495 ÷ 1,024

Energy for this model, from its own power run

A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 6,190 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

6,190 MiB

Measured

Decode

180.02 tok/s

Measured

Left on the card

10,113 MiB

Computed: 16,303 − 6,190

Share of the card

38.0%

Computed: 6,190 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.1

Method

llm-throughput-ollama-v1

Model

qwen2.5:7b digest 845dbda0ea48

Parameters

7.6B

Quantization

Q4_K_M

Context (num_ctx)

4096

Prompt cell 1

The row pinned on the map

Prompt tokens

488

Repetitions

3

Decode, median

180.02 tok/s

Prefill, median

10152.28 tok/s

Load, median

129.8 ms

Whole-card memory after the cell

6190 MiB

Prompt cell 2

Same configuration, longer prompt

Prompt tokens

1674

Repetitions

3

Decode, median

176.79 tok/s

Prefill, median

9672.39 tok/s

Load, median

123.4 ms

Whole-card memory after the cell

6190 MiB

Resident in GiB

6.04 GiB Computed: 6,190 ÷ 1,024

Energy for this model, from its own power run

A separate run of qwen2.5:7b on (UTC), stock power limit: 182.2 tok/s at 343.6 W board power, 1.886 J per token in the file. That is 1,886 J per 1,000 tokensComputed: 1.886 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 7,022 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

7,022 MiB

Measured

Decode

179.66 tok/s

Measured

Left on the card

9,281 MiB

Computed: 16,303 − 7,022

Share of the card

43.1%

Computed: 7,022 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.1

Method

llm-throughput-ollama-v1

Model

qwen2.5:7b digest 845dbda0ea48

Parameters

7.6B

Quantization

Q4_K_M

Context (num_ctx)

16384

Prompt cell 1

The row pinned on the map

Prompt tokens

488

Repetitions

3

Decode, median

179.66 tok/s

Prefill, median

10133.1 tok/s

Load, median

127.9 ms

Whole-card memory after the cell

7022 MiB

Prompt cell 2

Same configuration, longer prompt

Prompt tokens

1674

Repetitions

3

Decode, median

176.61 tok/s

Prefill, median

9984.37 tok/s

Load, median

127.7 ms

Whole-card memory after the cell

7034 MiB

Resident in GiB

6.86 GiB Computed: 7,022 ÷ 1,024

Energy for this model, from its own power run

A separate run of qwen2.5:7b on (UTC), stock power limit: 182.2 tok/s at 343.6 W board power, 1.886 J per token in the file. That is 1,886 J per 1,000 tokensComputed: 1.886 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 10,356 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

10,356 MiB

Measured

Decode

102.25 tok/s

Measured

Left on the card

5,947 MiB

Computed: 16,303 − 10,356

Share of the card

63.5%

Computed: 10,356 ÷ 16,303

Full measured row

Captured

from the file's name

App

ollama 0.32.15

Engine

vendored llama.cpp 9d77fa172

Condition

defaults

Model name in the file

qwen2.5:14b

Quantization

Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.

Context

4096

Runs

5

Decode, median (server counters)

102.25 tok/s

Decode, median (wall clock)

101.06 tok/s

Each run

102.13 · 102.34 · 102.25 · 102.3 · 102.25 tok/s

Prefill, long prompt, median

4983.8 tok/s at 453 prompt tokens

Time to first token

Empty in this file

Whole-card memory, median

10356 MiB

Whole-card memory before the load

1053 MiB

Added by loading the model

9,303 MiB Computed: 10,356 − 1,053

Load

5.51 s

Cold first request

5.67 s

App RAM while loaded

1604.1 MB

Driver

Not recorded in this file

Resident in GiB

10.11 GiB Computed: 10,356 ÷ 1,024

Energy for this model, from its own power run

A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 10,356 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

10,356 MiB

Measured

Decode

102.21 tok/s

Measured

Left on the card

5,947 MiB

Computed: 16,303 − 10,356

Share of the card

63.5%

Computed: 10,356 ÷ 16,303

Full measured row

Captured

from the file's name

App

ollama 0.32.15

Engine

vendored llama.cpp 9d77fa172

Condition

normalized

Model name in the file

qwen2.5:14b

Quantization

Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.

Context

4096

Runs

5

Decode, median (server counters)

102.21 tok/s

Decode, median (wall clock)

101.23 tok/s

Each run

102.21 · 102.21 · 102.17 · 102.21 · 102.21 tok/s

Prefill, long prompt, median

5038.3 tok/s at 452 prompt tokens

Time to first token

Empty in this file

Whole-card memory, median

10356 MiB

Whole-card memory before the load

1053 MiB

Added by loading the model

9,303 MiB Computed: 10,356 − 1,053

Load

5.52 s

Cold first request

5.68 s

App RAM while loaded

1617.6 MB

Driver

Not recorded in this file

Resident in GiB

10.11 GiB Computed: 10,356 ÷ 1,024

Energy for this model, from its own power run

A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 10,841 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

10,841 MiB

Measured

Decode

96.65 tok/s

Measured

Left on the card

5,462 MiB

Computed: 16,303 − 10,841

Share of the card

66.5%

Computed: 10,841 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.1

Method

llm-throughput-ollama-v1

Model

qwen2.5:14b digest 7cdf5a0187d5

Parameters

14.8B

Quantization

Q4_K_M

Context (num_ctx)

4096

Prompt cell 1

The row pinned on the map

Prompt tokens

488

Repetitions

3

Decode, median

96.65 tok/s

Prefill, median

5202.23 tok/s

Load, median

182.5 ms

Whole-card memory after the cell

10841 MiB

Prompt cell 2

Same configuration, longer prompt

Prompt tokens

1674

Repetitions

3

Decode, median

94.48 tok/s

Prefill, median

4876.41 tok/s

Load, median

125.8 ms

Whole-card memory after the cell

10855 MiB

Resident in GiB

10.59 GiB Computed: 10,841 ÷ 1,024

Energy for this model, from its own power run

A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 10,941 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

10,941 MiB

Measured

Decode

99.43 tok/s

Measured

Left on the card

5,362 MiB

Computed: 16,303 − 10,941

Share of the card

67.1%

Computed: 10,941 ÷ 16,303

Full measured row

Captured

from the file's footer notes

Runtime

ollama 0.32.1

Model name in the file

qwen2.5:14b

Quantization

Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.

Context

4096 from the file's settings note

Offload

Full offload from the file's settings note

Runs

5

Decode, median (server counters)

99.43 tok/s

Decode, median (wall clock)

98.59 tok/s

Each run

99.5 · 99.5 · 99.4 · 99.4 · 99.3 tok/s

Prefill, short prompt, median

3151.4 tok/s

Prefill, long prompt

5004 tok/s at 451 prompt tokens

Whole-card memory, median

10941 MiB

Load

5.6 s

Desktop baseline at bench start

1642 MiB one reading for the whole capture

Driver

Not recorded in this file

Resident in GiB

10.68 GiB Computed: 10,941 ÷ 1,024

Energy for this model, from its own power run

A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

gpt-oss:20bMXFP4 · 8,192 context · LM Studio 0.4.21 · defaults

Measured at 12,310 MiB resident on with LM Studio 0.4.21. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Ran partly off the card

LM Studio's default automatic GPU offload was in effect, at its default 8,192 context. The dataset attributes the lower decode rate to that setting. The normalized row is the same model with full offload. Runtime comparison dataset.

Resident, whole card

12,310 MiB

Measured

Decode

186.26 tok/s

Measured

Left on the card

3,993 MiB

Computed: 16,303 − 12,310

Share of the card

75.5%

Computed: 12,310 ÷ 16,303

Full measured row

Captured

from the file's name

App

LM Studio 0.4.21

Engine

llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1

Condition

defaults

Model name in the file

gpt-oss-20b

Quantization

MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.

Measured at 13,157 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

13,157 MiB

Measured

Decode

97.02 tok/s

Measured

Left on the card

3,146 MiB

Computed: 16,303 − 13,157

Share of the card

80.7%

Computed: 13,157 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.1

Method

llm-throughput-ollama-v1

Model

qwen2.5:14b digest 7cdf5a0187d5

Parameters

14.8B

Quantization

Q4_K_M

Context (num_ctx)

16384

Prompt cell 1

The row pinned on the map

Prompt tokens

488

Repetitions

3

Decode, median

97.02 tok/s

Prefill, median

5225.96 tok/s

Load, median

131.3 ms

Whole-card memory after the cell

13157 MiB

Prompt cell 2

Same configuration, longer prompt

Prompt tokens

1674

Repetitions

3

Decode, median

94.58 tok/s

Prefill, median

4864.67 tok/s

Load, median

128.9 ms

Whole-card memory after the cell

13159 MiB

Resident in GiB

12.85 GiB Computed: 13,157 ÷ 1,024

Energy for this model, from its own power run

A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 13,381 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

A re-capture of the same model. The dataset's change log notes it ran below the 2026-08-12 rows on a card several hours into continuous bench work. Throughput dataset, change log.

Resident, whole card

13,381 MiB

Measured

Decode

181.50 tok/s

Measured

Left on the card

2,922 MiB

Computed: 16,303 − 13,381

Share of the card

82.1%

Computed: 13,381 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.1

Method

llm-throughput-ollama-v1

Model

gpt-oss:20b digest 17052f91a42e

Parameters

20.9B

Quantization

MXFP4

Context (num_ctx)

4096

Prompt cell 1

The row pinned on the map

Prompt tokens

503

Repetitions

3

Decode, median

181.5 tok/s

Prefill, median

9607.49 tok/s

Load, median

383.7 ms

Whole-card memory after the cell

13381 MiB

Prompt cell 2

Same configuration, longer prompt

Prompt tokens

1360

Repetitions

3

Decode, median

178.37 tok/s

Prefill, median

11001.37 tok/s

Load, median

375.9 ms

Whole-card memory after the cell

13385 MiB

Resident in GiB

13.07 GiB Computed: 13,381 ÷ 1,024

Energy for this model, from its own power run

A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 13,570 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

13,570 MiB

Measured

Decode

213.54 tok/s

Measured

Left on the card

2,733 MiB

Computed: 16,303 − 13,570

Share of the card

83.2%

Computed: 13,570 ÷ 16,303

Full measured row

Captured

from the file's name

App

ollama 0.32.15

Engine

vendored llama.cpp 9d77fa172

Condition

defaults

Model name in the file

gpt-oss:20b

Quantization

MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.

Context

4096

Runs

5

Decode, median (server counters)

213.54 tok/s

Decode, median (wall clock)

212.03 tok/s

Each run

213.22 · 213.64 · 213.48 · 213.54 · 213.76 tok/s

Prefill, long prompt, median

9239.4 tok/s at 475 prompt tokens

Time to first token

Empty in this file

Whole-card memory, median

13570 MiB

Whole-card memory before the load

1053 MiB

Added by loading the model

12,517 MiB Computed: 13,570 − 1,053

Load

18.56 s

Cold first request

23.67 s

App RAM while loaded

2138.8 MB

Driver

Not recorded in this file

Resident in GiB

13.25 GiB Computed: 13,570 ÷ 1,024

Energy for this model, from its own power run

A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 13,570 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

13,570 MiB

Measured

Decode

213.65 tok/s

Measured

Left on the card

2,733 MiB

Computed: 16,303 − 13,570

Share of the card

83.2%

Computed: 13,570 ÷ 16,303

Full measured row

Captured

from the file's name

App

ollama 0.32.15

Engine

vendored llama.cpp 9d77fa172

Condition

normalized

Model name in the file

gpt-oss:20b

Quantization

MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.

Context

4096

Runs

5

Decode, median (server counters)

213.65 tok/s

Decode, median (wall clock)

211.8 tok/s

Each run

213.62 · 213.65 · 213.68 · 213.25 · 213.73 tok/s

Prefill, long prompt, median

9187.1 tok/s at 474 prompt tokens

Time to first token

Empty in this file

Whole-card memory, median

13570 MiB

Whole-card memory before the load

1053 MiB

Added by loading the model

12,517 MiB Computed: 13,570 − 1,053

Load

8.05 s

Cold first request

8.36 s

App RAM while loaded

2152.1 MB

Driver

Not recorded in this file

Resident in GiB

13.25 GiB Computed: 13,570 ÷ 1,024

Energy for this model, from its own power run

A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 13,683 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

A re-capture of the same model. The dataset's change log notes it ran below the 2026-08-12 rows on a card several hours into continuous bench work. Throughput dataset, change log.

Resident, whole card

13,683 MiB

Measured

Decode

183.00 tok/s

Measured

Left on the card

2,620 MiB

Computed: 16,303 − 13,683

Share of the card

83.9%

Computed: 13,683 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.1

Method

llm-throughput-ollama-v1

Model

gpt-oss:20b digest 17052f91a42e

Parameters

20.9B

Quantization

MXFP4

Context (num_ctx)

16384

Prompt cell 1

The row pinned on the map

Prompt tokens

503

Repetitions

3

Decode, median

183.0 tok/s

Prefill, median

9092.22 tok/s

Load, median

347.7 ms

Whole-card memory after the cell

13683 MiB

Prompt cell 2

Same configuration, longer prompt

Prompt tokens

1360

Repetitions

3

Decode, median

178.34 tok/s

Prefill, median

10328.62 tok/s

Load, median

345.3 ms

Whole-card memory after the cell

13683 MiB

Resident in GiB

13.36 GiB Computed: 13,683 ÷ 1,024

Energy for this model, from its own power run

A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 14,067 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

14,067 MiB

Measured

Decode

190.14 tok/s

Measured

Left on the card

2,236 MiB

Computed: 16,303 − 14,067

Share of the card

86.3%

Computed: 14,067 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.1

Method

llm-throughput-ollama-v1

Model

gpt-oss:20b digest 17052f91a42e

Parameters

20.9B

Quantization

MXFP4

Context (num_ctx)

4096

Prompt cell 1

The row pinned on the map

Prompt tokens

503

Repetitions

3

Decode, median

190.14 tok/s

Prefill, median

9078.28 tok/s

Load, median

297.7 ms

Whole-card memory after the cell

14067 MiB

Prompt cell 2

Same configuration, longer prompt

Prompt tokens

1658

Repetitions

3

Decode, median

187.27 tok/s

Prefill, median

10464.66 tok/s

Load, median

279.5 ms

Whole-card memory after the cell

14060 MiB

Resident in GiB

13.74 GiB Computed: 14,067 ÷ 1,024

Energy for this model, from its own power run

A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 14,152 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

14,152 MiB

Measured

Decode

195.79 tok/s

Measured

Left on the card

2,151 MiB

Computed: 16,303 − 14,152

Share of the card

86.8%

Computed: 14,152 ÷ 16,303

Full measured row

Captured

from the file's footer notes

Runtime

ollama 0.32.1

Model name in the file

gpt-oss:20b

Quantization

MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.

Context

4096 from the file's settings note

Offload

Full offload from the file's settings note

Runs

5

Decode, median (server counters)

195.79 tok/s

Decode, median (wall clock)

194.59 tok/s

Each run

195.8 · 195.7 · 196.3 · 196.1 · 195.7 tok/s

Prefill, short prompt, median

3744.3 tok/s

Prefill, long prompt

Empty in this file

Whole-card memory, median

14152 MiB

Load

8.6 s

Desktop baseline at bench start

1642 MiB one reading for the whole capture

Driver

Not recorded in this file

Resident in GiB

13.82 GiB Computed: 14,152 ÷ 1,024

Energy for this model, from its own power run

A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 14,355 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Resident, whole card

14,355 MiB

Measured

Decode

188.63 tok/s

Measured

Left on the card

1,948 MiB

Computed: 16,303 − 14,355

Share of the card

88.1%

Computed: 14,355 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.1

Method

llm-throughput-ollama-v1

Model

gpt-oss:20b digest 17052f91a42e

Parameters

20.9B

Quantization

MXFP4

Context (num_ctx)

16384

Prompt cell 1

The row pinned on the map

Prompt tokens

503

Repetitions

3

Decode, median

188.63 tok/s

Prefill, median

9286.61 tok/s

Load, median

299.2 ms

Whole-card memory after the cell

14355 MiB

Prompt cell 2

Same configuration, longer prompt

Prompt tokens

1658

Repetitions

3

Decode, median

187.73 tok/s

Prefill, median

10554.32 tok/s

Load, median

280.2 ms

Whole-card memory after the cell

14355 MiB

Resident in GiB

14.02 GiB Computed: 14,355 ÷ 1,024

Energy for this model, from its own power run

A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 14,997 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Measured at a 4,096 context only. The dataset reports that an 8K attempt spilled and decode collapsed, but no row for that attempt was published, so it has no pin here. Throughput dataset, coding models note.

Measured at 15,003 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Ran partly off the card

The model file is larger than the card, so Ollama ran it partially offloaded with part of it in system RAM. The dataset warns that its decode rate depends on how much system RAM was free when it loaded. Throughput dataset, coding models note.

Measured at 15,084 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Ran partly off the card

The capture's own ollama ps line reads 22%/78% CPU/GPU at context 131,072, so part of this model ran from system RAM.

Ollama's app-level context slider was set to 256k, which this model clamps to 131,072. The dataset describes the result as the KV cache spilling to the CPU. Runtime comparison dataset.

Resident, whole card

15,084 MiB

Measured

Decode

102.68 tok/s

Measured

Left on the card

1,219 MiB

Computed: 16,303 − 15,084

Share of the card

92.5%

Computed: 15,084 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.15

Condition

defaults

Schema

techfuelhq:lmstudio-vs-ollama:1

Model

llama3.2:3b

Quantization

Q4_K_M. Not a field in this file. Taken from the same model in the throughput CSV.

Context

131072 from the capture's ollama ps line

Processor split

22% CPU / 78% GPU from the capture's ollama ps line

Size reported by ollama ps

18 GB

Runs

5

Decode, median (server counters)

102.68 tok/s

Decode, median (wall clock)

101.58 tok/s

Each run

97.57 · 99.85 · 103.59 · 102.68 · 103.21 tok/s

Prefill, long prompt, median

9998.8 tok/s at 434 prompt tokens

Whole-card memory, median

15084 MiB

Whole-card memory before the load

1067 MiB

Added by loading the model

14,017 MiB Computed: 15,084 − 1,067

Load

4.02 s

Cold first request

4.2 s

App RAM while loaded

4745.7 MB

Resident in GiB

14.73 GiB Computed: 15,084 ÷ 1,024

Energy for this model, from its own power run

A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

Measured at 15,101 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Ran partly off the card

The model file is larger than the card, so Ollama ran it partially offloaded with part of it in system RAM. The dataset warns that its decode rate depends on how much system RAM was free when it loaded. Throughput dataset, coding models note.

Measured at 15,682 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.

Reading includes another model

gpt-oss:20b was still loaded from the cell before this one, so this reading is the whole card with two models on it, not Llama 3.2 3B alone. The 16,384-context row from the same capture was read after the larger model had been evicted. Throughput dataset, VRAM note.

Resident, whole card

15,682 MiB

Measured

Decode

311.96 tok/s

Measured

Left on the card

621 MiB

Computed: 16,303 − 15,682

Share of the card

96.2%

Computed: 15,682 ÷ 16,303

Full measured row

Captured (UTC)

GPU

NVIDIA GeForce RTX 5080

Driver

610.88

Runtime

Ollama 0.32.1

Method

llm-throughput-ollama-v1

Model

llama3.2:3b digest a80c4f17acd5

Parameters

3.2B

Quantization

Q4_K_M

Context (num_ctx)

4096

Prompt cell 1

The row pinned on the map

Prompt tokens

471

Repetitions

3

Decode, median

311.96 tok/s

Prefill, median

18196.57 tok/s

Load, median

162.8 ms

Whole-card memory after the cell

15682 MiB

Prompt cell 2

Same configuration, longer prompt

Prompt tokens

1657

Repetitions

3

Decode, median

298.21 tok/s

Prefill, median

17681.08 tok/s

Load, median

156.2 ms

Whole-card memory after the cell

15682 MiB

Resident in GiB

15.31 GiB Computed: 15,682 ÷ 1,024

Energy for this model, from its own power run

A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.

No measured configuration matches those filters. That combination was not on the bench, and I do not estimate here. For an estimate, use the LLM VRAM calculator.

How to read it

The map answers a narrow question: what did this model, at this context length, on this runtime version, take on a 16 GB card, and what speed came with it. Each record reads “measured at so many MiB resident on a date with a runtime version” because that is all a measurement can say. It is not a promise that the same model fits on your card today.

The line along the bottom is the card, from zero to the total it reports to nvidia-smi. A pin’s position across is the memory in use on the whole card while that model was loaded, Windows desktop included, and its height is the decode rate. Select a pin and the line fills to it in the measured colour. The hatched remainder is headroom, which I compute by subtracting.

A dashed connector joins the same model, runtime and capture at two context lengths. Flat means memory grew and speed held; a drop means the model stopped fitting. A ring is a run recorded as partly off the card. A diamond is a reading taken while another model was still loaded, so it overstates that model.

Where the cliff shows up

The cliff is not a memory number. In these files, a model that sits whole on the card decodes at the same speed whether it uses a third of the memory or nearly all of it. Speed falls when part of the model no longer fits and runs from system RAM instead.

Each row is the same model, runtime version and capture at two context lengths, and both ends are measured rows. Memory change is the second reading minus the first. Decode change is (second − first) ÷ first.

Model and runtimeContextResident MiB, measuredMemory change, computedDecode tok/s, measuredDecode change, computed

llama3.2:3bOllama 0.32.15, defaults · 131,072: ran partly off the card4,096 → 131,0723,760 → 15,084+11,324351.22 → 102.68−70.8%

qwen3-coder:30bOllama 0.32.1 · 16,384: ran partly off the card4,096 → 16,38415,101 → 15,003−9861.89 → 51.62−16.6%

The table has two rows where raising the context cost speed, and both are runs that left the card. Llama 3.2 3B is the small model in this set, and a large enough context still pushed it over. Qwen3-Coder 30B never fit whole, so its memory reading barely moved and its speed did.

Every other pair held its decode rate within one percent while the context quadrupled. What they paid was memory, and how much depended on the model: the two 14B Qwen models paid the most, and gpt-oss:20b, the largest of them by parameter count, paid the least.

What the files do not hold is a dense model caught in the act of falling off. The throughput dataset says Devstral 24B collapsed when I tried it at an 8K context, but that attempt was never written to the CSV, so there is no pin for it and I have not drawn one.

Energy per thousand tokens

llama3.2:3bQ4_K_M · named “Llama 3.2 3B” in the file · run logged UTC

833 J per 1,000 tokens Computed: 0.833 J per token × 1,000

Decode, median of 3

318.3 tok/s

Board power, median while loaded

265.2 W

Tokens per joule, as filed

1.2

Joules per token, as filed

0.833

More from the power file

Watt-hours per 1,000 tokens

0.231 Wh Computed: 231.0 Wh per million ÷ 1,000

Decode, slowest and fastest run

315.5 to 318.5 tok/s

Board power, peak

307.78 W

Memory in use, median

3,590 MiB

GPU temperature, median

49.0 °C

GPU load, median

82.0%

Telemetry samples while loaded

9

qwen2.5:7bQ4_K_M · named “Qwen 2.5 7B” in the file · run logged UTC

1,886 J per 1,000 tokens Computed: 1.886 J per token × 1,000

Decode, median of 3

182.2 tok/s

Board power, median while loaded

343.6 W

Tokens per joule, as filed

0.53

Joules per token, as filed

1.886

More from the power file

Watt-hours per 1,000 tokens

0.524 Wh Computed: 524.0 Wh per million ÷ 1,000

Decode, slowest and fastest run

182.1 to 182.3 tok/s

Board power, peak

346.08 W

Memory in use, median

5,665 MiB

GPU temperature, median

52.0 °C

GPU load, median

88.0%

Telemetry samples while loaded

14

qwen2.5:14bQ4_K_M · named “Qwen 2.5 14B” in the file · run logged UTC

3,154 J per 1,000 tokens Computed: 3.154 J per token × 1,000

Decode, median of 3

98.8 tok/s

Board power, median while loaded

311.6 W

Tokens per joule, as filed

0.317

Joules per token, as filed

3.154

More from the power file

Watt-hours per 1,000 tokens

0.876 Wh Computed: 876.0 Wh per million ÷ 1,000

Decode, slowest and fastest run

98.6 to 98.8 tok/s

Board power, peak

319.69 W

Memory in use, median

10,171 MiB

GPU temperature, median

50.0 °C

GPU load, median

93.0%

Telemetry samples while loaded

27

gpt-oss:20bMXFP4 · named “gpt-oss 20B” in the file · run logged UTC

1,504 J per 1,000 tokens Computed: 1.504 J per token × 1,000

Decode, median of 3

194.4 tok/s

Board power, median while loaded

292.4 W

Tokens per joule, as filed

0.665

Joules per token, as filed

1.504

More from the power file

Watt-hours per 1,000 tokens

0.418 Wh Computed: 418.0 Wh per million ÷ 1,000

Decode, slowest and fastest run

194.3 to 194.5 tok/s

Board power, peak

294.86 W

Memory in use, median

13,381 MiB

GPU temperature, median

51.0 °C

GPU load, median

84.0%

Telemetry samples while loaded

18

These four runs are a separate capture from the map: one short prompt, three iterations per model, board power sampled once a second and counted only while the GPU was at least half loaded. They were taken at the card’s stock 448 W power limit, which none of the runs reached. Board power is the graphics card alone, not the wall.

The power file records no context length, driver or runtime version, which is why these runs are not pinned on the map. Its dataset page gives the runtime as Ollama 0.32.1.

There is no undervolt or power-limit comparison here because I have not measured one for language models. My undervolt database holds one row for this card and it is a stock 3DMark baseline. Method and the idle-power finding are on the power efficiency dataset.

Versions on the bench, and how stale they are

Driver and runtime versions per capture, exactly as each file records them. Where a file is silent, the cell says so.

CapturedFileGPU driverRuntime versionsRows used here

llm-server-compare-2026-08-26-lmstudio.csvNot recorded in this fileLM Studio 0.4.21, Ollama 0.32.15Ollama 0.32.15: vendored llama.cpp 9d77fa172LM Studio 0.4.21: llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.112 configurations

Those are the versions that were installed on the capture dates. Drivers and runtimes have both moved since, and I have not re-run this matrix on newer ones. Read every pin as true for its date and versions, and expect a current install to differ.

The second table is why I say that plainly. Two captures six days apart put the same three models on two Ollama versions, and all three decoded faster on the later one. The runtime comparison dataset notes that the earlier capture had less control over background load, so not all of that gap is the version. Either way, a few percent between runtimes does not survive an update.

What the data covers

Model tags (8)

llama3.2:3b 3.2B · Q4_K_M 9 configurations

qwen2.5-coder:7b 7.6B · Q4_K_M 2 configurations

qwen2.5:7b 7.6B · Q4_K_M 2 configurations

qwen2.5-coder:14b 14.8B · Q4_K_M 2 configurations

qwen2.5:14b 14.8B · Q4_K_M 8 configurations

gpt-oss:20b 20.9B · MXFP4 10 configurations

devstral:24b 23.6B · Q4_K_M 1 configuration

qwen3-coder:30b 30.5B · Q4_K_M 2 configurations

Quantizations (2)

MXFP410

Q4_K_M26

No model was measured at more than one quantization.

One quantization per model. I cannot show the same model at three quantizations stepping down the card, because each model was measured at one.

No dense model past the cliff. The runs that left the card are a 3B model at an extreme context, a mixture-of-experts model that never fit, and one LM Studio default that offloaded part of a model that otherwise fits. A dense model that fits at one context and spills at the next is the row this map most needs.

A gap between 16,384 and 131,072 context. One run sits at the far end. Nothing was measured at 32K or 64K.

Energy at stock only. No undervolt, no reduced power limit, and no energy figure for a run that spilled.

One card. Another 16 GB card, or this one with a different desktop load, will read differently.

Method, exactly

I did not run anything new for this page. It reads files I had already published and puts them on one axis.

The throughput rows come from my capture script. For each model, context and prompt size it calls Ollama’s /api/generate three times at temperature 0, seed 42 and a 256-token output, with a unique prefix on every prompt so each run pays a full prefill. Decode and prefill rates are Ollama’s own eval_count over eval_duration and prompt_eval_count over prompt_eval_duration, and the CSV keeps the median of the three. After each cell the script asks nvidia-smi for memory.used and stores it as vram_resident_mib. That one reading is the pin’s position.

That reading is the whole card. It includes the desktop, and it includes any model Ollama had not yet unloaded, which is how one Llama 3.2 3B row came to read 15,682 MiB. The runtime comparison captures used a different script that took the median memory reading across runs and, in the later capture, also read the card before each load.

Each throughput configuration was measured with two prompt sizes. The pin is the first of the two, the shorter prompt, and both are in the record. The model names differ between files, so I join them on the name with punctuation and a trailing quantization tag removed. Only the throughput CSV has a quantization column; where another file’s row shows a quantization, the record says it was taken from the same model there.

The card total, 16,303 MiB, is the nvidia-smi total written into the raw capture file from 2026-08-26. Headroom, share of the card, GiB, the memory a load added, the changes between two rows and the per-thousand-token energy are computed when the site is built, from the measured values beside them, and each one is shown with its formula. Your browser does no arithmetic on them.

Download the files

rtx-5080-llm-throughput.csv34 rows. Decode, prefill, load time and whole-card memory for each model, context and prompt size on Ollama. Raw capture JSON and the capture script are listed in its manifest.

llm-server-compare-2026-08.csv6 rows. Ollama beside llama.cpp on three models, with its settings and capture date in the footer notes.

llm-server-compare-2026-08-26-lmstudio.csv12 rows. Ollama beside LM Studio on three models, at default and normalized settings, with the memory reading before each load.

All of it is CC BY 4.0. Attribute as “TechFuelHQ RTX 5080 16 GB VRAM cliff map” with a link to this page. To cite one configuration, open its row in the list and copy its link; the address ends in that row’s own id.
