Every pin is a local language model I ran on my own retail RTX 5080, placed by the memory in use on the whole card and the decode speed that came with it. Nothing here is estimated. For a card I do not own or a setup I have not run, the LLM VRAM calculator and the LLM speed calculator estimate, and the VRAM tiers guide covers other card sizes. This page only reports.
Measured. Copied from a capture file, shown with its capture date.
Computed. Arithmetic on measured values, shown with its formula.
Showing 36 of 36 measured configurations
The map
Across: memory in use on the whole card. Up: decode speed.
tok/s0100200300400
llama3.2:3b, 4,096 to 131,072 context351.22 to 102.68 tok/s
04,0968,19212,28816,303MiB in usecard total
A measured configuration
Ran partly off the card
Reading includes another model
Same setup, context raised
Every pin is one row from a capture file. Nothing between two pins was measured, and nothing here is an estimate.
Readout
Nothing selected
Click or tap near a pin, or pick a row in the list, and its measured record appears here.
Measured at 3,760 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
3,760 MiB
Measured
Decode
351.22 tok/s
Measured
Left on the card
12,543 MiB
Computed: 16,303 − 3,760 Share of the card
23.1%
Computed: 3,760 ÷ 16,303 Full measured row
Captured
from the file's name App
ollama 0.32.15
Engine
vendored llama.cpp 9d77fa172
Condition
defaults
Model name in the file
llama3.2:3b
Quantization
Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
Context
4096
Runs
5
Decode, median (server counters) 351.22 tok/s
Decode, median (wall clock) 345.19 tok/s
Each run
351.15 · 350.9 · 351.86 · 351.32 · 351.22 tok/s
Prefill, long prompt, median
17802.7 tok/s at 434 prompt tokens
Time to first token
Empty in this file
Whole-card memory, median
3760 MiB
Whole-card memory before the load
1053 MiB
Added by the model
2,707 MiB Computed: 3,760 − 1,053
Load
2.67 s
Cold first request
2.79 s
App RAM while loaded
1234.0 MB
Driver
Not recorded in this file
Resident in GiB
3.67 GiB Computed: 3,760 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 3,760 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
3,760 MiB
Measured
Decode
350.52 tok/s
Measured
Left on the card
12,543 MiB
Computed: 16,303 − 3,760 Share of the card
23.1%
Computed: 3,760 ÷ 16,303 Full measured row
Captured
from the file's name App
ollama 0.32.15
Engine
vendored llama.cpp 9d77fa172
Condition
normalized
Model name in the file
llama3.2:3b
Quantization
Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
Context
4096
Runs
5
Decode, median (server counters) 350.52 tok/s
Decode, median (wall clock) 338.11 tok/s
Each run
349.35 · 350.38 · 350.52 · 350.63 · 350.96 tok/s
Prefill, long prompt, median
17862.3 tok/s at 432 prompt tokens
Time to first token
Empty in this file
Whole-card memory, median
3760 MiB
Whole-card memory before the load
1057 MiB
Added by the model
2,703 MiB Computed: 3,760 − 1,057
Load
2.27 s
Cold first request
2.38 s
App RAM while loaded
1239.6 MB
Driver
Not recorded in this file
Resident in GiB
3.67 GiB Computed: 3,760 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 4,343 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
4,343 MiB
Measured
Decode
318.51 tok/s
Measured
Left on the card
11,960 MiB
Computed: 16,303 − 4,343 Share of the card
26.6%
Computed: 4,343 ÷ 16,303 Full measured row
Captured
from the file's footer notes Runtime
ollama 0.32.1
Model name in the file
llama3.2:3b
Quantization
Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
Context
4096 from the file's settings note
Offload
Full offload from the file's settings note
Runs
5
Decode, median (server counters) 318.51 tok/s
Decode, median (wall clock) 313.25 tok/s
Each run
319.0 · 318.7 · 318.2 · 318.5 · 317.5 tok/s
Prefill, short prompt, median
7038.1 tok/s
Prefill, long prompt
17444 tok/s at 433 prompt tokens
Whole-card memory, median
4343 MiB
Load
3.2 s
Desktop baseline at bench start
1642 MiB one reading for the whole capture
Driver
Not recorded in this file
Resident in GiB
4.24 GiB Computed: 4,343 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 5,495 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
5,495 MiB
Measured
Decode
312.89 tok/s
Measured
Left on the card
10,808 MiB
Computed: 16,303 − 5,495 Share of the card
33.7%
Computed: 5,495 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.1
Method
llm-throughput-ollama-v1 Model
llama3.2:3b digest a80c4f17acd5
Parameters
3.2B
Quantization
Q4_K_M
Context (num_ctx) 16384
Prompt cell 1
The row pinned on the map
Prompt tokens
471
Repetitions
3
Decode, median
312.89 tok/s
Prefill, median
17885.62 tok/s
Load, median
162.9 ms
Whole-card memory after the cell
5495 MiB
Prompt cell 2
Same configuration, longer prompt
Prompt tokens
1657
Repetitions
3
Decode, median
298.59 tok/s
Prefill, median
19397.36 tok/s
Load, median
158.0 ms
Whole-card memory after the cell
5501 MiB
Resident in GiB
5.37 GiB Computed: 5,495 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 6,190 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
6,190 MiB
Measured
Decode
180.02 tok/s
Measured
Left on the card
10,113 MiB
Computed: 16,303 − 6,190 Share of the card
38.0%
Computed: 6,190 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.1
Method
llm-throughput-ollama-v1 Model
qwen2.5:7b digest 845dbda0ea48
Parameters
7.6B
Quantization
Q4_K_M
Context (num_ctx) 4096
Prompt cell 1
The row pinned on the map
Prompt tokens
488
Repetitions
3
Decode, median
180.02 tok/s
Prefill, median
10152.28 tok/s
Load, median
129.8 ms
Whole-card memory after the cell
6190 MiB
Prompt cell 2
Same configuration, longer prompt
Prompt tokens
1674
Repetitions
3
Decode, median
176.79 tok/s
Prefill, median
9672.39 tok/s
Load, median
123.4 ms
Whole-card memory after the cell
6190 MiB
Resident in GiB
6.04 GiB Computed: 6,190 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:7b on (UTC), stock power limit: 182.2 tok/s at 343.6 W board power, 1.886 J per token in the file. That is 1,886 J per 1,000 tokensComputed: 1.886 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 7,022 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
7,022 MiB
Measured
Decode
179.66 tok/s
Measured
Left on the card
9,281 MiB
Computed: 16,303 − 7,022 Share of the card
43.1%
Computed: 7,022 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.1
Method
llm-throughput-ollama-v1 Model
qwen2.5:7b digest 845dbda0ea48
Parameters
7.6B
Quantization
Q4_K_M
Context (num_ctx) 16384
Prompt cell 1
The row pinned on the map
Prompt tokens
488
Repetitions
3
Decode, median
179.66 tok/s
Prefill, median
10133.1 tok/s
Load, median
127.9 ms
Whole-card memory after the cell
7022 MiB
Prompt cell 2
Same configuration, longer prompt
Prompt tokens
1674
Repetitions
3
Decode, median
176.61 tok/s
Prefill, median
9984.37 tok/s
Load, median
127.7 ms
Whole-card memory after the cell
7034 MiB
Resident in GiB
6.86 GiB Computed: 7,022 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:7b on (UTC), stock power limit: 182.2 tok/s at 343.6 W board power, 1.886 J per token in the file. That is 1,886 J per 1,000 tokensComputed: 1.886 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 10,356 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
10,356 MiB
Measured
Decode
102.25 tok/s
Measured
Left on the card
5,947 MiB
Computed: 16,303 − 10,356 Share of the card
63.5%
Computed: 10,356 ÷ 16,303 Full measured row
Captured
from the file's name App
ollama 0.32.15
Engine
vendored llama.cpp 9d77fa172
Condition
defaults
Model name in the file
qwen2.5:14b
Quantization
Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
Context
4096
Runs
5
Decode, median (server counters) 102.25 tok/s
Decode, median (wall clock) 101.06 tok/s
Each run
102.13 · 102.34 · 102.25 · 102.3 · 102.25 tok/s
Prefill, long prompt, median
4983.8 tok/s at 453 prompt tokens
Time to first token
Empty in this file
Whole-card memory, median
10356 MiB
Whole-card memory before the load
1053 MiB
Added by the model
9,303 MiB Computed: 10,356 − 1,053
Load
5.51 s
Cold first request
5.67 s
App RAM while loaded
1604.1 MB
Driver
Not recorded in this file
Resident in GiB
10.11 GiB Computed: 10,356 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 10,356 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
10,356 MiB
Measured
Decode
102.21 tok/s
Measured
Left on the card
5,947 MiB
Computed: 16,303 − 10,356 Share of the card
63.5%
Computed: 10,356 ÷ 16,303 Full measured row
Captured
from the file's name App
ollama 0.32.15
Engine
vendored llama.cpp 9d77fa172
Condition
normalized
Model name in the file
qwen2.5:14b
Quantization
Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
Context
4096
Runs
5
Decode, median (server counters) 102.21 tok/s
Decode, median (wall clock) 101.23 tok/s
Each run
102.21 · 102.21 · 102.17 · 102.21 · 102.21 tok/s
Prefill, long prompt, median
5038.3 tok/s at 452 prompt tokens
Time to first token
Empty in this file
Whole-card memory, median
10356 MiB
Whole-card memory before the load
1053 MiB
Added by the model
9,303 MiB Computed: 10,356 − 1,053
Load
5.52 s
Cold first request
5.68 s
App RAM while loaded
1617.6 MB
Driver
Not recorded in this file
Resident in GiB
10.11 GiB Computed: 10,356 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 10,841 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
10,841 MiB
Measured
Decode
96.65 tok/s
Measured
Left on the card
5,462 MiB
Computed: 16,303 − 10,841 Share of the card
66.5%
Computed: 10,841 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.1
Method
llm-throughput-ollama-v1 Model
qwen2.5:14b digest 7cdf5a0187d5
Parameters
14.8B
Quantization
Q4_K_M
Context (num_ctx) 4096
Prompt cell 1
The row pinned on the map
Prompt tokens
488
Repetitions
3
Decode, median
96.65 tok/s
Prefill, median
5202.23 tok/s
Load, median
182.5 ms
Whole-card memory after the cell
10841 MiB
Prompt cell 2
Same configuration, longer prompt
Prompt tokens
1674
Repetitions
3
Decode, median
94.48 tok/s
Prefill, median
4876.41 tok/s
Load, median
125.8 ms
Whole-card memory after the cell
10855 MiB
Resident in GiB
10.59 GiB Computed: 10,841 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 10,941 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
10,941 MiB
Measured
Decode
99.43 tok/s
Measured
Left on the card
5,362 MiB
Computed: 16,303 − 10,941 Share of the card
67.1%
Computed: 10,941 ÷ 16,303 Full measured row
Captured
from the file's footer notes Runtime
ollama 0.32.1
Model name in the file
qwen2.5:14b
Quantization
Q4_K_M. Not a column in this file. Taken from the same model in the throughput CSV.
Context
4096 from the file's settings note
Offload
Full offload from the file's settings note
Runs
5
Decode, median (server counters) 99.43 tok/s
Decode, median (wall clock) 98.59 tok/s
Each run
99.5 · 99.5 · 99.4 · 99.4 · 99.3 tok/s
Prefill, short prompt, median
3151.4 tok/s
Prefill, long prompt
5004 tok/s at 451 prompt tokens
Whole-card memory, median
10941 MiB
Load
5.6 s
Desktop baseline at bench start
1642 MiB one reading for the whole capture
Driver
Not recorded in this file
Resident in GiB
10.68 GiB Computed: 10,941 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
gpt-oss:20bMXFP4 · 8,192 context · LM Studio 0.4.21 · defaults
Measured at 12,310 MiB resident on with LM Studio 0.4.21. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Ran partly off the card
LM Studio's default automatic GPU offload was in effect, at its default 8,192 context. The dataset attributes the lower decode rate to that setting. The normalized row is the same model with full offload. Runtime comparison dataset.
Resident, whole card
12,310 MiB
Measured
Decode
186.26 tok/s
Measured
Left on the card
3,993 MiB
Computed: 16,303 − 12,310 Share of the card
75.5%
Computed: 12,310 ÷ 16,303 Full measured row
Captured
from the file's name App
LM Studio 0.4.21
Engine
llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.1 Condition
defaults
Model name in the file
gpt-oss-20b Quantization
MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
Measured at 13,157 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
13,157 MiB
Measured
Decode
97.02 tok/s
Measured
Left on the card
3,146 MiB
Computed: 16,303 − 13,157 Share of the card
80.7%
Computed: 13,157 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.1
Method
llm-throughput-ollama-v1 Model
qwen2.5:14b digest 7cdf5a0187d5
Parameters
14.8B
Quantization
Q4_K_M
Context (num_ctx) 16384
Prompt cell 1
The row pinned on the map
Prompt tokens
488
Repetitions
3
Decode, median
97.02 tok/s
Prefill, median
5225.96 tok/s
Load, median
131.3 ms
Whole-card memory after the cell
13157 MiB
Prompt cell 2
Same configuration, longer prompt
Prompt tokens
1674
Repetitions
3
Decode, median
94.58 tok/s
Prefill, median
4864.67 tok/s
Load, median
128.9 ms
Whole-card memory after the cell
13159 MiB
Resident in GiB
12.85 GiB Computed: 13,157 ÷ 1,024
Energy for this model, from its own power run
A separate run of qwen2.5:14b on (UTC), stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 13,381 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
A re-capture of the same model. The dataset's change log notes it ran below the 2026-08-12 rows on a card several hours into continuous bench work. Throughput dataset, change log.
Resident, whole card
13,381 MiB
Measured
Decode
181.50 tok/s
Measured
Left on the card
2,922 MiB
Computed: 16,303 − 13,381 Share of the card
82.1%
Computed: 13,381 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.1
Method
llm-throughput-ollama-v1 Model
gpt-oss:20b digest 17052f91a42e Parameters
20.9B
Quantization
MXFP4
Context (num_ctx) 4096
Prompt cell 1
The row pinned on the map
Prompt tokens
503
Repetitions
3
Decode, median
181.5 tok/s
Prefill, median
9607.49 tok/s
Load, median
383.7 ms
Whole-card memory after the cell
13381 MiB
Prompt cell 2
Same configuration, longer prompt
Prompt tokens
1360
Repetitions
3
Decode, median
178.37 tok/s
Prefill, median
11001.37 tok/s
Load, median
375.9 ms
Whole-card memory after the cell
13385 MiB
Resident in GiB
13.07 GiB Computed: 13,381 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 13,570 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
13,570 MiB
Measured
Decode
213.54 tok/s
Measured
Left on the card
2,733 MiB
Computed: 16,303 − 13,570 Share of the card
83.2%
Computed: 13,570 ÷ 16,303 Full measured row
Captured
from the file's name App
ollama 0.32.15
Engine
vendored llama.cpp 9d77fa172
Condition
defaults
Model name in the file
gpt-oss:20b Quantization
MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
Context
4096
Runs
5
Decode, median (server counters) 213.54 tok/s
Decode, median (wall clock) 212.03 tok/s
Each run
213.22 · 213.64 · 213.48 · 213.54 · 213.76 tok/s
Prefill, long prompt, median
9239.4 tok/s at 475 prompt tokens
Time to first token
Empty in this file
Whole-card memory, median
13570 MiB
Whole-card memory before the load
1053 MiB
Added by the model
12,517 MiB Computed: 13,570 − 1,053
Load
18.56 s
Cold first request
23.67 s
App RAM while loaded
2138.8 MB
Driver
Not recorded in this file
Resident in GiB
13.25 GiB Computed: 13,570 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 13,570 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
13,570 MiB
Measured
Decode
213.65 tok/s
Measured
Left on the card
2,733 MiB
Computed: 16,303 − 13,570 Share of the card
83.2%
Computed: 13,570 ÷ 16,303 Full measured row
Captured
from the file's name App
ollama 0.32.15
Engine
vendored llama.cpp 9d77fa172
Condition
normalized
Model name in the file
gpt-oss:20b Quantization
MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
Context
4096
Runs
5
Decode, median (server counters) 213.65 tok/s
Decode, median (wall clock) 211.8 tok/s
Each run
213.62 · 213.65 · 213.68 · 213.25 · 213.73 tok/s
Prefill, long prompt, median
9187.1 tok/s at 474 prompt tokens
Time to first token
Empty in this file
Whole-card memory, median
13570 MiB
Whole-card memory before the load
1053 MiB
Added by the model
12,517 MiB Computed: 13,570 − 1,053
Load
8.05 s
Cold first request
8.36 s
App RAM while loaded
2152.1 MB
Driver
Not recorded in this file
Resident in GiB
13.25 GiB Computed: 13,570 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 13,683 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
A re-capture of the same model. The dataset's change log notes it ran below the 2026-08-12 rows on a card several hours into continuous bench work. Throughput dataset, change log.
Resident, whole card
13,683 MiB
Measured
Decode
183.00 tok/s
Measured
Left on the card
2,620 MiB
Computed: 16,303 − 13,683 Share of the card
83.9%
Computed: 13,683 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.1
Method
llm-throughput-ollama-v1 Model
gpt-oss:20b digest 17052f91a42e Parameters
20.9B
Quantization
MXFP4
Context (num_ctx) 16384
Prompt cell 1
The row pinned on the map
Prompt tokens
503
Repetitions
3
Decode, median
183.0 tok/s
Prefill, median
9092.22 tok/s
Load, median
347.7 ms
Whole-card memory after the cell
13683 MiB
Prompt cell 2
Same configuration, longer prompt
Prompt tokens
1360
Repetitions
3
Decode, median
178.34 tok/s
Prefill, median
10328.62 tok/s
Load, median
345.3 ms
Whole-card memory after the cell
13683 MiB
Resident in GiB
13.36 GiB Computed: 13,683 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 14,067 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
14,067 MiB
Measured
Decode
190.14 tok/s
Measured
Left on the card
2,236 MiB
Computed: 16,303 − 14,067 Share of the card
86.3%
Computed: 14,067 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.1
Method
llm-throughput-ollama-v1 Model
gpt-oss:20b digest 17052f91a42e Parameters
20.9B
Quantization
MXFP4
Context (num_ctx) 4096
Prompt cell 1
The row pinned on the map
Prompt tokens
503
Repetitions
3
Decode, median
190.14 tok/s
Prefill, median
9078.28 tok/s
Load, median
297.7 ms
Whole-card memory after the cell
14067 MiB
Prompt cell 2
Same configuration, longer prompt
Prompt tokens
1658
Repetitions
3
Decode, median
187.27 tok/s
Prefill, median
10464.66 tok/s
Load, median
279.5 ms
Whole-card memory after the cell
14060 MiB
Resident in GiB
13.74 GiB Computed: 14,067 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 14,152 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
14,152 MiB
Measured
Decode
195.79 tok/s
Measured
Left on the card
2,151 MiB
Computed: 16,303 − 14,152 Share of the card
86.8%
Computed: 14,152 ÷ 16,303 Full measured row
Captured
from the file's footer notes Runtime
ollama 0.32.1
Model name in the file
gpt-oss:20b Quantization
MXFP4. Not a column in this file. Taken from the same model in the throughput CSV.
Context
4096 from the file's settings note
Offload
Full offload from the file's settings note
Runs
5
Decode, median (server counters) 195.79 tok/s
Decode, median (wall clock) 194.59 tok/s
Each run
195.8 · 195.7 · 196.3 · 196.1 · 195.7 tok/s
Prefill, short prompt, median
3744.3 tok/s
Prefill, long prompt
Empty in this file
Whole-card memory, median
14152 MiB
Load
8.6 s
Desktop baseline at bench start
1642 MiB one reading for the whole capture
Driver
Not recorded in this file
Resident in GiB
13.82 GiB Computed: 14,152 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 14,355 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Resident, whole card
14,355 MiB
Measured
Decode
188.63 tok/s
Measured
Left on the card
1,948 MiB
Computed: 16,303 − 14,355 Share of the card
88.1%
Computed: 14,355 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.1
Method
llm-throughput-ollama-v1 Model
gpt-oss:20b digest 17052f91a42e Parameters
20.9B
Quantization
MXFP4
Context (num_ctx) 16384
Prompt cell 1
The row pinned on the map
Prompt tokens
503
Repetitions
3
Decode, median
188.63 tok/s
Prefill, median
9286.61 tok/s
Load, median
299.2 ms
Whole-card memory after the cell
14355 MiB
Prompt cell 2
Same configuration, longer prompt
Prompt tokens
1658
Repetitions
3
Decode, median
187.73 tok/s
Prefill, median
10554.32 tok/s
Load, median
280.2 ms
Whole-card memory after the cell
14355 MiB
Resident in GiB
14.02 GiB Computed: 14,355 ÷ 1,024
Energy for this model, from its own power run
A separate run of gpt-oss:20b on (UTC), stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 14,997 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Measured at a 4,096 context only. The dataset reports that an 8K attempt spilled and decode collapsed, but no row for that attempt was published, so it has no pin here. Throughput dataset, coding models note.
Measured at 15,003 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Ran partly off the card
The model file is larger than the card, so Ollama ran it partially offloaded with part of it in system RAM. The dataset warns that its decode rate depends on how much system RAM was free when it loaded. Throughput dataset, coding models note.
Measured at 15,084 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Ran partly off the card
The capture's own ollama ps line reads 22%/78% CPU/GPU at context 131,072, so part of this model ran from system RAM.
Ollama's app-level context slider was set to 256k, which this model clamps to 131,072. The dataset describes the result as the KV cache spilling to the CPU. Runtime comparison dataset.
Resident, whole card
15,084 MiB
Measured
Decode
102.68 tok/s
Measured
Left on the card
1,219 MiB
Computed: 16,303 − 15,084 Share of the card
92.5%
Computed: 15,084 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.15
Condition
defaults
Schema
techfuelhq:lmstudio-vs-ollama:1 Model
llama3.2:3b
Quantization
Q4_K_M. Not a field in this file. Taken from the same model in the throughput CSV.
Context
131072 from the capture's ollama ps line
Processor split
22% CPU / 78% GPU from the capture's ollama ps line
Size reported by ollama ps
18 GB
Runs
5
Decode, median (server counters) 102.68 tok/s
Decode, median (wall clock) 101.58 tok/s
Each run
97.57 · 99.85 · 103.59 · 102.68 · 103.21 tok/s
Prefill, long prompt, median
9998.8 tok/s at 434 prompt tokens
Whole-card memory, median
15084 MiB
Whole-card memory before the load
1067 MiB
Added by the model
14,017 MiB Computed: 15,084 − 1,067
Load
4.02 s
Cold first request
4.2 s
App RAM while loaded
4745.7 MB
Resident in GiB
14.73 GiB Computed: 15,084 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
Measured at 15,101 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Ran partly off the card
The model file is larger than the card, so Ollama ran it partially offloaded with part of it in system RAM. The dataset warns that its decode rate depends on how much system RAM was free when it loaded. Throughput dataset, coding models note.
Measured at 15,682 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included.
Reading includes another model
gpt-oss:20b was still loaded from the cell before this one, so this reading is the whole card with two models on it, not Llama 3.2 3B alone. The 16,384-context row from the same capture was read after the larger model had been evicted. Throughput dataset, VRAM note.
Resident, whole card
15,682 MiB
Measured
Decode
311.96 tok/s
Measured
Left on the card
621 MiB
Computed: 16,303 − 15,682 Share of the card
96.2%
Computed: 15,682 ÷ 16,303 Full measured row
Captured (UTC) GPU
NVIDIA GeForce RTX 5080
Driver
610.88
Runtime
Ollama 0.32.1
Method
llm-throughput-ollama-v1 Model
llama3.2:3b digest a80c4f17acd5
Parameters
3.2B
Quantization
Q4_K_M
Context (num_ctx) 4096
Prompt cell 1
The row pinned on the map
Prompt tokens
471
Repetitions
3
Decode, median
311.96 tok/s
Prefill, median
18196.57 tok/s
Load, median
162.8 ms
Whole-card memory after the cell
15682 MiB
Prompt cell 2
Same configuration, longer prompt
Prompt tokens
1657
Repetitions
3
Decode, median
298.21 tok/s
Prefill, median
17681.08 tok/s
Load, median
156.2 ms
Whole-card memory after the cell
15682 MiB
Resident in GiB
15.31 GiB Computed: 15,682 ÷ 1,024
Energy for this model, from its own power run
A separate run of llama3.2:3b on (UTC), stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's.
No measured configuration matches those filters. That combination was not on the bench, and I do not estimate here. For an estimate, use the LLM VRAM calculator.
How to read it
The map answers a narrow question: what did this model, at this context length, on this runtime version, take on a 16 GB card, and what speed came with it. Each record reads “measured at so many MiB resident on a date with a runtime version” because that is all a measurement can say. It is not a promise that the same model fits on your card today.
The line along the bottom is the card, from zero to the total it reports to nvidia-smi. A pin’s position across is the memory in use on the whole card while that model was loaded, Windows desktop included, and its height is the decode rate. Select a pin and the line fills to it in the measured colour. The hatched remainder is headroom, which I compute by subtracting.
A dashed connector joins the same model, runtime and capture at two context lengths. Flat means memory grew and speed held; a drop means the model stopped fitting. A ring is a run recorded as partly off the card. A diamond is a reading taken while another model was still loaded, so it overstates that model.
Where the cliff shows up
The cliff is not a memory number. In these files, a model that sits whole on the card decodes at the same speed whether it uses a third of the memory or nearly all of it. Speed falls when part of the model no longer fits and runs from system RAM instead.
Each row is the same model, runtime version and capture at two context lengths, and both ends are measured rows. Memory change is the second reading minus the first. Decode change is (second − first) ÷ first.
Model and runtimeContextResident MiB, measuredMemory change, computedDecode tok/s, measuredDecode change, computed
llama3.2:3bOllama 0.32.15, defaults · 131,072: ran partly off the card4,096 → 131,0723,760 → 15,084+11,324351.22 → 102.68−70.8%
qwen3-coder:30bOllama 0.32.1 · 16,384: ran partly off the card4,096 → 16,38415,101 → 15,003−9861.89 → 51.62−16.6%
The table has two rows where raising the context cost speed, and both are runs that left the card. Llama 3.2 3B is the small model in this set, and a large enough context still pushed it over. Qwen3-Coder 30B never fit whole, so its memory reading barely moved and its speed did.
Every other pair held its decode rate within one percent while the context quadrupled. What they paid was memory, and how much depended on the model: the two 14B Qwen models paid the most, and gpt-oss:20b, the largest of them by parameter count, paid the least.
What the files do not hold is a dense model caught in the act of falling off. The throughput dataset says Devstral 24B collapsed when I tried it at an 8K context, but that attempt was never written to the CSV, so there is no pin for it and I have not drawn one.
Energy per thousand tokens
llama3.2:3bQ4_K_M · named “Llama 3.2 3B” in the file · run logged UTC
833 J per 1,000 tokens Computed: 0.833 J per token × 1,000
Decode, median of 3
318.3 tok/s
Board power, median while loaded
265.2 W
Tokens per joule, as filed
1.2
Joules per token, as filed
0.833
More from the power file
Watt-hours per 1,000 tokens
0.231 Wh Computed: 231.0 Wh per million ÷ 1,000
Decode, slowest and fastest run
315.5 to 318.5 tok/s
Board power, peak
307.78 W
Memory in use, median
3,590 MiB
GPU temperature, median
49.0 °C
GPU load, median
82.0%
Telemetry samples while loaded
9
qwen2.5:7bQ4_K_M · named “Qwen 2.5 7B” in the file · run logged UTC
1,886 J per 1,000 tokens Computed: 1.886 J per token × 1,000
Decode, median of 3
182.2 tok/s
Board power, median while loaded
343.6 W
Tokens per joule, as filed
0.53
Joules per token, as filed
1.886
More from the power file
Watt-hours per 1,000 tokens
0.524 Wh Computed: 524.0 Wh per million ÷ 1,000
Decode, slowest and fastest run
182.1 to 182.3 tok/s
Board power, peak
346.08 W
Memory in use, median
5,665 MiB
GPU temperature, median
52.0 °C
GPU load, median
88.0%
Telemetry samples while loaded
14
qwen2.5:14bQ4_K_M · named “Qwen 2.5 14B” in the file · run logged UTC
3,154 J per 1,000 tokens Computed: 3.154 J per token × 1,000
Decode, median of 3
98.8 tok/s
Board power, median while loaded
311.6 W
Tokens per joule, as filed
0.317
Joules per token, as filed
3.154
More from the power file
Watt-hours per 1,000 tokens
0.876 Wh Computed: 876.0 Wh per million ÷ 1,000
Decode, slowest and fastest run
98.6 to 98.8 tok/s
Board power, peak
319.69 W
Memory in use, median
10,171 MiB
GPU temperature, median
50.0 °C
GPU load, median
93.0%
Telemetry samples while loaded
27
gpt-oss:20bMXFP4 · named “gpt-oss 20B” in the file · run logged UTC
1,504 J per 1,000 tokens Computed: 1.504 J per token × 1,000
Decode, median of 3
194.4 tok/s
Board power, median while loaded
292.4 W
Tokens per joule, as filed
0.665
Joules per token, as filed
1.504
More from the power file
Watt-hours per 1,000 tokens
0.418 Wh Computed: 418.0 Wh per million ÷ 1,000
Decode, slowest and fastest run
194.3 to 194.5 tok/s
Board power, peak
294.86 W
Memory in use, median
13,381 MiB
GPU temperature, median
51.0 °C
GPU load, median
84.0%
Telemetry samples while loaded
18
These four runs are a separate capture from the map: one short prompt, three iterations per model, board power sampled once a second and counted only while the GPU was at least half loaded. They were taken at the card’s stock 448 W power limit, which none of the runs reached. Board power is the graphics card alone, not the wall.
The power file records no context length, driver or runtime version, which is why these runs are not pinned on the map. Its dataset page gives the runtime as Ollama 0.32.1.
There is no undervolt or power-limit comparison here because I have not measured one for language models. My undervolt database holds one row for this card and it is a stock 3DMark baseline. Method and the idle-power finding are on the power efficiency dataset.
Versions on the bench, and how stale they are
Driver and runtime versions per capture, exactly as each file records them. Where a file is silent, the cell says so.
CapturedFileGPU driverRuntime versionsRows used here
llm-server-compare-2026-08-26-lmstudio.csvNot recorded in this fileLM Studio 0.4.21, Ollama 0.32.15Ollama 0.32.15: vendored llama.cpp 9d77fa172LM Studio 0.4.21: llama.cpp-win-x86_64-nvidia-cuda12-avx2@2.29.112 configurations
Those are the versions that were installed on the capture dates. Drivers and runtimes have both moved since, and I have not re-run this matrix on newer ones. Read every pin as true for its date and versions, and expect a current install to differ.
The second table is why I say that plainly. Two captures six days apart put the same three models on two Ollama versions, and all three decoded faster on the later one. The runtime comparison dataset notes that the earlier capture had less control over background load, so not all of that gap is the version. Either way, a few percent between runtimes does not survive an update.
What the data covers
Model tags (8) llama3.2:3b 3.2B · Q4_K_M 9 configurations
qwen2.5-coder:7b 7.6B · Q4_K_M 2 configurations
qwen2.5:7b 7.6B · Q4_K_M 2 configurations
qwen2.5-coder:14b 14.8B · Q4_K_M 2 configurations
qwen2.5:14b 14.8B · Q4_K_M 8 configurations
gpt-oss:20b 20.9B · MXFP4 10 configurations
devstral:24b 23.6B · Q4_K_M 1 configuration
qwen3-coder:30b 30.5B · Q4_K_M 2 configurations
Quantizations (2) MXFP410
Q4_K_M26
No model was measured at more than one quantization.
One quantization per model. I cannot show the same model at three quantizations stepping down the card, because each model was measured at one.
No dense model past the cliff. The runs that left the card are a 3B model at an extreme context, a mixture-of-experts model that never fit, and one LM Studio default that offloaded part of a model that otherwise fits. A dense model that fits at one context and spills at the next is the row this map most needs.
A gap between 16,384 and 131,072 context. One run sits at the far end. Nothing was measured at 32K or 64K.
Energy at stock only. No undervolt, no reduced power limit, and no energy figure for a run that spilled.
One card. Another 16 GB card, or this one with a different desktop load, will read differently.
Method, exactly
I did not run anything new for this page. It reads files I had already published and puts them on one axis.
The throughput rows come from my capture script. For each model, context and prompt size it calls Ollama’s /api/generate three times at temperature 0, seed 42 and a 256-token output, with a unique prefix on every prompt so each run pays a full prefill. Decode and prefill rates are Ollama’s own eval_count over eval_duration and prompt_eval_count over prompt_eval_duration, and the CSV keeps the median of the three. After each cell the script asks nvidia-smi for memory.used and stores it as vram_resident_mib. That one reading is the pin’s position.
That reading is the whole card. It includes the desktop, and it includes any model Ollama had not yet unloaded, which is how one Llama 3.2 3B row came to read 15,682 MiB. The runtime comparison captures used a different script that took the median memory reading across runs and, in the later capture, also read the card before each load.
Each throughput configuration was measured with two prompt sizes. The pin is the first of the two, the shorter prompt, and both are in the record. The model names differ between files, so I join them on the name with punctuation and a trailing quantization tag removed. Only the throughput CSV has a quantization column; where another file’s row shows a quantization, the record says it was taken from the same model there.
The card total, 16,303 MiB, is the nvidia-smi total written into the raw capture file from 2026-08-26. Headroom, share of the card, GiB, the memory a load added, the changes between two rows and the per-thousand-token energy are computed when the site is built, from the measured values beside them, and each one is shown with its formula. Your browser does no arithmetic on them.
Download the files
rtx-5080-llm-throughput.csv34 rows. Decode, prefill, load time and whole-card memory for each model, context and prompt size on Ollama. Raw capture JSON and the capture script are listed in its manifest.
llm-server-compare-2026-08.csv6 rows. Ollama beside llama.cpp on three models, with its settings and capture date in the footer notes.
llm-server-compare-2026-08-26-lmstudio.csv12 rows. Ollama beside LM Studio on three models, at default and normalized settings, with the memory reading before each load.
All of it is CC BY 4.0. Attribute as “TechFuelHQ RTX 5080 16 GB VRAM cliff map” with a link to this page. To cite one configuration, open its row in the list and copy its link; the address ends in that row’s own id.