The 16 GB Cliff: Measured LLM Memory and Speed on an RTX 5080 A hands-on benchmark of 36 measured local language-model configurations on a retail Nvidia RTX 5080 found that llama3.2:3b at Q4_K_M quantization and 4,096 context ran at a median 351.22 tok/s decode while occupying 3,760 MiB of the card's 16,303 MiB, or 23.1% of total memory. The same model dropped from 351.22 tok/s at 4,096 context to 102.68 tok/s at 131,072 context, and a separate stock-power run measured 318.3 tok/s at 265.2 W board power, or 0.833 J per token. All figures were captured with Ollama 0.32.15 on its vendored llama.cpp build 9d77fa172, with whole-card memory read from nvidia-smi including the Windows desktop. Every pin is a local language model I ran on my own retail RTX 5080, placed by the memory in use on the whole card and the decode speed that came with it. Nothing here is estimated. For a card I do not own or a setup I have not run, the LLM VRAM calculator and the LLM speed calculator estimate, and the VRAM tiers guide covers other card sizes. This page only reports. Measured. Copied from a capture file, shown with its capture date. Computed. Arithmetic on measured values, shown with its formula. Showing 36 of 36 measured configurations The map Across: memory in use on the whole card. Up: decode speed. tok/s0100200300400 llama3.2:3b, 4,096 to 131,072 context351.22 to 102.68 tok/s 04,0968,19212,28816,303MiB in usecard total A measured configuration Ran partly off the card Reading includes another model Same setup, context raised Every pin is one row from a capture file. Nothing between two pins was measured, and nothing here is an estimate. Readout Nothing selected Click or tap near a pin, or pick a row in the list, and its measured record appears here. Measured at 3,760 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 3,760 MiB Measured Decode 351.22 tok/s Measured Left on the card 12,543 MiB Computed: 16,303 − 3,760 Share of the card 23.1% Computed: 3,760 ÷ 16,303 Full measured row Captured from the file's name App ollama 0.32.15 Engine vendored llama.cpp 9d77fa172 Condition defaults Model name in the file llama3.2:3b Quantization Q4 K M. Not a column in this file. Taken from the same model in the throughput CSV. Context 4096 Runs 5 Decode, median server counters 351.22 tok/s Decode, median wall clock 345.19 tok/s Each run 351.15 · 350.9 · 351.86 · 351.32 · 351.22 tok/s Prefill, long prompt, median 17802.7 tok/s at 434 prompt tokens Time to first token Empty in this file Whole-card memory, median 3760 MiB Whole-card memory before the load 1053 MiB Added by loading the model 2,707 MiB Computed: 3,760 − 1,053 Load 2.67 s Cold first request 2.79 s App RAM while loaded 1234.0 MB Driver Not recorded in this file Resident in GiB 3.67 GiB Computed: 3,760 ÷ 1,024 Energy for this model, from its own power run A separate run of llama3.2:3b on UTC , stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 3,760 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 3,760 MiB Measured Decode 350.52 tok/s Measured Left on the card 12,543 MiB Computed: 16,303 − 3,760 Share of the card 23.1% Computed: 3,760 ÷ 16,303 Full measured row Captured from the file's name App ollama 0.32.15 Engine vendored llama.cpp 9d77fa172 Condition normalized Model name in the file llama3.2:3b Quantization Q4 K M. Not a column in this file. Taken from the same model in the throughput CSV. Context 4096 Runs 5 Decode, median server counters 350.52 tok/s Decode, median wall clock 338.11 tok/s Each run 349.35 · 350.38 · 350.52 · 350.63 · 350.96 tok/s Prefill, long prompt, median 17862.3 tok/s at 432 prompt tokens Time to first token Empty in this file Whole-card memory, median 3760 MiB Whole-card memory before the load 1057 MiB Added by loading the model 2,703 MiB Computed: 3,760 − 1,057 Load 2.27 s Cold first request 2.38 s App RAM while loaded 1239.6 MB Driver Not recorded in this file Resident in GiB 3.67 GiB Computed: 3,760 ÷ 1,024 Energy for this model, from its own power run A separate run of llama3.2:3b on UTC , stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 4,343 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 4,343 MiB Measured Decode 318.51 tok/s Measured Left on the card 11,960 MiB Computed: 16,303 − 4,343 Share of the card 26.6% Computed: 4,343 ÷ 16,303 Full measured row Captured from the file's footer notes Runtime ollama 0.32.1 Model name in the file llama3.2:3b Quantization Q4 K M. Not a column in this file. Taken from the same model in the throughput CSV. Context 4096 from the file's settings note Offload Full offload from the file's settings note Runs 5 Decode, median server counters 318.51 tok/s Decode, median wall clock 313.25 tok/s Each run 319.0 · 318.7 · 318.2 · 318.5 · 317.5 tok/s Prefill, short prompt, median 7038.1 tok/s Prefill, long prompt 17444 tok/s at 433 prompt tokens Whole-card memory, median 4343 MiB Load 3.2 s Desktop baseline at bench start 1642 MiB one reading for the whole capture Driver Not recorded in this file Resident in GiB 4.24 GiB Computed: 4,343 ÷ 1,024 Energy for this model, from its own power run A separate run of llama3.2:3b on UTC , stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 5,495 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 5,495 MiB Measured Decode 312.89 tok/s Measured Left on the card 10,808 MiB Computed: 16,303 − 5,495 Share of the card 33.7% Computed: 5,495 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.1 Method llm-throughput-ollama-v1 Model llama3.2:3b digest a80c4f17acd5 Parameters 3.2B Quantization Q4 K M Context num ctx 16384 Prompt cell 1 The row pinned on the map Prompt tokens 471 Repetitions 3 Decode, median 312.89 tok/s Prefill, median 17885.62 tok/s Load, median 162.9 ms Whole-card memory after the cell 5495 MiB Prompt cell 2 Same configuration, longer prompt Prompt tokens 1657 Repetitions 3 Decode, median 298.59 tok/s Prefill, median 19397.36 tok/s Load, median 158.0 ms Whole-card memory after the cell 5501 MiB Resident in GiB 5.37 GiB Computed: 5,495 ÷ 1,024 Energy for this model, from its own power run A separate run of llama3.2:3b on UTC , stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 6,190 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 6,190 MiB Measured Decode 180.02 tok/s Measured Left on the card 10,113 MiB Computed: 16,303 − 6,190 Share of the card 38.0% Computed: 6,190 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.1 Method llm-throughput-ollama-v1 Model qwen2.5:7b digest 845dbda0ea48 Parameters 7.6B Quantization Q4 K M Context num ctx 4096 Prompt cell 1 The row pinned on the map Prompt tokens 488 Repetitions 3 Decode, median 180.02 tok/s Prefill, median 10152.28 tok/s Load, median 129.8 ms Whole-card memory after the cell 6190 MiB Prompt cell 2 Same configuration, longer prompt Prompt tokens 1674 Repetitions 3 Decode, median 176.79 tok/s Prefill, median 9672.39 tok/s Load, median 123.4 ms Whole-card memory after the cell 6190 MiB Resident in GiB 6.04 GiB Computed: 6,190 ÷ 1,024 Energy for this model, from its own power run A separate run of qwen2.5:7b on UTC , stock power limit: 182.2 tok/s at 343.6 W board power, 1.886 J per token in the file. That is 1,886 J per 1,000 tokensComputed: 1.886 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 7,022 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 7,022 MiB Measured Decode 179.66 tok/s Measured Left on the card 9,281 MiB Computed: 16,303 − 7,022 Share of the card 43.1% Computed: 7,022 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.1 Method llm-throughput-ollama-v1 Model qwen2.5:7b digest 845dbda0ea48 Parameters 7.6B Quantization Q4 K M Context num ctx 16384 Prompt cell 1 The row pinned on the map Prompt tokens 488 Repetitions 3 Decode, median 179.66 tok/s Prefill, median 10133.1 tok/s Load, median 127.9 ms Whole-card memory after the cell 7022 MiB Prompt cell 2 Same configuration, longer prompt Prompt tokens 1674 Repetitions 3 Decode, median 176.61 tok/s Prefill, median 9984.37 tok/s Load, median 127.7 ms Whole-card memory after the cell 7034 MiB Resident in GiB 6.86 GiB Computed: 7,022 ÷ 1,024 Energy for this model, from its own power run A separate run of qwen2.5:7b on UTC , stock power limit: 182.2 tok/s at 343.6 W board power, 1.886 J per token in the file. That is 1,886 J per 1,000 tokensComputed: 1.886 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 10,356 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 10,356 MiB Measured Decode 102.25 tok/s Measured Left on the card 5,947 MiB Computed: 16,303 − 10,356 Share of the card 63.5% Computed: 10,356 ÷ 16,303 Full measured row Captured from the file's name App ollama 0.32.15 Engine vendored llama.cpp 9d77fa172 Condition defaults Model name in the file qwen2.5:14b Quantization Q4 K M. Not a column in this file. Taken from the same model in the throughput CSV. Context 4096 Runs 5 Decode, median server counters 102.25 tok/s Decode, median wall clock 101.06 tok/s Each run 102.13 · 102.34 · 102.25 · 102.3 · 102.25 tok/s Prefill, long prompt, median 4983.8 tok/s at 453 prompt tokens Time to first token Empty in this file Whole-card memory, median 10356 MiB Whole-card memory before the load 1053 MiB Added by loading the model 9,303 MiB Computed: 10,356 − 1,053 Load 5.51 s Cold first request 5.67 s App RAM while loaded 1604.1 MB Driver Not recorded in this file Resident in GiB 10.11 GiB Computed: 10,356 ÷ 1,024 Energy for this model, from its own power run A separate run of qwen2.5:14b on UTC , stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 10,356 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 10,356 MiB Measured Decode 102.21 tok/s Measured Left on the card 5,947 MiB Computed: 16,303 − 10,356 Share of the card 63.5% Computed: 10,356 ÷ 16,303 Full measured row Captured from the file's name App ollama 0.32.15 Engine vendored llama.cpp 9d77fa172 Condition normalized Model name in the file qwen2.5:14b Quantization Q4 K M. Not a column in this file. Taken from the same model in the throughput CSV. Context 4096 Runs 5 Decode, median server counters 102.21 tok/s Decode, median wall clock 101.23 tok/s Each run 102.21 · 102.21 · 102.17 · 102.21 · 102.21 tok/s Prefill, long prompt, median 5038.3 tok/s at 452 prompt tokens Time to first token Empty in this file Whole-card memory, median 10356 MiB Whole-card memory before the load 1053 MiB Added by loading the model 9,303 MiB Computed: 10,356 − 1,053 Load 5.52 s Cold first request 5.68 s App RAM while loaded 1617.6 MB Driver Not recorded in this file Resident in GiB 10.11 GiB Computed: 10,356 ÷ 1,024 Energy for this model, from its own power run A separate run of qwen2.5:14b on UTC , stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 10,841 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 10,841 MiB Measured Decode 96.65 tok/s Measured Left on the card 5,462 MiB Computed: 16,303 − 10,841 Share of the card 66.5% Computed: 10,841 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.1 Method llm-throughput-ollama-v1 Model qwen2.5:14b digest 7cdf5a0187d5 Parameters 14.8B Quantization Q4 K M Context num ctx 4096 Prompt cell 1 The row pinned on the map Prompt tokens 488 Repetitions 3 Decode, median 96.65 tok/s Prefill, median 5202.23 tok/s Load, median 182.5 ms Whole-card memory after the cell 10841 MiB Prompt cell 2 Same configuration, longer prompt Prompt tokens 1674 Repetitions 3 Decode, median 94.48 tok/s Prefill, median 4876.41 tok/s Load, median 125.8 ms Whole-card memory after the cell 10855 MiB Resident in GiB 10.59 GiB Computed: 10,841 ÷ 1,024 Energy for this model, from its own power run A separate run of qwen2.5:14b on UTC , stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 10,941 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 10,941 MiB Measured Decode 99.43 tok/s Measured Left on the card 5,362 MiB Computed: 16,303 − 10,941 Share of the card 67.1% Computed: 10,941 ÷ 16,303 Full measured row Captured from the file's footer notes Runtime ollama 0.32.1 Model name in the file qwen2.5:14b Quantization Q4 K M. Not a column in this file. Taken from the same model in the throughput CSV. Context 4096 from the file's settings note Offload Full offload from the file's settings note Runs 5 Decode, median server counters 99.43 tok/s Decode, median wall clock 98.59 tok/s Each run 99.5 · 99.5 · 99.4 · 99.4 · 99.3 tok/s Prefill, short prompt, median 3151.4 tok/s Prefill, long prompt 5004 tok/s at 451 prompt tokens Whole-card memory, median 10941 MiB Load 5.6 s Desktop baseline at bench start 1642 MiB one reading for the whole capture Driver Not recorded in this file Resident in GiB 10.68 GiB Computed: 10,941 ÷ 1,024 Energy for this model, from its own power run A separate run of qwen2.5:14b on UTC , stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. gpt-oss:20bMXFP4 · 8,192 context · LM Studio 0.4.21 · defaults Measured at 12,310 MiB resident on with LM Studio 0.4.21. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Ran partly off the card LM Studio's default automatic GPU offload was in effect, at its default 8,192 context. The dataset attributes the lower decode rate to that setting. The normalized row is the same model with full offload. Runtime comparison dataset. Resident, whole card 12,310 MiB Measured Decode 186.26 tok/s Measured Left on the card 3,993 MiB Computed: 16,303 − 12,310 Share of the card 75.5% Computed: 12,310 ÷ 16,303 Full measured row Captured from the file's name App LM Studio 0.4.21 Engine llama.cpp-win-x86 64-nvidia-cuda12-avx2@2.29.1 Condition defaults Model name in the file gpt-oss-20b Quantization MXFP4. Not a column in this file. Taken from the same model in the throughput CSV. Measured at 13,157 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 13,157 MiB Measured Decode 97.02 tok/s Measured Left on the card 3,146 MiB Computed: 16,303 − 13,157 Share of the card 80.7% Computed: 13,157 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.1 Method llm-throughput-ollama-v1 Model qwen2.5:14b digest 7cdf5a0187d5 Parameters 14.8B Quantization Q4 K M Context num ctx 16384 Prompt cell 1 The row pinned on the map Prompt tokens 488 Repetitions 3 Decode, median 97.02 tok/s Prefill, median 5225.96 tok/s Load, median 131.3 ms Whole-card memory after the cell 13157 MiB Prompt cell 2 Same configuration, longer prompt Prompt tokens 1674 Repetitions 3 Decode, median 94.58 tok/s Prefill, median 4864.67 tok/s Load, median 128.9 ms Whole-card memory after the cell 13159 MiB Resident in GiB 12.85 GiB Computed: 13,157 ÷ 1,024 Energy for this model, from its own power run A separate run of qwen2.5:14b on UTC , stock power limit: 98.8 tok/s at 311.6 W board power, 3.154 J per token in the file. That is 3,154 J per 1,000 tokensComputed: 3.154 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 13,381 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. A re-capture of the same model. The dataset's change log notes it ran below the 2026-08-12 rows on a card several hours into continuous bench work. Throughput dataset, change log. Resident, whole card 13,381 MiB Measured Decode 181.50 tok/s Measured Left on the card 2,922 MiB Computed: 16,303 − 13,381 Share of the card 82.1% Computed: 13,381 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.1 Method llm-throughput-ollama-v1 Model gpt-oss:20b digest 17052f91a42e Parameters 20.9B Quantization MXFP4 Context num ctx 4096 Prompt cell 1 The row pinned on the map Prompt tokens 503 Repetitions 3 Decode, median 181.5 tok/s Prefill, median 9607.49 tok/s Load, median 383.7 ms Whole-card memory after the cell 13381 MiB Prompt cell 2 Same configuration, longer prompt Prompt tokens 1360 Repetitions 3 Decode, median 178.37 tok/s Prefill, median 11001.37 tok/s Load, median 375.9 ms Whole-card memory after the cell 13385 MiB Resident in GiB 13.07 GiB Computed: 13,381 ÷ 1,024 Energy for this model, from its own power run A separate run of gpt-oss:20b on UTC , stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 13,570 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 13,570 MiB Measured Decode 213.54 tok/s Measured Left on the card 2,733 MiB Computed: 16,303 − 13,570 Share of the card 83.2% Computed: 13,570 ÷ 16,303 Full measured row Captured from the file's name App ollama 0.32.15 Engine vendored llama.cpp 9d77fa172 Condition defaults Model name in the file gpt-oss:20b Quantization MXFP4. Not a column in this file. Taken from the same model in the throughput CSV. Context 4096 Runs 5 Decode, median server counters 213.54 tok/s Decode, median wall clock 212.03 tok/s Each run 213.22 · 213.64 · 213.48 · 213.54 · 213.76 tok/s Prefill, long prompt, median 9239.4 tok/s at 475 prompt tokens Time to first token Empty in this file Whole-card memory, median 13570 MiB Whole-card memory before the load 1053 MiB Added by loading the model 12,517 MiB Computed: 13,570 − 1,053 Load 18.56 s Cold first request 23.67 s App RAM while loaded 2138.8 MB Driver Not recorded in this file Resident in GiB 13.25 GiB Computed: 13,570 ÷ 1,024 Energy for this model, from its own power run A separate run of gpt-oss:20b on UTC , stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 13,570 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 13,570 MiB Measured Decode 213.65 tok/s Measured Left on the card 2,733 MiB Computed: 16,303 − 13,570 Share of the card 83.2% Computed: 13,570 ÷ 16,303 Full measured row Captured from the file's name App ollama 0.32.15 Engine vendored llama.cpp 9d77fa172 Condition normalized Model name in the file gpt-oss:20b Quantization MXFP4. Not a column in this file. Taken from the same model in the throughput CSV. Context 4096 Runs 5 Decode, median server counters 213.65 tok/s Decode, median wall clock 211.8 tok/s Each run 213.62 · 213.65 · 213.68 · 213.25 · 213.73 tok/s Prefill, long prompt, median 9187.1 tok/s at 474 prompt tokens Time to first token Empty in this file Whole-card memory, median 13570 MiB Whole-card memory before the load 1053 MiB Added by loading the model 12,517 MiB Computed: 13,570 − 1,053 Load 8.05 s Cold first request 8.36 s App RAM while loaded 2152.1 MB Driver Not recorded in this file Resident in GiB 13.25 GiB Computed: 13,570 ÷ 1,024 Energy for this model, from its own power run A separate run of gpt-oss:20b on UTC , stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 13,683 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. A re-capture of the same model. The dataset's change log notes it ran below the 2026-08-12 rows on a card several hours into continuous bench work. Throughput dataset, change log. Resident, whole card 13,683 MiB Measured Decode 183.00 tok/s Measured Left on the card 2,620 MiB Computed: 16,303 − 13,683 Share of the card 83.9% Computed: 13,683 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.1 Method llm-throughput-ollama-v1 Model gpt-oss:20b digest 17052f91a42e Parameters 20.9B Quantization MXFP4 Context num ctx 16384 Prompt cell 1 The row pinned on the map Prompt tokens 503 Repetitions 3 Decode, median 183.0 tok/s Prefill, median 9092.22 tok/s Load, median 347.7 ms Whole-card memory after the cell 13683 MiB Prompt cell 2 Same configuration, longer prompt Prompt tokens 1360 Repetitions 3 Decode, median 178.34 tok/s Prefill, median 10328.62 tok/s Load, median 345.3 ms Whole-card memory after the cell 13683 MiB Resident in GiB 13.36 GiB Computed: 13,683 ÷ 1,024 Energy for this model, from its own power run A separate run of gpt-oss:20b on UTC , stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 14,067 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 14,067 MiB Measured Decode 190.14 tok/s Measured Left on the card 2,236 MiB Computed: 16,303 − 14,067 Share of the card 86.3% Computed: 14,067 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.1 Method llm-throughput-ollama-v1 Model gpt-oss:20b digest 17052f91a42e Parameters 20.9B Quantization MXFP4 Context num ctx 4096 Prompt cell 1 The row pinned on the map Prompt tokens 503 Repetitions 3 Decode, median 190.14 tok/s Prefill, median 9078.28 tok/s Load, median 297.7 ms Whole-card memory after the cell 14067 MiB Prompt cell 2 Same configuration, longer prompt Prompt tokens 1658 Repetitions 3 Decode, median 187.27 tok/s Prefill, median 10464.66 tok/s Load, median 279.5 ms Whole-card memory after the cell 14060 MiB Resident in GiB 13.74 GiB Computed: 14,067 ÷ 1,024 Energy for this model, from its own power run A separate run of gpt-oss:20b on UTC , stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 14,152 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 14,152 MiB Measured Decode 195.79 tok/s Measured Left on the card 2,151 MiB Computed: 16,303 − 14,152 Share of the card 86.8% Computed: 14,152 ÷ 16,303 Full measured row Captured from the file's footer notes Runtime ollama 0.32.1 Model name in the file gpt-oss:20b Quantization MXFP4. Not a column in this file. Taken from the same model in the throughput CSV. Context 4096 from the file's settings note Offload Full offload from the file's settings note Runs 5 Decode, median server counters 195.79 tok/s Decode, median wall clock 194.59 tok/s Each run 195.8 · 195.7 · 196.3 · 196.1 · 195.7 tok/s Prefill, short prompt, median 3744.3 tok/s Prefill, long prompt Empty in this file Whole-card memory, median 14152 MiB Load 8.6 s Desktop baseline at bench start 1642 MiB one reading for the whole capture Driver Not recorded in this file Resident in GiB 13.82 GiB Computed: 14,152 ÷ 1,024 Energy for this model, from its own power run A separate run of gpt-oss:20b on UTC , stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 14,355 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Resident, whole card 14,355 MiB Measured Decode 188.63 tok/s Measured Left on the card 1,948 MiB Computed: 16,303 − 14,355 Share of the card 88.1% Computed: 14,355 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.1 Method llm-throughput-ollama-v1 Model gpt-oss:20b digest 17052f91a42e Parameters 20.9B Quantization MXFP4 Context num ctx 16384 Prompt cell 1 The row pinned on the map Prompt tokens 503 Repetitions 3 Decode, median 188.63 tok/s Prefill, median 9286.61 tok/s Load, median 299.2 ms Whole-card memory after the cell 14355 MiB Prompt cell 2 Same configuration, longer prompt Prompt tokens 1658 Repetitions 3 Decode, median 187.73 tok/s Prefill, median 10554.32 tok/s Load, median 280.2 ms Whole-card memory after the cell 14355 MiB Resident in GiB 14.02 GiB Computed: 14,355 ÷ 1,024 Energy for this model, from its own power run A separate run of gpt-oss:20b on UTC , stock power limit: 194.4 tok/s at 292.4 W board power, 1.504 J per token in the file. That is 1,504 J per 1,000 tokensComputed: 1.504 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 14,997 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Measured at a 4,096 context only. The dataset reports that an 8K attempt spilled and decode collapsed, but no row for that attempt was published, so it has no pin here. Throughput dataset, coding models note. Measured at 15,003 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Ran partly off the card The model file is larger than the card, so Ollama ran it partially offloaded with part of it in system RAM. The dataset warns that its decode rate depends on how much system RAM was free when it loaded. Throughput dataset, coding models note. Measured at 15,084 MiB resident on with Ollama 0.32.15. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Ran partly off the card The capture's own ollama ps line reads 22%/78% CPU/GPU at context 131,072, so part of this model ran from system RAM. Ollama's app-level context slider was set to 256k, which this model clamps to 131,072. The dataset describes the result as the KV cache spilling to the CPU. Runtime comparison dataset. Resident, whole card 15,084 MiB Measured Decode 102.68 tok/s Measured Left on the card 1,219 MiB Computed: 16,303 − 15,084 Share of the card 92.5% Computed: 15,084 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.15 Condition defaults Schema techfuelhq:lmstudio-vs-ollama:1 Model llama3.2:3b Quantization Q4 K M. Not a field in this file. Taken from the same model in the throughput CSV. Context 131072 from the capture's ollama ps line Processor split 22% CPU / 78% GPU from the capture's ollama ps line Size reported by ollama ps 18 GB Runs 5 Decode, median server counters 102.68 tok/s Decode, median wall clock 101.58 tok/s Each run 97.57 · 99.85 · 103.59 · 102.68 · 103.21 tok/s Prefill, long prompt, median 9998.8 tok/s at 434 prompt tokens Whole-card memory, median 15084 MiB Whole-card memory before the load 1067 MiB Added by loading the model 14,017 MiB Computed: 15,084 − 1,067 Load 4.02 s Cold first request 4.2 s App RAM while loaded 4745.7 MB Resident in GiB 14.73 GiB Computed: 15,084 ÷ 1,024 Energy for this model, from its own power run A separate run of llama3.2:3b on UTC , stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. Measured at 15,101 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Ran partly off the card The model file is larger than the card, so Ollama ran it partially offloaded with part of it in system RAM. The dataset warns that its decode rate depends on how much system RAM was free when it loaded. Throughput dataset, coding models note. Measured at 15,682 MiB resident on with Ollama 0.32.1. That figure is the whole card as nvidia-smi reported it, Windows desktop included. Reading includes another model gpt-oss:20b was still loaded from the cell before this one, so this reading is the whole card with two models on it, not Llama 3.2 3B alone. The 16,384-context row from the same capture was read after the larger model had been evicted. Throughput dataset, VRAM note. Resident, whole card 15,682 MiB Measured Decode 311.96 tok/s Measured Left on the card 621 MiB Computed: 16,303 − 15,682 Share of the card 96.2% Computed: 15,682 ÷ 16,303 Full measured row Captured UTC GPU NVIDIA GeForce RTX 5080 Driver 610.88 Runtime Ollama 0.32.1 Method llm-throughput-ollama-v1 Model llama3.2:3b digest a80c4f17acd5 Parameters 3.2B Quantization Q4 K M Context num ctx 4096 Prompt cell 1 The row pinned on the map Prompt tokens 471 Repetitions 3 Decode, median 311.96 tok/s Prefill, median 18196.57 tok/s Load, median 162.8 ms Whole-card memory after the cell 15682 MiB Prompt cell 2 Same configuration, longer prompt Prompt tokens 1657 Repetitions 3 Decode, median 298.21 tok/s Prefill, median 17681.08 tok/s Load, median 156.2 ms Whole-card memory after the cell 15682 MiB Resident in GiB 15.31 GiB Computed: 15,682 ÷ 1,024 Energy for this model, from its own power run A separate run of llama3.2:3b on UTC , stock power limit: 318.3 tok/s at 265.2 W board power, 0.833 J per token in the file. That is 833 J per 1,000 tokensComputed: 0.833 × 1,000. The power file records no context length, driver or runtime version, so read it as the model's energy at stock, not this row's. No measured configuration matches those filters. That combination was not on the bench, and I do not estimate here. For an estimate, use the LLM VRAM calculator. How to read it The map answers a narrow question: what did this model, at this context length, on this runtime version, take on a 16 GB card, and what speed came with it. Each record reads “measured at so many MiB resident on a date with a runtime version” because that is all a measurement can say. It is not a promise that the same model fits on your card today. The line along the bottom is the card, from zero to the total it reports to nvidia-smi. A pin’s position across is the memory in use on the whole card while that model was loaded, Windows desktop included, and its height is the decode rate. Select a pin and the line fills to it in the measured colour. The hatched remainder is headroom, which I compute by subtracting. A dashed connector joins the same model, runtime and capture at two context lengths. Flat means memory grew and speed held; a drop means the model stopped fitting. A ring is a run recorded as partly off the card. A diamond is a reading taken while another model was still loaded, so it overstates that model. Where the cliff shows up The cliff is not a memory number. In these files, a model that sits whole on the card decodes at the same speed whether it uses a third of the memory or nearly all of it. Speed falls when part of the model no longer fits and runs from system RAM instead. Each row is the same model, runtime version and capture at two context lengths, and both ends are measured rows. Memory change is the second reading minus the first. Decode change is second − first ÷ first. Model and runtimeContextResident MiB, measuredMemory change, computedDecode tok/s, measuredDecode change, computed llama3.2:3bOllama 0.32.15, defaults · 131,072: ran partly off the card4,096 → 131,0723,760 → 15,084+11,324351.22 → 102.68−70.8% qwen3-coder:30bOllama 0.32.1 · 16,384: ran partly off the card4,096 → 16,38415,101 → 15,003−9861.89 → 51.62−16.6% The table has two rows where raising the context cost speed, and both are runs that left the card. Llama 3.2 3B is the small model in this set, and a large enough context still pushed it over. Qwen3-Coder 30B never fit whole, so its memory reading barely moved and its speed did. Every other pair held its decode rate within one percent while the context quadrupled. What they paid was memory, and how much depended on the model: the two 14B Qwen models paid the most, and gpt-oss:20b, the largest of them by parameter count, paid the least. What the files do not hold is a dense model caught in the act of falling off. The throughput dataset says Devstral 24B collapsed when I tried it at an 8K context, but that attempt was never written to the CSV, so there is no pin for it and I have not drawn one. Energy per thousand tokens llama3.2:3bQ4 K M · named “Llama 3.2 3B” in the file · run logged UTC 833 J per 1,000 tokens Computed: 0.833 J per token × 1,000 Decode, median of 3 318.3 tok/s Board power, median while loaded 265.2 W Tokens per joule, as filed 1.2 Joules per token, as filed 0.833 More from the power file Watt-hours per 1,000 tokens 0.231 Wh Computed: 231.0 Wh per million ÷ 1,000 Decode, slowest and fastest run 315.5 to 318.5 tok/s Board power, peak 307.78 W Memory in use, median 3,590 MiB GPU temperature, median 49.0 °C GPU load, median 82.0% Telemetry samples while loaded 9 qwen2.5:7bQ4 K M · named “Qwen 2.5 7B” in the file · run logged UTC 1,886 J per 1,000 tokens Computed: 1.886 J per token × 1,000 Decode, median of 3 182.2 tok/s Board power, median while loaded 343.6 W Tokens per joule, as filed 0.53 Joules per token, as filed 1.886 More from the power file Watt-hours per 1,000 tokens 0.524 Wh Computed: 524.0 Wh per million ÷ 1,000 Decode, slowest and fastest run 182.1 to 182.3 tok/s Board power, peak 346.08 W Memory in use, median 5,665 MiB GPU temperature, median 52.0 °C GPU load, median 88.0% Telemetry samples while loaded 14 qwen2.5:14bQ4 K M · named “Qwen 2.5 14B” in the file · run logged UTC 3,154 J per 1,000 tokens Computed: 3.154 J per token × 1,000 Decode, median of 3 98.8 tok/s Board power, median while loaded 311.6 W Tokens per joule, as filed 0.317 Joules per token, as filed 3.154 More from the power file Watt-hours per 1,000 tokens 0.876 Wh Computed: 876.0 Wh per million ÷ 1,000 Decode, slowest and fastest run 98.6 to 98.8 tok/s Board power, peak 319.69 W Memory in use, median 10,171 MiB GPU temperature, median 50.0 °C GPU load, median 93.0% Telemetry samples while loaded 27 gpt-oss:20bMXFP4 · named “gpt-oss 20B” in the file · run logged UTC 1,504 J per 1,000 tokens Computed: 1.504 J per token × 1,000 Decode, median of 3 194.4 tok/s Board power, median while loaded 292.4 W Tokens per joule, as filed 0.665 Joules per token, as filed 1.504 More from the power file Watt-hours per 1,000 tokens 0.418 Wh Computed: 418.0 Wh per million ÷ 1,000 Decode, slowest and fastest run 194.3 to 194.5 tok/s Board power, peak 294.86 W Memory in use, median 13,381 MiB GPU temperature, median 51.0 °C GPU load, median 84.0% Telemetry samples while loaded 18 These four runs are a separate capture from the map: one short prompt, three iterations per model, board power sampled once a second and counted only while the GPU was at least half loaded. They were taken at the card’s stock 448 W power limit, which none of the runs reached. Board power is the graphics card alone, not the wall. The power file records no context length, driver or runtime version, which is why these runs are not pinned on the map. Its dataset page gives the runtime as Ollama 0.32.1. There is no undervolt or power-limit comparison here because I have not measured one for language models. My undervolt database holds one row for this card and it is a stock 3DMark baseline. Method and the idle-power finding are on the power efficiency dataset. Versions on the bench, and how stale they are Driver and runtime versions per capture, exactly as each file records them. Where a file is silent, the cell says so. CapturedFileGPU driverRuntime versionsRows used here llm-server-compare-2026-08-26-lmstudio.csvNot recorded in this fileLM Studio 0.4.21, Ollama 0.32.15Ollama 0.32.15: vendored llama.cpp 9d77fa172LM Studio 0.4.21: llama.cpp-win-x86 64-nvidia-cuda12-avx2@2.29.112 configurations Those are the versions that were installed on the capture dates. Drivers and runtimes have both moved since, and I have not re-run this matrix on newer ones. Read every pin as true for its date and versions, and expect a current install to differ. The second table is why I say that plainly. Two captures six days apart put the same three models on two Ollama versions, and all three decoded faster on the later one. The runtime comparison dataset notes that the earlier capture had less control over background load, so not all of that gap is the version. Either way, a few percent between runtimes does not survive an update. What the data covers Model tags 8 llama3.2:3b 3.2B · Q4 K M 9 configurations qwen2.5-coder:7b 7.6B · Q4 K M 2 configurations qwen2.5:7b 7.6B · Q4 K M 2 configurations qwen2.5-coder:14b 14.8B · Q4 K M 2 configurations qwen2.5:14b 14.8B · Q4 K M 8 configurations gpt-oss:20b 20.9B · MXFP4 10 configurations devstral:24b 23.6B · Q4 K M 1 configuration qwen3-coder:30b 30.5B · Q4 K M 2 configurations Quantizations 2 MXFP410 Q4 K M26 No model was measured at more than one quantization. One quantization per model. I cannot show the same model at three quantizations stepping down the card, because each model was measured at one. No dense model past the cliff. The runs that left the card are a 3B model at an extreme context, a mixture-of-experts model that never fit, and one LM Studio default that offloaded part of a model that otherwise fits. A dense model that fits at one context and spills at the next is the row this map most needs. A gap between 16,384 and 131,072 context. One run sits at the far end. Nothing was measured at 32K or 64K. Energy at stock only. No undervolt, no reduced power limit, and no energy figure for a run that spilled. One card. Another 16 GB card, or this one with a different desktop load, will read differently. Method, exactly I did not run anything new for this page. It reads files I had already published and puts them on one axis. The throughput rows come from my capture script. For each model, context and prompt size it calls Ollama’s /api/generate three times at temperature 0, seed 42 and a 256-token output, with a unique prefix on every prompt so each run pays a full prefill. Decode and prefill rates are Ollama’s own eval count over eval duration and prompt eval count over prompt eval duration, and the CSV keeps the median of the three. After each cell the script asks nvidia-smi for memory.used and stores it as vram resident mib. That one reading is the pin’s position. That reading is the whole card. It includes the desktop, and it includes any model Ollama had not yet unloaded, which is how one Llama 3.2 3B row came to read 15,682 MiB. The runtime comparison captures used a different script that took the median memory reading across runs and, in the later capture, also read the card before each load. Each throughput configuration was measured with two prompt sizes. The pin is the first of the two, the shorter prompt, and both are in the record. The model names differ between files, so I join them on the name with punctuation and a trailing quantization tag removed. Only the throughput CSV has a quantization column; where another file’s row shows a quantization, the record says it was taken from the same model there. The card total, 16,303 MiB, is the nvidia-smi total written into the raw capture file from 2026-08-26. Headroom, share of the card, GiB, the memory a load added, the changes between two rows and the per-thousand-token energy are computed when the site is built, from the measured values beside them, and each one is shown with its formula. Your browser does no arithmetic on them. Download the files rtx-5080-llm-throughput.csv34 rows. Decode, prefill, load time and whole-card memory for each model, context and prompt size on Ollama. Raw capture JSON and the capture script are listed in its manifest. llm-server-compare-2026-08.csv6 rows. Ollama beside llama.cpp on three models, with its settings and capture date in the footer notes. llm-server-compare-2026-08-26-lmstudio.csv12 rows. Ollama beside LM Studio on three models, at default and normalized settings, with the memory reading before each load. All of it is CC BY 4.0. Attribute as “TechFuelHQ RTX 5080 16 GB VRAM cliff map” with a link to this page. To cite one configuration, open its row in the list and copy its link; the address ends in that row’s own id.