How many tokens will an old 3090 produce?

An RTX 3090 can generate about 40 tokens per second when running Qwen3.8-27B, according to crowd-sourced benchmarks from llamabench.ai and user reports. The 24 GB VRAM of the 3090 is sufficient for quantized models of this size, making it a capable card for local LLM inference despite its age.

Oh. It depends on the backend you’re using, but that seems to be about the expected speed : 482 tokens in about 12 seconds works out to roughly 40 tokens/s . If those 12 seconds are the actual generation/decode time, that number does not look suspiciously high for Qwen3.8-27B on a 3090. As a sanity check rather than an exact apples-to-apples benchmark, the current crowd-sourced llamabench.ai results for Qwen3.8-27B on an RTX 3090 https://llamabench.ai/browse/rtx3090/qwen3-8-27b include non-speculative runs at 38.2, 40.0, 41.1, and 45.6 tok/s . Those runs use llama.cpp and somewhat different quant/cache settings, so I would not treat them as a reproduction of your Ollama run, but your ~40.2 tok/s sits right in that range. The cheapest way to verify what you are seeing is probably: ollama run <your-model --verbose ollama ps For the first command, the useful lines are approximately: prompt eval count: ... prompt eval duration: ... prompt eval rate: ... eval count: ... eval duration: ... eval rate: ... tokens/s Ollama’s API usage documentation https://docs.ollama.com/api/usage makes the distinction explicit: prompt eval is processing the input prompt. eval count is the number of generated output tokens. eval duration is the time spent generating those output tokens. total duration also includes other work such as model loading and prompt processing.So if Ollama itself reports an eval rate around 40 tok/s, there is not much ambiguity left: you really are getting about 40 generated tokens/s. ollama ps gives the other high-value check. According to the Ollama FAQ https://docs.ollama.com/faq , 100% GPU means the model is loaded entirely on the GPU, while a CPU/GPU percentage indicates that it is split between system RAM and VRAM. A useful decision tree is therefore: Is ~12 s the generation time / Ollama eval duration? | +-- Yes | | | +-- ~482 / ~12 s = ~40 tok/s | | | +-- ollama ps says 100% GPU | | | | | +-- This looks quite normal for this model/GPU class. | | | +-- CPU/GPU split | | | +-- Then context size/offloading becomes important | before comparing the number with other benchmarks. | +-- No / not sure | +-- Use --verbose or the API metrics once and compare eval rate, rather than dividing visible tokens by wall-clock time. In other words, I would not worry that the 3090 has somehow produced an impossible result. Its age is a bit misleading here: 24 GB of VRAM is still a very useful configuration for local inference , because a quantized model of this size can fit without having to spill a large part of it into CPU memory. So, for the result you posted, my default interpretation would simply be: yes, ~40 tok/s is believable. If ollama run ... --verbose reports roughly that eval rate and ollama ps says 100% GPU , I would consider the basic mystery solved. The fact that a 3090 is an older card does not prevent it from still being a very capable local-LLM card when a ~27B quantized model fits comfortably inside its 24 GB VRAM.