{"slug": "how-many-tokens-will-an-old-3090-produce", "title": "How many tokens will an old 3090 produce?", "summary": "An RTX 3090 can generate about 40 tokens per second when running Qwen3.8-27B, according to crowd-sourced benchmarks from llamabench.ai and user reports. The 24 GB VRAM of the 3090 is sufficient for quantized models of this size, making it a capable card for local LLM inference despite its age.", "body_md": "Oh. It depends on the backend you’re using, but that **seems to be about the expected speed**:\n\n482 tokens in about 12 seconds works out to roughly **40 tokens/s**. If those 12 seconds are the actual generation/decode time, that number does not look suspiciously high for Qwen3.8-27B on a 3090.\n\nAs a sanity check rather than an exact apples-to-apples benchmark, the current crowd-sourced [llamabench.ai results for Qwen3.8-27B on an RTX 3090](https://llamabench.ai/browse/rtx3090/qwen3-8-27b) include non-speculative runs at **38.2, 40.0, 41.1, and 45.6 tok/s**. Those runs use llama.cpp and somewhat different quant/cache settings, so I would not treat them as a reproduction of your Ollama run, but your ~40.2 tok/s sits right in that range.\n\nThe cheapest way to verify what you are seeing is probably:\n\n```\nollama run <your-model> --verbose\nollama ps\n```\n\nFor the first command, the useful lines are approximately:\n\n```\nprompt eval count:     ...\nprompt eval duration:  ...\nprompt eval rate:      ...\n\neval count:            ...\neval duration:         ...\neval rate:             ... tokens/s\n```\n\nOllama’s [API usage documentation](https://docs.ollama.com/api/usage) makes the distinction explicit:\n\n`prompt_eval_*`\n\nis processing the input prompt.`eval_count`\n\nis the number of generated output tokens.`eval_duration`\n\nis the time spent generating those output tokens.`total_duration`\n\nalso includes other work such as model loading and prompt processing.So if Ollama itself reports an `eval rate`\n\naround 40 tok/s, there is not much ambiguity left: you really are getting about 40 generated tokens/s.\n\n`ollama ps`\n\ngives the other high-value check. According to the [Ollama FAQ](https://docs.ollama.com/faq), `100% GPU`\n\nmeans the model is loaded entirely on the GPU, while a CPU/GPU percentage indicates that it is split between system RAM and VRAM.\n\nA useful decision tree is therefore:\n\n```\nIs ~12 s the generation time / Ollama eval_duration?\n|\n+-- Yes\n|   |\n|   +-- ~482 / ~12 s = ~40 tok/s\n|       |\n|       +-- ollama ps says 100% GPU\n|       |   |\n|       |   +-- This looks quite normal for this model/GPU class.\n|       |\n|       +-- CPU/GPU split\n|           |\n|           +-- Then context size/offloading becomes important\n|               before comparing the number with other benchmarks.\n|\n+-- No / not sure\n    |\n    +-- Use --verbose (or the API metrics) once and compare eval_rate,\n        rather than dividing visible tokens by wall-clock time.\n```\n\nIn other words, I would not worry that the 3090 has somehow produced an impossible result. Its age is a bit misleading here: **24 GB of VRAM is still a very useful configuration for local inference**, because a quantized model of this size can fit without having to spill a large part of it into CPU memory.\n\nSo, for the result you posted, my default interpretation would simply be:\n\n**yes, ~40 tok/s is believable.**\n\nIf `ollama run ... --verbose`\n\nreports roughly that `eval rate`\n\nand `ollama ps`\n\nsays `100% GPU`\n\n, I would consider the basic mystery solved. The fact that a 3090 is an older card does not prevent it from still being a very capable local-LLM card when a ~27B quantized model fits comfortably inside its 24 GB VRAM.", "url": "https://wpnews.pro/news/how-many-tokens-will-an-old-3090-produce", "canonical_source": "https://discuss.huggingface.co/t/how-many-tokens-will-an-old-3090-produce/179251#post_2", "published_at": "2026-08-26 01:00:33+00:00", "updated_at": "2026-08-26 01:13:36.973073+00:00", "lang": "en", "topics": ["artificial-intelligence", "large-language-models", "ai-infrastructure"], "entities": ["NVIDIA RTX 3090", "Qwen3.8-27B", "llamabench.ai", "Ollama", "llama.cpp"], "alternates": {"html": "https://wpnews.pro/news/how-many-tokens-will-an-old-3090-produce", "markdown": "https://wpnews.pro/news/how-many-tokens-will-an-old-3090-produce.md", "text": "https://wpnews.pro/news/how-many-tokens-will-an-old-3090-produce.txt", "jsonld": "https://wpnews.pro/news/how-many-tokens-will-an-old-3090-produce.jsonld"}}