{"slug": "i-cut-my-rtx-3090-ti-from-450w-to-300w-local-llm-inference-lost-only-3", "title": "I Cut My RTX 3090 Ti From 450W to 300W. Local LLM Inference Lost Only 3%.", "summary": "Capping an NVIDIA RTX 3090 Ti from its default 450 W power limit to 300 W via `nvidia-smi -pl 300` reduced local LLM decode throughput by only 2.9%, from 52.6 tok/s to 51.1 tok/s, on Qwen 3.8 27B Q4 GGUF running through llama.cpp and the author's ThreadShelf application. Observed GPU draw fell from roughly 400 W to roughly 300 W, cutting decode energy from about 7.60 J/token to 5.87 J/token — roughly 23% less energy per token, or a 30% increase in tokens per joule — while GPU temperature dropped from 64°C to 45–51°C and fan speed from about 88% to 63–83%. The author reported that MSI Afterburner settings were sometimes effectively ignored on the test machine, making NVIDIA's documented `-pl`/`--power-limit` interface the reliable control for the test.", "body_md": "# I Cut My RTX 3090 Ti From 450W to 300W. Local LLM Inference Lost Only 3%.\n\nCapping an RTX 3090 Ti at 300 W instead of 450 W cost 2.9% of local LLM decode throughput (Qwen 3.8 27B, llama.cpp) and cut energy per token by about 23%.\n\nThe RTX 3090 Ti is a strange but excellent local-LLM card.\n\nIt has 24 GB of VRAM and enough memory bandwidth to run surprisingly large quantized models at useful speeds. It also has a default 450 W power limit.\n\nI wanted to know how much of that power my actual LLM workload needed.\n\nThe answer, on my machine, was: not much.\n\nI reduced the GPU power limit from 450 W to 300 W. The configured power budget fell by one third.\n\nDecode throughput fell by **2.9%**.\n\n## My test\n\nThis was not a synthetic CUDA benchmark. I measured the setup I actually use:\n\n- NVIDIA RTX 3090 Ti, 24 GB VRAM\n- Qwen 3.8 27B, Q4 GGUF\n- `llama.cpp` /`llama-server`\n- inference launched through **ThreadShelf** , my local-first application for archiving, searching and continuing AI conversations\n- local generation stays on-device: ThreadShelf manages a loopback-only `llama-server` and streams the model output back into the application\n\nThreadShelf’s local generation path is built directly around `llama.cpp`, so this was a real interactive inference workload rather than a standalone GPU stress test.\n\nI tested two power limits:\n\n| GPU power limit | Observed generation speed | Observed GPU draw | GPU temperature | Fan speed | \n|---|---|---|---|---|\n| 450 W | 52.6 tok/s | ~400 W | 64°C | ~88% | \n| 300 W | 51.1 tok/s | ~300 W | 45–51°C | ~63–83% | \n\nThe interesting part is not that the 300 W configuration was more efficient. That was expected.\n\nIt is how little performance I lost.\n\n```\n52.6 tok/s → 51.1 tok/s\n\nperformance loss ≈ 2.9%\n```\n\nMeanwhile, observed GPU draw dropped from roughly 400 W to roughly 300 W.\n\nUsing those observed power figures:\n\n```\n400 W / 52.6 tok/s ≈ 7.60 J/token\n300 W / 51.1 tok/s ≈ 5.87 J/token\n```\n\nSo decode energy per generated token fell by roughly **23%**.\n\nEquivalently:\n\n```\n52.6 / 400 ≈ 0.132 tokens/J\n51.1 / 300 ≈ 0.170 tokens/J\n```\n\nThat is roughly a **30% increase in tokens per joule**.\n\nAnd there was another difference that mattered more to me in daily use than the electricity bill: the card became much less obnoxious.\n\nAt the high-power setting I saw around 64°C and fan speed near 88%.\n\nAt 300 W it generally stayed around 45–51°C, with lower fan speeds.\n\nFor a GPU sitting next to me and generating for minutes at a time, that is a meaningful difference.\n\n## `nvidia-smi` worked. MSI Afterburner did not.\n\nI initially tried doing this through MSI Afterburner.\n\nOn my machine, it was not reliable. Settings that appeared to be applied were sometimes effectively ignored, which made it a bad basis for a benchmark. I did not want to wonder whether a voltage curve, profile, driver reset or another Afterburner setting was actually active during a run.\n\nWhat worked reliably was much simpler:\n\n```\nnvidia-smi -pl 300\n```\n\nAnd to restore the original limit:\n\n```\nnvidia-smi -pl 450\n```\n\nI could then verify the requested and enforced limits directly with `nvidia-smi`.\n\nThis is not an unofficial trick. NVIDIA documents `-pl` / `--power-limit` as the interface for setting the GPU’s maximum power limit. NVIDIA also exposes the current, requested, default, minimum and maximum power limits through the same tooling. When power management is active, the GPU stays inside that envelope by changing its performance state.\n\nThat distinction matters.\n\nFor my use case, a power limit turned out to be a much better control knob than manually editing a voltage/frequency curve:\n\n```\nI specify:      maximum 300 W\nGPU decides:    clocks / voltage / performance state\nI measure:      throughput, power and temperature\n```\n\nIt is simple, reproducible and difficult to misread.\n\nI am not claiming that MSI Afterburner generally does not work. It clearly does for many cards and configurations. It simply did not give me sufficiently reliable control on this machine, while the driver-level `nvidia-smi` limit did.\n\n## Power limiting is not the same thing as undervolting\n\nThese terms are often mixed together.\n\nThey are related, but they are not identical.\n\nA **power limit** says:\n\nThe board may consume at most N watts.\n\nA traditional **undervolt** says something closer to:\n\nAt this GPU frequency, use a lower voltage than the default voltage/frequency curve requests.\n\nWith NVIDIA GPU Boost, the GPU continuously selects operating points according to available thermal and electrical headroom. NVIDIA describes GPU Boost as dynamically changing clocks and voltages until the GPU encounters its power target or another constraint.\n\nSo when I impose a 300 W power cap, I am not explicitly saying:\n\n```\nrun at 0.800 V and 1800 MHz\n```\n\nI am giving GPU Boost a smaller envelope.\n\nThe card then manages frequency and voltage inside it.\n\nA manually tuned undervolt can potentially do better because it can tell the GPU:\n\n```\nyou were using more voltage than this particular chip\nactually needs to remain stable at this frequency\n```\n\nThat exploits manufacturing margin.\n\nBut it also adds another variable: stability.\n\nA power cap leaves NVIDIA’s normal V/F management in charge. That is one reason I prefer it for an everyday workstation.\n\n## Why voltage matters so much\n\nA simplified model for CMOS dynamic power is:\n\n```\nPdynamic ∝ C × V² × f\n```\n\nwhere:\n\n- `C` is effective switched capacitance,\n- `V` is voltage,\n- `f` is operating frequency.\n\nThe important term is `V²`.\n\nIncreasing frequency toward the top of a GPU’s operating range generally also requires increasing voltage. That means the final few hundred MHz can become disproportionately expensive in watts.\n\nThe general GPU DVFS literature has documented this for years: the relationship between clock speed, voltage, performance and energy is nonlinear, and the optimal operating point depends heavily on the workload.\n\nThis is why a GPU can have a performance curve roughly shaped like this:\n\n```\nperformance\n│                         ________\n│                   _____/\n│              ____/\n│          ___/\n│       __/\n│_____/\n└────────────────────────────── power\n                  ↑\n         expensive last few %\n```\n\nThe manufacturer has good reasons to ship near the right side of that curve.\n\nA flagship GPU is sold partly on winning benchmarks. If another 80 or 100 watts buys several additional percentage points of performance, that can be commercially rational even when it is terrible from a performance-per-watt perspective.\n\nThe RTX 3090 Ti is an unusually good example because NVIDIA shipped the reference design with a 450 W power target.\n\n## Why did my LLM lose only 3%?\n\nThis is where LLM inference becomes interesting.\n\nA game and autoregressive LLM decoding do not stress a GPU in the same way.\n\nFor single-stream LLM generation, the decode phase frequently spends much of its time moving data rather than performing arithmetic.\n\nEach generated token requires running through the model again. For a dense transformer, that means repeatedly bringing a huge amount of model-weight data through GPU memory, together with KV-cache traffic and intermediate data.\n\nRecent reviews of LLM inference characterize autoregressive decode as strongly memory-bound at low batch sizes: arithmetic intensity is low enough that memory traffic, rather than peak FLOPS, becomes the dominant constraint.\n\nThat distinction explains my result.\n\nThe 3090 Ti has a huge amount of compute capability, but my 27B Q4 model does not necessarily need all of it to generate the next token.\n\nOnce the GPU cores can process data approximately as quickly as the memory subsystem can supply it, pushing the cores harder has diminishing returns.\n\nVery roughly:\n\n```\n                 450 W       300 W\n\nSM compute       faster      slightly slower\nvoltage          higher      lower operating points\nheat             much more   much less\n\nVRAM bandwidth   similar     similar enough\n\nLLM decode       52.6        51.1 tok/s\n```\n\nThat final line is what matters.\n\nRemoving around 100 W of actual GPU consumption did not remove 25% of the bottleneck, because those extra watts were not buying 25% more effective memory throughput.\n\nThey were largely pushing the GPU further up its voltage/frequency curve.\n\n## Games behave differently\n\nThere is a useful sanity check here: other people have already power-limited the RTX 3090 Ti for gaming, and the performance penalty is larger than what I measured for LLM decode.\n\nA 2022 test reported by Tom’s Hardware compared an RTX 3090 Ti running near its normal power envelope with a tuned configuration around 300 W. Across ten games at 4K, the full-power card averaged about 107 FPS while the 300 W configuration averaged about 96 FPS.\n\nThat is roughly a **10% performance loss**, not my 3%.\n\nComputerBase later measured the 3090 Ti at 300 W versus 450 W and found roughly a **15% gaming-performance difference** in its power-scaling comparison.\n\nHardwareluxx performed another explicit power-scaling experiment from 450 W down to 200 W. At 350 W, its RTX 3090 Ti retained around 90% of its 450 W performance in Control; it placed the efficiency sweet spot around 375 W for that particular gaming workload.\n\nSo the rough comparison is:\n\n| Workload | 450 → 300 W performance cost | \n|---|---|\n| My Qwen 27B decode | ~2.9% | \n| Published 3090 Ti gaming tests | roughly 10–15% | \n\nThey are not directly comparable benchmarks, but that is precisely the point.\n\nThere is no universal “best RTX 3090 Ti power limit”.\n\nThe optimum depends on what the GPU is doing.\n\nFor my local LLM decode workload, 300 W sits in an extremely attractive region of the curve.\n\nFor games, the last 150 W buys noticeably more performance.\n\n## What research says about undervolting\n\nThere is also experimental research behind the broader idea.\n\nMendes, Tomás and Roma studied decoupled GPU voltage/frequency scaling for deep-learning workloads. Their experiments were on AMD Vega 10 and Radeon 5700 XT hardware rather than Ampere, so the numerical results should not be transferred to a 3090 Ti.\n\nBut the mechanism is relevant.\n\nThey found that GPUs could operate safely at non-default voltage/frequency combinations and reported reductions of up to **38% in energy consumption** and **41% in energy-delay product** in their DNN experiments without degrading network accuracy. They also found that different GPU components had different undervolting limits: the ALUs and DRAM/cache controller were among the more voltage-sensitive components.\n\nThat last point is important.\n\nUndervolting is not free energy.\n\nAt some point you leave the stable region.\n\nThe practical problem is finding:\n\n```\nlowest voltage\n    subject to\nrequired frequency\n    subject to\nzero computation errors\n    across\nevery workload you actually care about\n```\n\nA stable gaming undervolt is not automatically a stable CUDA inference undervolt.\n\nAnd a configuration that survives five minutes is not necessarily one I want running a multi-hour evaluation.\n\nThat is another reason I like power limiting.\n\nI give up some theoretical optimization potential, but I keep the manufacturer’s voltage/frequency management.\n\n## LLM-specific research is moving in the same direction\n\nPower management is now being studied directly for LLM serving rather than borrowed from older HPC and CNN work.\n\nA 2026 COMPSAC paper evaluated adaptive GPU power capping on eight NVIDIA H200 GPUs serving Qwen3-32B. Instead of choosing one permanent power cap, the system adjusted the cap dynamically according to latency/SLA feedback.\n\nThe authors reported **3.84% lower total energy consumption than a 700 W baseline while maintaining empirical SLA non-inferiority**.\n\nThat is a very different scale from one 3090 Ti under my desk, but the principle is identical:\n\nmaximum GPU power is not necessarily the power level that maximizes useful work per joule.\n\nAnother useful piece of work is TokenPowerBench, published at AAAI 2026.\n\nInstead of treating “LLM inference power” as one number, it measures energy separately for:\n\n```\nprefill\n+\ndecode\n```\n\nand reports metrics such as energy per token and energy per request. The benchmark explicitly studies variables including model choice, context length, batch size, parallelism and quantization.\n\nThat separation is important because my result should specifically be understood as a **decode-throughput result**.\n\n## Prefill is a different workload\n\nMy headline measurement is:\n\n```\n52.6 tok/s vs 51.1 tok/s\n```\n\nThat measures generated-token throughput.\n\nIt does not imply that every phase of an LLM request becomes only 3% slower at 300 W.\n\nPrefill processes many prompt tokens in parallel and has much higher arithmetic intensity than single-token decode. It can therefore make much better use of the GPU’s compute units.\n\nLong contexts also increase KV-cache traffic and attention work.\n\nBatching changes the balance again.\n\nSo the shape of the power-performance curve depends on:\n\n- model architecture,\n- quantization,\n- context length,\n- prompt size,\n- batch size / concurrent slots,\n- prefill versus decode,\n- inference engine,\n- CUDA kernels.\n\nThis is consistent with decades of GPU DVFS research: optimal voltage/frequency settings are workload-dependent rather than a fixed property of a GPU model.\n\nIt also explains something from my initial experiment that I would **not** use as a headline result.\n\nI observed complete conversation times of about 56 seconds in one high-power run and 73 seconds in one 300 W run.\n\nThat gap is far too large to be explained by a 2.9% difference in decode throughput.\n\nThose requests were not controlled well enough for prompt-processing time, generated-token count, context state and server scheduling.\n\nSo I consider:\n\n```\n52.6 vs 51.1 tok/s\n```\n\na useful measurement.\n\nI do **not** consider:\n\n```\n56 vs 73 seconds\n```\n\nevidence that the power cap increased end-to-end latency by 30%.\n\nA proper benchmark should measure prefill and decode independently.\n\n## Power cap or manual undervolt?\n\nFor my machine, the answer is currently power cap.\n\n|  | Power limit | Manual undervolt | \n|---|---|---|\n| What I control | Maximum watts | Voltage/frequency curve | \n| GPU Boost remains in control | Yes | Partially | \n| Setup | One command | Curve tuning | \n| Reproducibility | Excellent on my machine | Worse in my experience | \n| Stability testing | Minimal | Necessary | \n| Efficiency potential | Very good | Potentially better | \n| `nvidia-smi` support | Native | Not traditional Ampere V/F tuning | \n| My choice for LLM work | **Yes** | Not currently | \n\nThe crucial distinction is that manual undervolting tries to optimize the silicon itself.\n\nPower limiting optimizes the operating envelope.\n\nFor a hobby benchmark, finding the lowest stable voltage can be interesting.\n\nFor a workstation where I want to start ThreadShelf, load a model and know exactly what configuration I am running, this:\n\n```\nnvidia-smi -pl 300\n```\n\nis hard to beat.\n\n## The RTX 3090 Ti was pushed far past its efficiency sweet spot\n\nThe broader result is not really about saving 100 W.\n\nIt is about diminishing returns.\n\nThe 3090 Ti was NVIDIA’s Ampere flagship. At stock, the card can consume around 450 W. Reviews at launch already showed how aggressively NVIDIA had pushed it: substantial extra power was being spent to obtain the final slice of performance.\n\nMy local-LLM workload makes this unusually visible.\n\n```\nObserved GPU draw     ~400 W       ~300 W\nDecode throughput     52.6 tok/s   51.1 tok/s\nTemperature           64°C         45–51°C\n\nPower                 -25%\nDecode throughput     -2.9%\n```\n\nFor me, that is an easy trade.\n\nIn games the trade is less extreme: external testing suggests I should expect a larger FPS penalty at 300 W.\n\nBut for a large quantized local model doing mostly single-user autoregressive decoding, the RTX 3090 Ti appears to have a very large power-hungry tail that provides little additional throughput.\n\nThe reason is not mysterious.\n\nIt is the combination of:\n\n1. an aggressively clocked flagship GPU,\n2. the nonlinear voltage/frequency/power curve,\n3. GPU Boost spending additional power to chase higher clocks,\n4. and an LLM decode workload constrained heavily by memory movement rather than raw compute.\n\nThe last 100 W simply have much less useful work left to buy.\n\n## What I want to test next\n\nTwo points are enough to show that 450 W is wasteful for my workload.\n\nThey are not enough to locate the optimum.\n\nThe proper experiment is a power sweep:\n\n```\n200 W\n225 W\n250 W\n275 W\n300 W\n325 W\n350 W\n375 W\n400 W\n425 W\n450 W\n```\n\nFor each setting I want to record separately:\n\n```\nprompt processing tok/s\ndecode tok/s\naverage GPU W\npeak GPU W\nJ / output token\nJ / request\ntemperature\nfan speed\nGPU clock\nmemory clock\n```\n\nwith a fixed prompt, fixed context, fixed output length and repeated runs.\n\nI expect a knee in the curve: a region where reducing power initially removes mostly wasted voltage/frequency headroom, followed by a point where memory throughput or required compute starts falling enough that tokens per second collapse.\n\nMy first two measurements strongly suggest that **300 W is already much closer to that efficient region than 450 W for my Qwen 27B setup**.\n\nBut that is the next experiment, not a result I have measured yet.\n\nFor now, the result is simpler:\n\n**I cut the RTX 3090 Ti’s configured power limit by one third and lost only 2.9% of local-LLM decode throughput.**\n\nFor a GPU infamous for consuming 450 watts, that is a much more useful characteristic than its stock TDP.", "url": "https://wpnews.pro/news/i-cut-my-rtx-3090-ti-from-450w-to-300w-local-llm-inference-lost-only-3", "canonical_source": "https://chrystianschutz.com/blog/rtx-3090-ti-power-limit-llm/", "published_at": "2026-09-25 00:00:00+00:00", "updated_at": "2026-10-17 20:35:50.969941+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-chips", "ai-tools"], "entities": ["NVIDIA", "RTX 3090 Ti", "Qwen 3.8 27B", "llama.cpp", "ThreadShelf", "MSI Afterburner", "nvidia-smi"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/i-cut-my-rtx-3090-ti-from-450w-to-300w-local-llm-inference-lost-only-3", "markdown": "https://wpnews.pro/news/i-cut-my-rtx-3090-ti-from-450w-to-300w-local-llm-inference-lost-only-3.md", "text": "https://wpnews.pro/news/i-cut-my-rtx-3090-ti-from-450w-to-300w-local-llm-inference-lost-only-3.txt", "jsonld": "https://wpnews.pro/news/i-cut-my-rtx-3090-ti-from-450w-to-300w-local-llm-inference-lost-only-3.jsonld"}}