I Cut My RTX 3090 Ti From 450W to 300W. Local LLM Inference Lost Only 3%.

Capping an NVIDIA RTX 3090 Ti from its default 450 W power limit to 300 W via `nvidia-smi -pl 300` reduced local LLM decode throughput by only 2.9%, from 52.6 tok/s to 51.1 tok/s, on Qwen 3.8 27B Q4 GGUF running through llama.cpp and the author's ThreadShelf application. Observed GPU draw fell from roughly 400 W to roughly 300 W, cutting decode energy from about 7.60 J/token to 5.87 J/token — roughly 23% less energy per token, or a 30% increase in tokens per joule — while GPU temperature dropped from 64°C to 45–51°C and fan speed from about 88% to 63–83%. The author reported that MSI Afterburner settings were sometimes effectively ignored on the test machine, making NVIDIA's documented `-pl`/`--power-limit` interface the reliable control for the test.

I Cut My RTX 3090 Ti From 450W to 300W. Local LLM Inference Lost Only 3%. Capping an RTX 3090 Ti at 300 W instead of 450 W cost 2.9% of local LLM decode throughput Qwen 3.8 27B, llama.cpp and cut energy per token by about 23%. The RTX 3090 Ti is a strange but excellent local-LLM card. It has 24 GB of VRAM and enough memory bandwidth to run surprisingly large quantized models at useful speeds. It also has a default 450 W power limit. I wanted to know how much of that power my actual LLM workload needed. The answer, on my machine, was: not much. I reduced the GPU power limit from 450 W to 300 W. The configured power budget fell by one third. Decode throughput fell by 2.9% . My test This was not a synthetic CUDA benchmark. I measured the setup I actually use: - NVIDIA RTX 3090 Ti, 24 GB VRAM - Qwen 3.8 27B, Q4 GGUF - llama.cpp / llama-server - inference launched through ThreadShelf , my local-first application for archiving, searching and continuing AI conversations - local generation stays on-device: ThreadShelf manages a loopback-only llama-server and streams the model output back into the application ThreadShelf’s local generation path is built directly around llama.cpp , so this was a real interactive inference workload rather than a standalone GPU stress test. I tested two power limits: | GPU power limit | Observed generation speed | Observed GPU draw | GPU temperature | Fan speed | |---|---|---|---|---| | 450 W | 52.6 tok/s | ~400 W | 64°C | ~88% | | 300 W | 51.1 tok/s | ~300 W | 45–51°C | ~63–83% | The interesting part is not that the 300 W configuration was more efficient. That was expected. It is how little performance I lost. 52.6 tok/s → 51.1 tok/s performance loss ≈ 2.9% Meanwhile, observed GPU draw dropped from roughly 400 W to roughly 300 W. Using those observed power figures: 400 W / 52.6 tok/s ≈ 7.60 J/token 300 W / 51.1 tok/s ≈ 5.87 J/token So decode energy per generated token fell by roughly 23% . Equivalently: 52.6 / 400 ≈ 0.132 tokens/J 51.1 / 300 ≈ 0.170 tokens/J That is roughly a 30% increase in tokens per joule . And there was another difference that mattered more to me in daily use than the electricity bill: the card became much less obnoxious. At the high-power setting I saw around 64°C and fan speed near 88%. At 300 W it generally stayed around 45–51°C, with lower fan speeds. For a GPU sitting next to me and generating for minutes at a time, that is a meaningful difference. nvidia-smi worked. MSI Afterburner did not. I initially tried doing this through MSI Afterburner. On my machine, it was not reliable. Settings that appeared to be applied were sometimes effectively ignored, which made it a bad basis for a benchmark. I did not want to wonder whether a voltage curve, profile, driver reset or another Afterburner setting was actually active during a run. What worked reliably was much simpler: nvidia-smi -pl 300 And to restore the original limit: nvidia-smi -pl 450 I could then verify the requested and enforced limits directly with nvidia-smi . This is not an unofficial trick. NVIDIA documents -pl / --power-limit as the interface for setting the GPU’s maximum power limit. NVIDIA also exposes the current, requested, default, minimum and maximum power limits through the same tooling. When power management is active, the GPU stays inside that envelope by changing its performance state. That distinction matters. For my use case, a power limit turned out to be a much better control knob than manually editing a voltage/frequency curve: I specify: maximum 300 W GPU decides: clocks / voltage / performance state I measure: throughput, power and temperature It is simple, reproducible and difficult to misread. I am not claiming that MSI Afterburner generally does not work. It clearly does for many cards and configurations. It simply did not give me sufficiently reliable control on this machine, while the driver-level nvidia-smi limit did. Power limiting is not the same thing as undervolting These terms are often mixed together. They are related, but they are not identical. A power limit says: The board may consume at most N watts. A traditional undervolt says something closer to: At this GPU frequency, use a lower voltage than the default voltage/frequency curve requests. With NVIDIA GPU Boost, the GPU continuously selects operating points according to available thermal and electrical headroom. NVIDIA describes GPU Boost as dynamically changing clocks and voltages until the GPU encounters its power target or another constraint. So when I impose a 300 W power cap, I am not explicitly saying: run at 0.800 V and 1800 MHz I am giving GPU Boost a smaller envelope. The card then manages frequency and voltage inside it. A manually tuned undervolt can potentially do better because it can tell the GPU: you were using more voltage than this particular chip actually needs to remain stable at this frequency That exploits manufacturing margin. But it also adds another variable: stability. A power cap leaves NVIDIA’s normal V/F management in charge. That is one reason I prefer it for an everyday workstation. Why voltage matters so much A simplified model for CMOS dynamic power is: Pdynamic ∝ C × V² × f where: - C is effective switched capacitance, - V is voltage, - f is operating frequency. The important term is V² . Increasing frequency toward the top of a GPU’s operating range generally also requires increasing voltage. That means the final few hundred MHz can become disproportionately expensive in watts. The general GPU DVFS literature has documented this for years: the relationship between clock speed, voltage, performance and energy is nonlinear, and the optimal operating point depends heavily on the workload. This is why a GPU can have a performance curve roughly shaped like this: performance │ │ / │ / │ / │ / │ / └────────────────────────────── power ↑ expensive last few % The manufacturer has good reasons to ship near the right side of that curve. A flagship GPU is sold partly on winning benchmarks. If another 80 or 100 watts buys several additional percentage points of performance, that can be commercially rational even when it is terrible from a performance-per-watt perspective. The RTX 3090 Ti is an unusually good example because NVIDIA shipped the reference design with a 450 W power target. Why did my LLM lose only 3%? This is where LLM inference becomes interesting. A game and autoregressive LLM decoding do not stress a GPU in the same way. For single-stream LLM generation, the decode phase frequently spends much of its time moving data rather than performing arithmetic. Each generated token requires running through the model again. For a dense transformer, that means repeatedly bringing a huge amount of model-weight data through GPU memory, together with KV-cache traffic and intermediate data. Recent reviews of LLM inference characterize autoregressive decode as strongly memory-bound at low batch sizes: arithmetic intensity is low enough that memory traffic, rather than peak FLOPS, becomes the dominant constraint. That distinction explains my result. The 3090 Ti has a huge amount of compute capability, but my 27B Q4 model does not necessarily need all of it to generate the next token. Once the GPU cores can process data approximately as quickly as the memory subsystem can supply it, pushing the cores harder has diminishing returns. Very roughly: 450 W 300 W SM compute faster slightly slower voltage higher lower operating points heat much more much less VRAM bandwidth similar similar enough LLM decode 52.6 51.1 tok/s That final line is what matters. Removing around 100 W of actual GPU consumption did not remove 25% of the bottleneck, because those extra watts were not buying 25% more effective memory throughput. They were largely pushing the GPU further up its voltage/frequency curve. Games behave differently There is a useful sanity check here: other people have already power-limited the RTX 3090 Ti for gaming, and the performance penalty is larger than what I measured for LLM decode. A 2022 test reported by Tom’s Hardware compared an RTX 3090 Ti running near its normal power envelope with a tuned configuration around 300 W. Across ten games at 4K, the full-power card averaged about 107 FPS while the 300 W configuration averaged about 96 FPS. That is roughly a 10% performance loss , not my 3%. ComputerBase later measured the 3090 Ti at 300 W versus 450 W and found roughly a 15% gaming-performance difference in its power-scaling comparison. Hardwareluxx performed another explicit power-scaling experiment from 450 W down to 200 W. At 350 W, its RTX 3090 Ti retained around 90% of its 450 W performance in Control; it placed the efficiency sweet spot around 375 W for that particular gaming workload. So the rough comparison is: | Workload | 450 → 300 W performance cost | |---|---| | My Qwen 27B decode | ~2.9% | | Published 3090 Ti gaming tests | roughly 10–15% | They are not directly comparable benchmarks, but that is precisely the point. There is no universal “best RTX 3090 Ti power limit”. The optimum depends on what the GPU is doing. For my local LLM decode workload, 300 W sits in an extremely attractive region of the curve. For games, the last 150 W buys noticeably more performance. What research says about undervolting There is also experimental research behind the broader idea. Mendes, Tomás and Roma studied decoupled GPU voltage/frequency scaling for deep-learning workloads. Their experiments were on AMD Vega 10 and Radeon 5700 XT hardware rather than Ampere, so the numerical results should not be transferred to a 3090 Ti. But the mechanism is relevant. They found that GPUs could operate safely at non-default voltage/frequency combinations and reported reductions of up to 38% in energy consumption and 41% in energy-delay product in their DNN experiments without degrading network accuracy. They also found that different GPU components had different undervolting limits: the ALUs and DRAM/cache controller were among the more voltage-sensitive components. That last point is important. Undervolting is not free energy. At some point you leave the stable region. The practical problem is finding: lowest voltage subject to required frequency subject to zero computation errors across every workload you actually care about A stable gaming undervolt is not automatically a stable CUDA inference undervolt. And a configuration that survives five minutes is not necessarily one I want running a multi-hour evaluation. That is another reason I like power limiting. I give up some theoretical optimization potential, but I keep the manufacturer’s voltage/frequency management. LLM-specific research is moving in the same direction Power management is now being studied directly for LLM serving rather than borrowed from older HPC and CNN work. A 2026 COMPSAC paper evaluated adaptive GPU power capping on eight NVIDIA H200 GPUs serving Qwen3-32B. Instead of choosing one permanent power cap, the system adjusted the cap dynamically according to latency/SLA feedback. The authors reported 3.84% lower total energy consumption than a 700 W baseline while maintaining empirical SLA non-inferiority . That is a very different scale from one 3090 Ti under my desk, but the principle is identical: maximum GPU power is not necessarily the power level that maximizes useful work per joule. Another useful piece of work is TokenPowerBench, published at AAAI 2026. Instead of treating “LLM inference power” as one number, it measures energy separately for: prefill + decode and reports metrics such as energy per token and energy per request. The benchmark explicitly studies variables including model choice, context length, batch size, parallelism and quantization. That separation is important because my result should specifically be understood as a decode-throughput result . Prefill is a different workload My headline measurement is: 52.6 tok/s vs 51.1 tok/s That measures generated-token throughput. It does not imply that every phase of an LLM request becomes only 3% slower at 300 W. Prefill processes many prompt tokens in parallel and has much higher arithmetic intensity than single-token decode. It can therefore make much better use of the GPU’s compute units. Long contexts also increase KV-cache traffic and attention work. Batching changes the balance again. So the shape of the power-performance curve depends on: - model architecture, - quantization, - context length, - prompt size, - batch size / concurrent slots, - prefill versus decode, - inference engine, - CUDA kernels. This is consistent with decades of GPU DVFS research: optimal voltage/frequency settings are workload-dependent rather than a fixed property of a GPU model. It also explains something from my initial experiment that I would not use as a headline result. I observed complete conversation times of about 56 seconds in one high-power run and 73 seconds in one 300 W run. That gap is far too large to be explained by a 2.9% difference in decode throughput. Those requests were not controlled well enough for prompt-processing time, generated-token count, context state and server scheduling. So I consider: 52.6 vs 51.1 tok/s a useful measurement. I do not consider: 56 vs 73 seconds evidence that the power cap increased end-to-end latency by 30%. A proper benchmark should measure prefill and decode independently. Power cap or manual undervolt? For my machine, the answer is currently power cap. | | Power limit | Manual undervolt | |---|---|---| | What I control | Maximum watts | Voltage/frequency curve | | GPU Boost remains in control | Yes | Partially | | Setup | One command | Curve tuning | | Reproducibility | Excellent on my machine | Worse in my experience | | Stability testing | Minimal | Necessary | | Efficiency potential | Very good | Potentially better | | nvidia-smi support | Native | Not traditional Ampere V/F tuning | | My choice for LLM work | Yes | Not currently | The crucial distinction is that manual undervolting tries to optimize the silicon itself. Power limiting optimizes the operating envelope. For a hobby benchmark, finding the lowest stable voltage can be interesting. For a workstation where I want to start ThreadShelf, load a model and know exactly what configuration I am running, this: nvidia-smi -pl 300 is hard to beat. The RTX 3090 Ti was pushed far past its efficiency sweet spot The broader result is not really about saving 100 W. It is about diminishing returns. The 3090 Ti was NVIDIA’s Ampere flagship. At stock, the card can consume around 450 W. Reviews at launch already showed how aggressively NVIDIA had pushed it: substantial extra power was being spent to obtain the final slice of performance. My local-LLM workload makes this unusually visible. Observed GPU draw ~400 W ~300 W Decode throughput 52.6 tok/s 51.1 tok/s Temperature 64°C 45–51°C Power -25% Decode throughput -2.9% For me, that is an easy trade. In games the trade is less extreme: external testing suggests I should expect a larger FPS penalty at 300 W. But for a large quantized local model doing mostly single-user autoregressive decoding, the RTX 3090 Ti appears to have a very large power-hungry tail that provides little additional throughput. The reason is not mysterious. It is the combination of: 1. an aggressively clocked flagship GPU, 2. the nonlinear voltage/frequency/power curve, 3. GPU Boost spending additional power to chase higher clocks, 4. and an LLM decode workload constrained heavily by memory movement rather than raw compute. The last 100 W simply have much less useful work left to buy. What I want to test next Two points are enough to show that 450 W is wasteful for my workload. They are not enough to locate the optimum. The proper experiment is a power sweep: 200 W 225 W 250 W 275 W 300 W 325 W 350 W 375 W 400 W 425 W 450 W For each setting I want to record separately: prompt processing tok/s decode tok/s average GPU W peak GPU W J / output token J / request temperature fan speed GPU clock memory clock with a fixed prompt, fixed context, fixed output length and repeated runs. I expect a knee in the curve: a region where reducing power initially removes mostly wasted voltage/frequency headroom, followed by a point where memory throughput or required compute starts falling enough that tokens per second collapse. My first two measurements strongly suggest that 300 W is already much closer to that efficient region than 450 W for my Qwen 27B setup . But that is the next experiment, not a result I have measured yet. For now, the result is simpler: I cut the RTX 3090 Ti’s configured power limit by one third and lost only 2.9% of local-LLM decode throughput. For a GPU infamous for consuming 450 watts, that is a much more useful characteristic than its stock TDP.