How Developers Cut GPU Bills: Choosing the Right GPU and Paying Only for the Hours You Use

On-demand GPU cloud prices as of 28 October 2026 start at about $0.85 per GPU hour for an RTX 5090, $1.60 for an H100 and $1.83 for an H200 on Lium, and matching the card to the job plus per-second billing are the main levers teams can pull to cut AI compute bills, according to a guide on GPU cost management. NVIDIA founder and chief executive Jensen Huang said at the company's GTC Berlin keynote in October 2026 that capacity was the constraint on supply. The guide notes that Amazon EC2 bills per second on many instance types with a 60-second minimum, and that NVIDIA rates an eight-GPU H100 system's useful life at five to six years and GB300 NVL72 systems at nine to ten years.

GPU compute is now one of the biggest line items for teams building with AI, and demand has run well ahead of supply. At NVIDIA’s GTC Berlin keynote in October 2026, founder and chief executive Jensen Huang said capacity was the constraint. The good news for anyone paying the bills is that most of the waste comes from a handful of habits, and each one is straightforward to fix. The market now stretches from consumer cards to the newest data-center systems, and the price of the very same GPU can vary several times over from one provider to the next. The few minutes you spend matching a card to a job, and the billing settings you pick before you launch it, decide most of what you will pay. 1. Match the GPU to the job The most common mistake is renting the biggest card for every task. - Inference for small and mid-size models: a consumer-class card such as the GeForce RTX 5090 is often enough, at a fraction of data-center prices. - Fine-tuning and larger models: a 48 GB professional card or a single H100 usually fits. - Large-scale training: this is where H100, H200 and B200 nodes earn their price. The RTX 5090 is NVIDIA’s top consumer card, built on the Blackwell architecture with 32 GB of GDDR7 memory. In the cloud it sits well below H100 and H200 rates and is aimed at smaller inference, development and graphics work that does not need data-center features or large memory. The H100 remains the data-center workhorse and is still widely available from cloud and specialist providers, even as H200, B200 and B300 systems arrive above it. Move up to those systems and the scale changes: an eight-GPU H100 system carries 640 GB of total GPU memory, an eight-GPU H200 system carries 1,128 GB, and the single-board H200 SXM5 ships with 141 GB for memory-heavy models. On large training nodes the GPUs use the SXM form factor and are linked directly by fourth-generation NVLink at 900 GB/s of GPU-to-GPU bandwidth, far more than cards connected only over PCIe. That is what large-scale training needs, and what smaller jobs do not. As of 28 October 2026, on-demand prices on Lium start from about $0.85 per GPU hour for an RTX 5090, $1.60 for an H100 and $1.83 for an H200. The gap between a card that fits and one that is oversized can be several times the price for the same result. 2. Pay for seconds, not hours Short jobs add up. Debugging runs, evaluations and quick experiments often last minutes, and with hourly billing every one of them is rounded up. Per-second billing removes that waste without any change to your code. It is now common across specialist GPU clouds and marketplaces, which bill by the second for the compute you actually use. The big clouds have moved the same way: Amazon EC2 supports per-second billing on many instance types and operating systems, though it applies a 60-second minimum. The practical test is simple. If you run many short jobs, a provider that charges by the second will almost always beat one that rounds up to the hour. 3. Turn GPUs off An idle GPU costs exactly as much as a busy one, so the habits that keep utilization high pay for themselves. A few simple ones go a long way: - stop instances as soon as a job finishes, from a script rather than by memory; - set a budget alert or an automatic shutdown on long jobs; - keep data and checkpoints on storage that does not need a running GPU. 4. Avoid commitments you do not need Reserved capacity looks cheaper per hour, but you pay for every reserved hour whether you use it or not. Unless your GPUs are busy nearly all the time, on demand is usually cheaper overall. Commit only after a few weeks of real usage data. The discounts for committing are real, which is exactly why they are tempting, but they only pay off at high, steady use. Buying outright is the deepest commitment of all: NVIDIA describes an eight-GPU H100 system as having a useful life of five to six years, and its newer GB300 NVL72 systems as lasting nine to ten years, so owning hardware ties up capital against a depreciating asset for years. Renting on demand keeps spend matched to real usage and lets you move to newer cards as they arrive. 5. Shop across the market GPU prices differ enormously between hyperscalers, specialist GPU clouds and marketplaces. AWS, Google Cloud and Microsoft Azure sit at one end; specialist clouds such as RunPod occupy the middle; and marketplaces such as Vast.ai, where many data centers list spare capacity, sit at the other. The spread for a single card is wider than most teams expect. On the big clouds, rented on demand and priced per GPU-hour, the same H100 looked like this in October 2026: | Provider | On-demand H100, per GPU-hour | |---|---| | Amazon Web Services | about $6.88 | | Google Cloud | $11.06 | | Microsoft Azure | $12.29 | Specialist clouds land between those poles: in a September 2026 comparison, RunPod listed an H100 at $2.89 per GPU-hour. Even inside a single cloud, the newest card is not always the cheapest way to finish the job. AWS reported in October 2026 that, in its US East N. Virginia region, its G6e instances were about 45% cheaper per hour than P5 while running only about 6% slower, and P5en was about 44% cheaper than P6 while running only 1% to 2% slower. So compare by workload rather than reaching for the latest silicon. Marketplaces, where many data centers list spare capacity, tend to be the cheapest and carry the widest range of cards, and Lium publishes current, per-second GPU cloud pricing https://lium.io/pricing for every card, which makes that comparison quick. Whatever you choose, check prices on the day you rent, because they move with supply. 6. Spot and interruptible capacity for fault-tolerant work If your job can resume from a checkpoint, interruptible capacity is cheaper still. Save checkpoints often and it costs you little when an instance is reclaimed. The savings are substantial. Interruptible and spot tiers often run well below on-demand prices, and some providers also offer discounted batch or deferred-work tiers. For anything that tolerates a restart, these options turn spare and deferred capacity into the cheapest hours on the market. A quick checklist - Smallest GPU that fits the model and batch size - Per-second billing - Automatic shutdown when jobs end - No long-term commitment until usage is proven - Compare live prices across providers Across all six, the pattern is the same: pay for the exact card, for the exact seconds, that the work needs, and nothing more. Lium lists live on-demand prices for every card from consumer GPUs to B300, billed per second with no commitment, so each of these checks takes seconds rather than a support ticket. Prices quoted are on-demand “from” prices as read on 28 October 2026 and change with supply.