# SDXL Performance on Low VRAM > Source: > Published: 2026-08-21 00:19:00+00:00 For now, here’s what I got from a few experiments on a Colab T4: I tried to treat this as a **standard-Diffusers control**, rather than as an attempt to reproduce Celestium itself. The main thing that stood out to me is that, at least in this control, the low-VRAM boundary moved quite a lot depending on **what was resident at the same time and at what granularity the runtime offloaded it**. The nominal “4 / 6 / 8 GB” number by itself was much less informative. One important caveat up front: I used a 16 GB-class T4 and limited the **PyTorch caching allocator** to 4 / 6 / 8 GiB with [ torch.cuda.memory.set_per_process_memory_fraction()](https://docs.pytorch.org/docs/main/generated/torch.cuda.memory.set_per_process_memory_fraction.html). So these are With that limitation, the most useful part of the result looked like this: | Allocator budget | Diffusers policy / control | Result | Peak PyTorch GPU memory | |---|---|---|---| | 4 GiB | model CPU offload | OOM | failed around 3.82 GiB allocated | | 4 GiB | group offload, block-level | PASS | ~3.30 GiB allocated / 3.92 GiB reserved | | 4 GiB | group offload, leaf-level | PASS | ~3.01 GiB / 3.69 GiB | | 4 GiB | sequential CPU offload | PASS | ~3.01 GiB / 3.02 GiB | | 6 GiB | plain full CUDA residency | OOM | failed while placing the pipeline | | 6 GiB | model CPU offload | PASS | ~5.21 GiB / 5.77 GiB | | 6 GiB | Base → explicit unload → Refiner | PASS | ~5.77 GiB max reserved | | 8 GiB | model CPU offload | PASS | ~5.21 GiB / 5.77 GiB | So, very roughly, my control behaved more like: ``` "Does SDXL fit in N GiB?" ↓ "Which working set has to fit in N GiB at the same time?" ↓ "At what granularity can the runtime evict / reload it?" ``` That seems potentially relevant to your 4 / 6 / 8 GB results, especially the **6 GB Base + Refiner = partially stable** boundary. Diffusers’ own offloading terminology maps fairly cleanly onto this distinction. Its [group-offloading documentation](https://huggingface.co/docs/diffusers/v0.36.0/en/api/utilities) describes: That was almost exactly what I saw at the artificial 4 GiB boundary: model-level offload did not fit, while finer-grained group/leaf/sequential policies did. The successful block-group, leaf-group, and sequential 4 GiB runs also produced the **same output file hash** with the same seed/prompt in this sanity check, so I did not see an obvious output-correctness difference from changing only the residency policy. I also tried a deliberately simple stage-lifecycle control: ``` SDXL Base → generate latent → move latent to CPU → release Base hooks → delete Base → gc.collect() → torch.cuda.empty_cache() → load Refiner → run Refiner ``` That completed under a **6 GiB allocator budget**. I would not interpret that as “6 GB should always work” — a physical 6 GB RTX 3050 is a very different environment — but it suggests that **the exact Base → Refiner residency transition may be a high-information boundary**. In other words, your “partially stable” result may become easier to compare with other runtimes if a log only identifies which broad stage the unstable runs reach: ``` Base load → Base denoise → Base/Refiner transition → Refiner → VAE decode → later post-processing ``` Even just the stage label would separate several possibilities without requiring a large new benchmark. I also followed up on the fragmentation side because the first OOM snapshots did show the classic pattern where there was nontrivial reserved-but-unused space, but no sufficiently large usable block. I reran three boundary cases with: ``` PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True ``` PyTorch’s recent [CUDA caching-allocator write-up](https://docs.pytorch.org/devlogs/eager/2026-06-01-cuda-caching-allocator/) is useful here: allocations in separate normal CUDA segments cannot simply be merged, while expandable segments remove some of those cross-segment barriers. It also explicitly notes that expandable segments do **not** eliminate every kind of fragmentation. In my runs, expandable segments substantially reduced the inactive/free-block fragmentation: | Control | Inactive blocks, default | With expandable segments | Pass/fail changed? | |---|---|---|---| | plain 6 GiB | ~0.253 GiB | ~0.0006 GiB | No | | model-offload 4 GiB | ~0.169 GiB | ~0.0056 GiB | No | | plain 8 GiB | ~0.315 GiB | ~0.054 GiB | No | So for these particular controls, I would separate two claims: At 6 GiB with expandable segments, for example, the pipeline had essentially filled the allocator budget with active allocations before failing. At 4 GiB, model-level offload similarly ended up essentially filling the budget, while finer-grained offloading succeeded. That makes me think it could be useful to treat: ``` working-set / residency limit offload granularity allocator fragmentation long-session state stability ``` as related but separate axes when comparing Celestium with other low-VRAM runtimes. That is not a diagnosis of Celestium; it is just the separation that gave the clearest results in the T4 control. For model CPU offload at a 6 GiB allocator budget, repeated inference in the same process looked approximately like: ``` run 1: 33.1 s run 2: 15.9 s run 3: 15.5 s run 4: 15.6 s run 5: 15.2 s ``` Peak GPU allocation stayed around **5.21 GiB**, with ~** 5.77 GiB reserved**. So if performance numbers are compared between runtimes, I think it would help to distinguish **cold first generation** from **warm/steady-state generation**. Otherwise the initialization / paging / transfer cost can dominate the apparent result. I also saw process RSS rise substantially during the first couple of offloaded runs and then roughly plateau around 9.4–9.5 GiB in this small Colab environment. PyTorch’s pinned-host allocator counters stayed essentially near zero, so I would **not** call this a pinned-memory leak; RSS includes much more than the pinned allocator. But it did reinforce that peak/system RAM is useful to record next to peak VRAM when evaluating an offload-heavy strategy. A plain 8 GiB run eventually OOMed while executing `self.vae.decode(...)` , specifically inside an upsampling convolution in the VAE decoder, on a 512 MiB allocation. I also tried `enable_vae_tiling()` , and at first glance it appeared to make no difference: same OOM, same memory state. There is a subtle reason not to over-interpret that result. Diffusers’ [ AutoencoderKL implementation](https://github.com/huggingface/diffusers/blob/main/src/diffusers/models/autoencoders/autoencoder_kl.py) only switches to tiled decode when the latent dimension is `sample_size: 1024` ; with the four VAE resolution blocks, that corresponds to a 128-latent threshold. A 1024×1024 SDXL latent is also 128×128, so the strict `>` test means my exact 1024×1024 control followed the normal decode path even though tiling had been enabled.So I would treat that A/B as a useful little warning about benchmark setup, **not** as evidence that VAE tiling is ineffective. A deliberate tiling test should use a resolution above the default threshold or explicitly inspect/configure the tiling threshold. So my current read from the control is: **the interesting comparison is probably not just “can SDXL generate on 4/6/8 GB?”, but “what residency/eviction policy gets it there, what working headroom does it preserve, and does that policy stay stable over stage transitions and repeated runs?”** Your 6 GB Base+Refiner boundary in particular looks like a good place to learn a lot with very little extra instrumentation. A stage label plus peak GPU memory — and ideally system RAM / cold-vs-warm timing — would already make it much easier to compare against Diffusers-style offloading without assuming that the two runtimes are doing the same thing internally.