[WIP RELEASE] Celestium Neon - Fix OOM on 4GB GPUs for SDXL | GTX 1650 Benchmark

A developer's work-in-progress release of Celestium Neon aims to fix out-of-memory (OOM) errors on 4GB GPUs for SDXL, with a GTX 1650 benchmark. The release notes highlight Windows-specific issues, including the deprecated pynvml package (now nvidia-ml-py) and the need to distinguish PyTorch cache cleanup from GPU reset, which on Windows is a driver restart requiring admin privileges. The developer suggests adding telemetry for peak memory usage and OOM counters, and notes that gc.collect() plus torch.cuda.empty_cache() may not free memory held by live tensors, proposing a regression test for zombie tensors.

For now, there may be some Windows-specific gotchas here: A couple of the TODOs in the screenshot look less like GTX 1650 limitations and more like Windows / driver / memory-accounting boundaries , so I think you can probably make Neon more predictable without changing its basic direction. The two most immediate points are: For the NVML dependency, the NVIDIA-maintained Python package is currently nvidia-ml-py https://pypi.org/project/nvidia-ml-py/ , while the import surface is still pynvml : pip install nvidia-ml-py python from pynvml import nvmlInit The separate pynvml PyPI project is now deprecated and points users toward nvidia-ml-py . The --gpu-reset is not recognized line on driver 591.86 may not mean “consumer GPU reset unsupported.” NVIDIA’s nvidia-smi changelog https://docs.nvidia.com/deploy/nvidia-smi/index.html says Windows support for nvidia-smi -r / --gpu-reset was added between the nvidia-smi updates.More importantly, current NVIDIA documentation describes GPU reset on Windows as a driver restart that disables/re-enables the affected device and requires administrator privileges. That is a fairly different operation from freeing PyTorch cache. So if these aren’t already separate internally, I would probably keep the cleanup path conceptually split something like: Python/tensor lifetime cleanup ↓ PyTorch allocator cache cleanup ↓ external/stale process handling ↓ CUDA/GPU/device recovery ↓ Windows driver restart That lets RESET , Kill Zombie , periodic cleanup, etc. keep their useful roles without making every “free VRAM” operation equivalent to a device reset. For testing, I also think a very small amount of extra telemetry would make the 4 GB result much easier to interpret: generation peak: torch peak allocated torch peak reserved after generation: allocated / reserved after Neon cleanup: allocated / reserved OOM counters: num ooms num alloc retries Windows: dedicated GPU memory shared GPU memory recovery: after a real OOM, can the same Python process generate again? That last one may be particularly useful for Neon: “OOM happened, cleanup ran, then the same process successfully generated again” is a stronger recovery test than only measuring a low post-cleanup idle number. I did a small synthetic sanity check on a T4 with PyTorch’s caching allocator capped at approximately 4 GiB . It is not a Neon/SDXL benchmark and it does not reproduce Windows, but it produced one distinction that may be useful as a regression-test idea: gc.collect + torch.cuda.empty cache ran; gc.collect + empty cache left that full That lines up with PyTorch’s documented distinction: empty cache releases unused cached allocator memory, but does not free memory occupied by live tensors https://docs.pytorch.org/docs/main/notes/cuda.html , and PyTorch has also documented CUDA tensors being kept alive by So if “zombie tensor” in Neon is intended to include still-live Python references , something like a retained-exception/traceback case could be a useful tiny regression test. If Kill Zombie is instead about stale external GPU processes, then it is really a separate layer, which is fine too. One design boundary I would be particularly careful about is GPU RESET . According to current NVIDIA documentation, on Windows it is not just a more aggressive equivalent of: torch.cuda.empty cache It is a driver restart . NVIDIA also says GPU reset is not guaranteed to work in every situation and is not currently recommended as a general production recovery mechanism. So I would treat that as the last recovery tier, separately from normal cleanup and external-process handling, and perhaps probe whether the installed nvidia-smi actually supports -r before presenting the action. That also avoids an odd upgrade trap: on the 591.86 setup shown here, the command is simply unrecognized; on a v595+ Windows stack the same call can become a real driver restart. Overall, I think the useful thing Neon can expose is not just “VRAM went from X to Y.” It can make the different failure states visible: live Python/CUDA object allocator cache fragmented/reserved allocator state external GPU process Windows shared-memory fallback device/driver state and then use the least invasive recovery path that matches the layer. That would preserve the practical “make SDXL usable/recoverable on a 4 GB machine” goal while also making the 1650/3060/4090 tester results much easier to compare and interpret.