{"slug": "wip-release-celestium-neon-fix-oom-on-4gb-gpus-for-sdxl-gtx-1650-benchmark", "title": "[WIP RELEASE] Celestium Neon - Fix OOM on 4GB GPUs for SDXL | GTX 1650 Benchmark", "summary": "A developer's work-in-progress release of Celestium Neon aims to fix out-of-memory (OOM) errors on 4GB GPUs for SDXL, with a GTX 1650 benchmark. The release notes highlight Windows-specific issues, including the deprecated pynvml package (now nvidia-ml-py) and the need to distinguish PyTorch cache cleanup from GPU reset, which on Windows is a driver restart requiring admin privileges. The developer suggests adding telemetry for peak memory usage and OOM counters, and notes that gc.collect() plus torch.cuda.empty_cache() may not free memory held by live tensors, proposing a regression test for zombie tensors.", "body_md": "For now, there may be some Windows-specific gotchas here:\n\nA couple of the TODOs in the screenshot look less like GTX 1650 limitations and more like **Windows / driver / memory-accounting boundaries**, so I think you can probably make Neon more predictable without changing its basic direction.\n\nThe two most immediate points are:\n\nFor the NVML dependency, the NVIDIA-maintained Python package is currently [ nvidia-ml-py](https://pypi.org/project/nvidia-ml-py/), while the import surface is still\n\n`pynvml`\n\n:\n\n```\npip install nvidia-ml-py\npython\nfrom pynvml import nvmlInit\n```\n\nThe separate `pynvml`\n\nPyPI project is now deprecated and points users toward `nvidia-ml-py`\n\n.\n\nThe `--gpu-reset is not recognized`\n\nline on driver **591.86** may not mean “consumer GPU reset unsupported.” NVIDIA’s [ nvidia-smi changelog](https://docs.nvidia.com/deploy/nvidia-smi/index.html) says Windows support for\n\n`nvidia-smi -r / --gpu-reset`\n\nwas added between the `nvidia-smi`\n\nupdates.More importantly, current NVIDIA documentation describes GPU reset on Windows as a **driver restart** that disables/re-enables the affected device and requires administrator privileges. That is a fairly different operation from freeing PyTorch cache.\n\nSo if these aren’t already separate internally, I would probably keep the cleanup path conceptually split something like:\n\n```\nPython/tensor lifetime cleanup\n        ↓\nPyTorch allocator cache cleanup\n        ↓\nexternal/stale process handling\n        ↓\nCUDA/GPU/device recovery\n        ↓\nWindows driver restart\n```\n\nThat lets `RESET`\n\n, `Kill Zombie`\n\n, periodic cleanup, etc. keep their useful roles without making every “free VRAM” operation equivalent to a device reset.\n\nFor testing, I also think a very small amount of extra telemetry would make the 4 GB result much easier to interpret:\n\n```\ngeneration peak:\n    torch peak allocated\n    torch peak reserved\n\nafter generation:\n    allocated / reserved\n\nafter Neon cleanup:\n    allocated / reserved\n\nOOM counters:\n    num_ooms\n    num_alloc_retries\n\nWindows:\n    dedicated GPU memory\n    shared GPU memory\n\nrecovery:\n    after a real OOM, can the same Python process generate again?\n```\n\nThat last one may be particularly useful for Neon: **“OOM happened, cleanup ran, then the same process successfully generated again”** is a stronger recovery test than only measuring a low post-cleanup idle number.\n\nI did a small synthetic sanity check on a T4 with PyTorch’s caching allocator capped at approximately **4 GiB**. It is not a Neon/SDXL benchmark and it does not reproduce Windows, but it produced one distinction that may be useful as a regression-test idea:\n\n`gc.collect()`\n\n+ `torch.cuda.empty_cache()`\n\nran;`gc.collect()`\n\n+ `empty_cache()`\n\nleft that full That lines up with PyTorch’s documented distinction: [ empty_cache() releases unused cached allocator memory, but does not free memory occupied by live tensors](https://docs.pytorch.org/docs/main/notes/cuda.html), and PyTorch has also documented CUDA tensors being kept alive by\n\nSo if “zombie tensor” in Neon is intended to include **still-live Python references**, something like a retained-exception/traceback case could be a useful tiny regression test. If `Kill Zombie`\n\nis instead about stale external GPU processes, then it is really a separate layer, which is fine too.\n\nOne design boundary I would be particularly careful about is `GPU RESET`\n\n.\n\nAccording to current NVIDIA documentation, on Windows it is not just a more aggressive equivalent of:\n\n```\ntorch.cuda.empty_cache()\n```\n\nIt is a **driver restart**. NVIDIA also says GPU reset is not guaranteed to work in every situation and is not currently recommended as a general production recovery mechanism.\n\nSo I would treat that as the last recovery tier, separately from normal cleanup and external-process handling, and perhaps probe whether the installed `nvidia-smi`\n\nactually supports `-r`\n\nbefore presenting the action.\n\nThat also avoids an odd upgrade trap: on the 591.86 setup shown here, the command is simply unrecognized; on a v595+ Windows stack the same call can become a real driver restart.\n\nOverall, I think the useful thing Neon can expose is not just “VRAM went from X to Y.” It can make the different failure states visible:\n\n```\nlive Python/CUDA object\nallocator cache\nfragmented/reserved allocator state\nexternal GPU process\nWindows shared-memory fallback\ndevice/driver state\n```\n\nand then use the least invasive recovery path that matches the layer.\n\nThat would preserve the practical “make SDXL usable/recoverable on a 4 GB machine” goal while also making the 1650/3060/4090 tester results much easier to compare and interpret.", "url": "https://wpnews.pro/news/wip-release-celestium-neon-fix-oom-on-4gb-gpus-for-sdxl-gtx-1650-benchmark", "canonical_source": "https://discuss.huggingface.co/t/wip-release-celestium-neon-fix-oom-on-4gb-gpus-for-sdxl-gtx-1650-benchmark/179283#post_2", "published_at": "2026-08-26 23:44:57+00:00", "updated_at": "2026-08-26 23:48:30.251416+00:00", "lang": "en", "topics": ["ai-tools", "ai-infrastructure"], "entities": ["Celestium Neon", "SDXL", "GTX 1650", "NVIDIA", "nvidia-ml-py", "pynvml", "PyTorch", "T4"], "alternates": {"html": "https://wpnews.pro/news/wip-release-celestium-neon-fix-oom-on-4gb-gpus-for-sdxl-gtx-1650-benchmark", "markdown": "https://wpnews.pro/news/wip-release-celestium-neon-fix-oom-on-4gb-gpus-for-sdxl-gtx-1650-benchmark.md", "text": "https://wpnews.pro/news/wip-release-celestium-neon-fix-oom-on-4gb-gpus-for-sdxl-gtx-1650-benchmark.txt", "jsonld": "https://wpnews.pro/news/wip-release-celestium-neon-fix-oom-on-4gb-gpus-for-sdxl-gtx-1650-benchmark.jsonld"}}