NVIDIA graphics card faulting after sleep/wake cycle on Void Linux

On Void Linux, an NVIDIA MX150 GPU in a ThinkPad T480 was experiencing CUDA initialization failures after sleep/wake cycles due to corrupted GPU memory mappings and MMU faults. The solution involves adding the kernel parameters `nvidia-drm.modeset=1` and `nvidia.NVreg_PreserveVideoMemoryAllocations=1` to the GRUB command line, which preserves GPU memory allocations across suspend/resume and prevents the hardware from entering an inconsistent state.

NVIDIA graphics card faulting after sleep/wake cycle on Void Linux I run Void Linux on a Thinkpad T480 with Integrated Intel graphics and a discrete Nvidia MX150. I’ve decided to start working with more CUDA development and was running into an issue where my Nvidia GPU would suddenly stop being detected by CUDA applications. The only way I could figure out to get it back online was by rebooting my computer. Eventually I became so frustrated that I decided to dive in and find a solution. The Problem Here’s what I was experiencing: - GPU appeared normal in nvidia-smi - All NVIDIA kernel modules were loaded correctly - Device files in /dev/nvidia existed with proper permissions - But testing CUDA availability using Pytorch, by running the following command would return FAlSE: 1 python -c "import torch; print torch.cuda.is available " The root cause was eventually found in the kernel logs dmesg : NVRM: Xid PCI:0000:01:00 : 31, pid=1596, name=modprobe, Ch 00000003, intr 10000000. MMU Fault: ENGINE HOST6 HUBCLIENT HOST faulted @ 0x1 01010000. Fault is of type FAULT PDE ACCESS TYPE READ NVRM: Xid PCI:0000:01:00 : 154, GPU recovery action changed from 0x0 None to 0x2 Node Reboot Required The Root Cause The issue stems from NVIDIA GPU memory management during sleep/resume cycles. When the system suspends: - The GPU’s memory mappings and contexts can become corrupted - The GPU’s Memory Management Unit MMU enters a faulted state - While the driver stack appears to reload correctly, the GPU hardware itself is in an inconsistent state - CUDA runtime fails to initialize because it can’t establish proper memory contexts This seems to be common on mobile NVIDIA GPUs like the MX series in laptops with hybrid graphics setups running Linux. At least, I’ve seen a few postings about this. The Solution The fix is to add these parameters to your kernel command line in /etc/default/grub : 1GRUB CMDLINE LINUX DEFAULT="... nvidia-drm.modeset=1 nvidia.NVreg PreserveVideoMemoryAllocations=1" Then update GRUB: 1sudo update-grub And reboot to apply the changes. What These Parameters Do nvidia-drm.modeset=1 : Enables kernel mode setting for the NVIDIA driver, providing better integration with the display subsystem and more robust power managementnvidia.NVreg PreserveVideoMemoryAllocations=1 : Tells the NVIDIA driver to preserve GPU memory allocations across suspend/resume cycles, preventing the MMU faults Why This Works These parameters ensure that: - GPU memory contexts are properly preserved during sleep - The kernel’s display management system maintains better control over the GPU state - Memory mappings remain consistent across suspend/resume cycles - The GPU’s MMU doesn’t enter the faulted state that breaks CUDA Conclusion Haven’t had an issue since making this fix Hopefully someone else can benefit from this as well