{"slug": "all-paid-gpu-pools-fail-to-start-mount-attr-idmap-error-at-container-init-exit", "title": "All paid GPU pools fail to start: MOUNT_ATTR_IDMAP error at container init (Exit code 128)", "summary": "Every paid GPU pool on Hugging Face Jobs and Spaces has failed to start since about 00:10 UTC on 2026-09-25, aborting 4 seconds after launch with exit code 128 and the error \"failed to set MOUNT_ATTR_IDMAP on /usr/bin/nvidia-cuda-mps-control: invalid argument,\" according to a user report. The affected flavors are t4-small, l4x1 and a10g-small, while cpu-basic still runs; the last successful GPU job completed at 20:34 UTC on 2026-09-24. The failures reproduce on stock public images including nvidia/cuda:12.4.1-base-ubuntu22.04, pytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel and python:3.12-slim, even when running only a GPU check, and Hugging Face's status page still lists Jobs as operational.", "body_md": "Same error on Hugging Face Jobs, not only Spaces, and it started earlier\n\nthan the first reports here.\n\nSince about 00:10 UTC on 2026-09-25, every GPU job on my account fails\n\n4 s after it starts with exit code 128:\n\nfailed to set MOUNT_ATTR_IDMAP on /usr/bin/nvidia-cuda-mps-control:\n\ninvalid argument\n\nFlavours affected: t4-small, l4x1, a10g-small. cpu-basic still runs.\n\nThe last GPU job that succeeded was at 20:34 UTC on 2026-09-24.\n\nIt fails on stock public images running nothing but a GPU check:\n\nnvidia/cuda:12.4.1-base-ubuntu22.04   nvidia-smi                 07:47 and ~08:10 UTC\n\npytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel  torch.cuda check   ~07:51 UTC\n\npython:3.12-slim                      nvidia-smi                 07:45 UTC\n\nThe status page still shows Jobs as operational. Job IDs are available\n\nto HF staff on request.", "url": "https://wpnews.pro/news/all-paid-gpu-pools-fail-to-start-mount-attr-idmap-error-at-container-init-exit", "canonical_source": "https://discuss.huggingface.co/t/all-paid-gpu-pools-fail-to-start-mount-attr-idmap-error-at-container-init-exit-code-128/180730#post_2", "published_at": "2026-09-25 08:29:49+00:00", "updated_at": "2026-09-25 08:31:08.383275+00:00", "lang": "en", "topics": ["ai-infrastructure"], "entities": ["Hugging Face", "Hugging Face Jobs", "Hugging Face Spaces", "nvidia-cuda-mps-control", "nvidia/cuda:12.4.1-base-ubuntu22.04", "pytorch/pytorch:2.6.0-cuda12.4-cudnn9-devel", "python:3.12-slim"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/all-paid-gpu-pools-fail-to-start-mount-attr-idmap-error-at-container-init-exit", "markdown": "https://wpnews.pro/news/all-paid-gpu-pools-fail-to-start-mount-attr-idmap-error-at-container-init-exit.md", "text": "https://wpnews.pro/news/all-paid-gpu-pools-fail-to-start-mount-attr-idmap-error-at-container-init-exit.txt", "jsonld": "https://wpnews.pro/news/all-paid-gpu-pools-fail-to-start-mount-attr-idmap-error-at-container-init-exit.jsonld"}}