{"slug": "hugging-face-adds-skypilot-storage-backend", "title": "Hugging Face Adds SkyPilot Storage Backend", "summary": "Hugging Face and SkyPilot integrated on July 7 to make Hugging Face Storage a first-class SkyPilot backend, allowing AI teams to mount Hub repos or Hugging Face Buckets into jobs via hf:// URLs while SkyPilot schedules compute across cloud, Kubernetes, Slurm, and on-prem capacity. The integration reduces egress fees and coordination costs by keeping models and datasets on the Hub while running GPU jobs on available capacity, though it requires FUSE and newer Linux runtimes for mount mode.", "body_md": "# Hugging Face Adds SkyPilot Storage Backend\n\nHugging Face and SkyPilot shipped a new infrastructure path for AI teams that split model storage from GPU execution. The July 7 integration makes Hugging Face Storage a first-class SkyPilot backend, letting developers mount Hub repos or Hugging Face Buckets into jobs with an hf:// URL while SkyPilot schedules compute across cloud, Kubernetes, Slurm, and on-prem capacity. For practitioners, the useful change is not another model release; it is a cleaner way to keep models, datasets, and checkpoints on the Hub while running training or inference wherever scarce GPUs are available. The launch also matters for cost control because the primary read path avoids Hugging Face egress fees, reducing pressure to duplicate large artifacts across vendors.\n\n### Why practitioners should care\n\nAI teams increasingly buy or reserve GPU capacity wherever they can get it, but their model weights, datasets, and checkpoints often sit in a different cloud or storage account. Hugging Face and SkyPilot's July 7 integration targets that operational gap: keep artifacts on the Hugging Face Hub, then let SkyPilot run jobs on the available GPU estate without first copying every model or dataset into cloud-specific buckets.\n\n### What changed\n\nThe release makes Hugging Face Storage a first-class SkyPilot backend. A SkyPilot task can now mount a Hugging Face Bucket or Hub repository through an hf:// path, using the same Hugging Face token teams already use for model and dataset access. The Hugging Face post says the backend supports read-write buckets for checkpoints, read-only model or dataset repos, lazy reads through the hf-mount FUSE backend, and COPY mode when a runtime cannot support FUSE. SkyPilot's storage documentation separately lists HF buckets alongside S3, GCS, Azure, R2, CoreWeave, VastData, OCI, and IBM COS as supported object storage sources.\n\n### Operational readout\n\nThe practitioner value is mostly in reducing coordination cost. If a team stores a base model on Hugging Face but has available H100s on another provider, it can launch the same job spec against that capacity instead of waiting for GPUs in the storage provider's region or maintaining duplicate object-store copies. Hugging Face also says reads from its storage layer carry no egress or CDN fees, which can matter when repeatedly streaming checkpoints or model weights into short-lived training and inference nodes.\n\nThe release still has deployment constraints. The blog and docs note that mount mode depends on hf-mount, glibc 2.34 or newer, and direct /dev/fuse access; COPY mode is the fallback where those requirements are not available. That makes this a useful infrastructure improvement rather than a universal replacement for cloud-native buckets. For LDS readers, the signal is that open AI infrastructure keeps moving toward portable model and data paths that are less tied to any single hyperscaler.\n\n## Key Points\n\n- 1Hugging Face Storage now works as a first-class SkyPilot backend through hf:// mounts for AI workloads.\n- 2The integration helps teams keep models and datasets on the Hub while scheduling GPU jobs across clouds.\n- 3Mount mode depends on FUSE and newer Linux runtimes, with COPY mode available as the safer fallback.\n\n## Scoring Rationale\n\nThis is a solid infrastructure improvement for teams managing open models, checkpoints, and multi-cloud GPU capacity. It is not a frontier model launch, but it reduces deployment friction and egress-sensitive data movement for practical ML workloads.\n\n## Sources\n\nPublic references used for this report.\n\nPractice interview problems based on real data\n\n1,625 SQL & Python problems across 15 industry datasets — the exact type of data you work with.\n\n[Try 250 free problems](/problems)", "url": "https://wpnews.pro/news/hugging-face-adds-skypilot-storage-backend", "canonical_source": "https://letsdatascience.com/news/hugging-face-adds-skypilot-storage-backend-e31d5b51", "published_at": "2026-07-07 23:46:50+00:00", "updated_at": "2026-07-08 00:05:33.804376+00:00", "lang": "en", "topics": ["ai-infrastructure", "ai-tools", "developer-tools"], "entities": ["Hugging Face", "SkyPilot", "Hugging Face Hub", "Hugging Face Buckets", "Kubernetes", "Slurm", "CoreWeave", "VastData"], "alternates": {"html": "https://wpnews.pro/news/hugging-face-adds-skypilot-storage-backend", "markdown": "https://wpnews.pro/news/hugging-face-adds-skypilot-storage-backend.md", "text": "https://wpnews.pro/news/hugging-face-adds-skypilot-storage-backend.txt", "jsonld": "https://wpnews.pro/news/hugging-face-adds-skypilot-storage-backend.jsonld"}}