Amazon Web Services published a reference architecture for running multi-tenant GPU clusters on Amazon SageMaker HyperPod with Amazon EKS, using AWS IAM Identity Center for authentication, per-team Sa…
Amazon published best practices for administering Amazon SageMaker HyperPod through Amazon SageMaker Unified Studio, laying out four layers of control — organization, project, cluster, and workload — …
NVIDIA released Topograph, an open source toolkit that discovers cluster network topology from cloud APIs or on-premises fabric systems and publishes it as Kubernetes node labels, Slurm topology confi…
A developer proposes a unified mental model for distributed compute systems, arguing that frameworks like Kubernetes, Slurm, Ray, and Spark share fundamental challenges in scheduling, resource managem…
SkyPilot, an open-source framework, enables users to schedule jobs across multiple Slurm clusters, Kubernetes, and cloud providers from a single interface, addressing the pain points of managing multi…
ParallelClusterMaker, an open-source CLI toolkit co-written with Anthropic's Claude Code, automates the creation and destruction of AWS ParallelCluster v3 stacks, enabling researchers and engineers to…
Nvprobe, an open-source, zero-setup CLI tool for CUDA benchmarks, has been released on GitHub. The tool automates CUDA workloads including HPL, HPCG, MLPerf inference, and custom kernels, and generate…
AMD has open-sourced Spur, a modern GPU job scheduler written in Rust under Apache 2.0, to address the complexity of scheduling AI and HPC workloads on GPU clusters. Spur provides Slurm-compatible CLI…
Hugging Face and SkyPilot integrated on July 7 to make Hugging Face Storage a first-class SkyPilot backend, allowing AI teams to mount Hub repos or Hugging Face Buckets into jobs via hf:// URLs while …
A researcher using nanoGPT on a MacBook Pro faces dependency failures when moving to a GPU cluster, prompting a solution using Nix and Flox to create reproducible, cross-platform runtime environments …
NVIDIA and SchedMD have released a new topology/block plugin for Slurm 23.11 that enables topology-aware job scheduling on NVIDIA GB200 NVL72 systems, allowing workloads to be aligned with NVLink doma…
Convergent infrastructure requirements for the foundation model lifecycle—including tightly coupled accelerator compute, high-bandwidth networking, and distributed storage—and highlights the growing r…