A new open-source tool, Mock Nvidia GPUs, lets Linux developers simulate NVIDIA GPU telemetry for monitoring software by intercepting NVML and nvidia-smi calls, without emulating CUDA or actual GPU ha…
Fine-tuning a 7B-parameter LLM requires 12 to 120 GB of VRAM depending on the method, according to a practical guide. Full fine-tuning in fp16 needs 80–120 GB, LoRA needs 24–32 GB, QLoRA needs 12–16 G…
Host overhead, caused by the CPU blocking the GPU, is a major source of inefficiency in AI inference, leading to low GPU kernel utilization and doubling GPU costs when at 50%. Modal recommends using t…
A developer created a daemon to reduce GPU temperatures on the NVIDIA DGX Spark by clock-locking via nvidia-smi -lgc. The daemon samples temperature every 30 seconds and adjusts clock ceilings, droppi…
A developer found that `nvidia-smi` reported 97% GPU utilization on an H100 cluster while actual training throughput was less than half of expected benchmarks. Tracing via eBPF revealed the GPU was id…
A GPU training pipeline at a major AI company breached its SLA despite showing 97% GPU utilization across all monitoring tools (Datadog, Grafana, nvidia-smi). Using eBPF kernel tracing, an SRE identif…
The "CUDA out of memory" error in Stable Diffusion WebUI is often caused by configuration issues rather than insufficient GPU hardware, particularly due to PyTorch's memory allocator failing to releas…
The **nvidia-smi** (NVIDIA System Management Interface) is a command-line tool for monitoring, managing, and diagnosing NVIDIA GPU devices, providing data on performance, temperature, utilization, pow…