nvidia-smi (NVIDIA System Management Interface) is a command-line tool that provides monitoring, management, and diagnostic information for NVIDIA GPU devices. It communicates directly with the NVIDIA driver and GPU, and can:
- Monitor GPU performance, temperature, and utilization
- Manage power, clock speeds, and ECC
- Control persistence mode and compute modes
- Query detailed metrics for automation and monitoring nvidia-smi Shows a summary table with:
- GPU index, name, and UUID
- Driver & CUDA versions
- GPU & memory utilization
- Power consumption and temperature
- Active processes using the GPU
nvidia-smi -l 5
nvidia-smi -lms 500
nvidia-smi --filename=/var/log/gpu.log -l 5
nvidia-smi dmon Example: 0 85 64 23 5 0 0 405 1110 nvidia-smi --query-gpu=index,name,uuid,temperature.gpu,utilization.gpu,memory.used,memory.total --format=csv Output: index, name, uuid, temperature.gpu, utilization.gpu [%], memory.used [MiB], memory.total [MiB] 0, NVIDIA RTX A6000, GPU-02afcc1a-…, 58, 72 %, 13456 MiB, 49152 MiB
nvidia-smi --query-gpu=memory.used,memory.total --format=csv
nvidia-smi --query-gpu=temperature.gpu,power.draw --format=csv,noheader,nounits
nvidia-smi --query-gpu=name --format=csv,noheader
nvidia-smi pmon -c 1
nvidia-smi pmon Example output: 0 3024 C 23 5 0 0 python3 Terminate a process:
sudo kill -9 <pid>
sudo nvidia-smi -pm 1
sudo nvidia-smi -pm 0
sudo nvidia-smi -pl 250
sudo nvidia-smi --lock-gpu-clocks=900,1500
sudo nvidia-smi --reset-gpu-clocks
sudo nvidia-smi --lock-memory-clocks=405,1215
sudo nvidia-smi -i 0 --gpu-reset
nvidia-smi -q -d ECC
sudo nvidia-smi -e 1
Example:
sudo nvidia-smi -c 3
nvidia-smi -q -d SUPPORTED_CLOCKS
nvidia-smi -q -d PERFORMANCE
nvidia-smi -q -d PCI
nvidia-smi -q -d FAN
nvidia-smi --query-gpu=name,utilization.gpu --format=csv,noheader
nvidia-smi --query-gpu=temperature.gpu --format=csv,nounits
nvidia-smi --query-gpu=index,uuid,temperature.gpu,power.draw --format=csv -l 10 --filename=gpu_stats.csv
- Use
--query-gpu
with--format=csv,noheader,nounits
in scripts. - Use GPU UUIDs for consistent identification.
- Combine with watch :watch -n 2 nvidia-smi #!/bin/bash LOGFILE="/var/log/nvidia_smi_monitor.csv" echo "timestamp,gpu_index,uuid,utilization.gpu,memory.used,memory.total,temperature.gpu,power.draw" > $LOGFILE
while true; do
nvidia-smi --query-gpu=timestamp,index,uuid,utilization.gpu,memory.used,memory.total,temperature.gpu,power.draw --format=csv,noheader >> $LOGFILE
sleep 5 done
source & further reading
gist.github.com — original article
agent-proxy — a zero-dep proxy that shows the bloat in Claude Code's requests (ranked tool table + full readable Markdown of every request)
PowerShell profile and test suite
A plan-of-plans for a production-grade end-to-end e-commerce website.