# Nvprobe – Open-source, zero-setup CLI for CUDA benchmarks

> Source: <https://github.com/SergioZ3R0/nvprobe>
> Published: 2026-07-25 22:43:21+00:00

**NVIDIA GPU & CUDA Benchmark Suite**

*Automate CUDA workloads • HPL & HPCG • MLPerf inference • Custom kernels • Interactive reports*

[nvprobe.scszero.com](https://nvprobe.scszero.com/)

```
pip install nvprobe && nvprobe setup && nvprobe run --local
```

Bandwidth |
MatMul / Attention |
Conv2D |
|---|---|---|
| H2D / D2H / D2D across buffer sizes | fp32, fp16, int8 custom CUDA kernels | 2D convolution benchmarks |
HPL (FP64 Linpack) |
HPCG |
MLPerf Inference |
| Datacenter GPUs: A100, H100, B200, L40S… | Conjugate Gradients | ONNX Runtime via cmx4mlperf |

**Bundled CUDA runtime**— CuPy`[ctk]`

via pip, no system toolkit required**Auto-downloaded HPC tools**— NVIDIA HPC Benchmarks cached in`~/.nvprobe/tools/`

**Interactive HTML reports**— Chart.js charts with GPU / transfer / precision dropdowns and oscilloscope-style glow** A/B comparison**— compare two result sets side-by-side** Slurm integration**— generate, submit, monitor, collect from HPC clusters** SQLite storage**— all results persisted; CSV / JSON export

| Step | Command | What it does |
|---|---|---|
| 1 | `pip install nvprobe` |
Install the package |
| 2 | `nvprobe setup` |
Install CuPy, download HPL/HPCG, generate configs |
| 3 | `nvprobe env` |
Verify GPU detection, driver, CUDA version |
| 4 | `nvprobe run --local` |
Run all benchmarks locally |
| 5 | `nvprobe report --open` |
Generate & open interactive HTML report |

Or from source:

```
git clone https://github.com/SergioZ3R0/nvprobe.git && cd nvprobe
pip install -e . && nvprobe setup && nvprobe run --local
```

| Command | Description |
|---|---|
`nvprobe compare --a results/run1 --b results/run2` |
Compare two runs |
`nvprobe run --config configs/cluster.yaml` |
Run with custom YAML config |
`nvprobe slurm submit --config configs/cluster.yaml` |
Submit Slurm job |
`nvprobe slurm status` |
Check Slurm job status |
`nvprobe setup --cuda 13` |
Setup with specific CUDA version |

Chart.js canvas-based charts with interactive controls:

**Bandwidth**— filter by GPU and transfer type (H2D / D2H / D2D)** MatMul / Attention**— filter by GPU and precision (fp32 / fp16)** Range slider**— zoom into any x-axis region** Moving average**— smoother trend lines for dense data

```
name: my-run
gpu:
  models: ["L40S", "B200"]
slurm:
  enabled: true
  partition: gpu
  gpus_per_node: 8
precisions: [fp32, fp16]
benchmarks:
  - name: bandwidth
    params:
      sizes_mb: [1, 4, 16, 64, 256, 1024]
  - name: custom
    params:
      kernels: [matmul, attention]
nvprobe/
├── nvprobe/
│   ├── cli.py                     # CLI entry point
│   ├── config.py                  # YAML config loader
│   ├── runner.py                  # Benchmark orchestration
│   ├── slurm.py                   # Slurm job management
│   ├── reporter.py                # Plotly HTML report generator
│   ├── db.py                      # SQLite storage + CSV/JSON export
│   └── benchmarks/
│       ├── base.py                # Base class, GPU detection, diagnostics
│       ├── bandwidth.py           # Memory bandwidth tests
│       ├── custom.py              # Custom CUDA kernels
│       ├── hpl.py                 # HPL wrapper
│       ├── hpcg.py                # HPCG wrapper
│       ├── mlperf.py              # MLPerf via cmx4mlperf
│       └── _cuda/                 # Raw CUDA kernels
├── configs/
│   ├── default.yaml
│   └── local.yaml
├── nvprobe.svg
├── index.html
├── README.md
└── pyproject.toml
```

**HPL / HPCG**— NVIDIA HPC Benchmarks binaries are validated for datacenter GPUs (A100, H100, B200, L40S…). They may crash (SIGSEGV) on RTX series. Bandwidth and custom kernels work on any CUDA GPU.**MLPerf cuDNN**—`mlcr`

discovers cuDNN via system CUDA paths. If installed via`pip install nvidia-cudnn-cuXX`

, pre-register with:`mlcr get,cudnn,nvidia --input=$(python3 -c 'import nvidia.cudnn; print(nvidia.cudnn.__path__[0]'))`

Python 3.10+ • NVIDIA GPU with CUDA drivers • `nvidia-smi`

in PATH • Slurm (optional)