{"slug": "gpu-benchmark-software-12-steps-80-min-test-plan", "title": "GPU Benchmark Software: 12 Steps, 80-Min Test Plan", "summary": "Llama.cpp reached version 1.0.0 on November 13, 2026, its first major stable release, and the project now follows semantic-versioning rules for its public API, according to a SmartScope write-up of the 1.0 changes. The release gives GPU testers a stable local-AI throughput yardstick via llama-bench, which reports prompt processing (pp512) and token generation (tg128) in tokens per second alongside graphics benchmarks such as 3DMark and Unigine Superposition and nvidia-smi telemetry. A 12-step test plan built on those three free tools is estimated at about 80 minutes for a first pass and under 20 minutes per re-test.", "body_md": "llama.cpp reached version 1.0.0 on November 13, 2026, its first major stable release, and the project now follows semantic-versioning rules for its public API, according to a [SmartScope write-up of the 1.0 changes](https://smartscope.blog/en/blog/llama-cpp-1-0-changed-api-abi-handling-2026/). That matters for anyone who tests graphics cards. Local AI workloads finally have a stable yardstick that sits next to the older graphics tests, and a fair run of GPU benchmark software in late 2026 needs both.\n\nMost people grab one tool, run it once and post a score. That number tells you very little. A single score hides the clock speed, power draw and temperature behind it, and it cannot show whether a driver update helped or the card simply ran cooler that afternoon. The plan below fixes that with 12 steps, a few hundred lines of Python and three free building blocks: a graphics benchmark, the llama-bench tool for AI throughput, and nvidia-smi for telemetry. Expect about 80 minutes for the first pass and under 20 minutes for every re-test after that.\n\nIf you are still choosing a card for local models, the [Nvidia vs AMD GPU for local AI comparison](https://shattered.io/nvidia-vs-amd-gpu-local-ai-32gb-vs-24gb-2026/) covers the memory and price trade-offs. The steps here assume the GPU is already in the machine and you want proof of what it does.\n\nDon't miss new tech stories on Google\n\nAdd Shattered once in the Google app and our stories show up in your news suggestions.\n\n[Add now](https://www.google.com/preferences/source?hl=en&q=https%3A%2F%2Fshattered.io%2F)\n\n## What GPU Benchmark Software Measures (and What It Misses)\n\nNo single program answers “is my GPU fast?” Each tool isolates one slice of behavior, and the slices do not predict each other well. A card that tops a gaming score can fall behind in token generation if its memory bandwidth is lower. A card with huge memory can post a modest graphics score and still run larger models than anything else in its class.\n\nGraphics benchmarks such as 3DMark and Unigine Superposition render a fixed scene and report a score. They are repeatable, they are widely shared, and they let you compare your result against public databases. UL Solutions runs a public results service for 3DMark, and its [GPU ranking pages](https://benchmarks.ul.com/en/compare/best-gpus?amount=0&sortBy=VALUE_FOR_MONEY&reverseOrder=true&types=DESKTOP&minRating=0&search=nvidia) list hardware by score, which gives you a sanity check on your own number. What these tools do not do is tell you anything about language-model speed. Based on the 3DMark pages available today, UL presents the suite as a graphics and processor benchmark, not an inference test.\n\nFor AI work, llama-bench ships with llama.cpp and reports two rates in tokens per second. Prompt processing (often written pp512 for a 512-token prompt) is compute-heavy. Token generation (tg128 for 128 generated tokens) leans on memory bandwidth. Reading both numbers separately tells you which resource limits the card.\n\nThen there is telemetry. NVIDIA’s own documentation for the System Management Interface recommends nvidia-smi for testing, monitoring and benchmarking. It does not render anything and it does not run a model. It records utilization, memory, temperature, power and clocks while another workload runs, which is exactly what turns a bare score into an explanation.\n\n| Tool | What it measures | Output | Best use | \n|---|---|---|---|\n| 3DMark (Steel Nomad and CPU Profile) | Graphics and processor performance in a fixed scene | A score plus public ranking | Comparing against other owners of the same card | \n| Unigine Superposition | Sustained GPU rendering load | A score and frame rate for a chosen preset | Stability and thermal soak runs | \n| llama-bench (llama.cpp) | Local AI inference speed | Tokens per second for prompt and generation | Judging a card for local models | \n| nvidia-smi | Power, clocks, temperature, VRAM, utilization | Sampled CSV rows | Explaining why a score moved | \n| A game’s built-in benchmark | Real engine behavior with your settings | Average and low frame rates | Final check on the games you actually play | \n\nThe takeaway: pick one graphics tool, one AI tool and one logger, then use them the same way every time. Consistency beats variety. Readers who want the wider buying picture can find it in the [AMD vs Nvidia GPU buyer’s guide for AI](https://shattered.io/amd-vs-nvidia-gpu-ai-buyers-guide-97-percent-growth-vs-30x-claim-2026/), and the hardware pillar page collects more testing guides under [Hardware and Chips coverage](https://shattered.io/category/hardware/).\n\n## Prerequisites and Versions for the GPU Benchmark Software Setup\n\nThe harness uses an NVIDIA card because nvidia-smi supplies the telemetry. The graphics tools and llama.cpp also run on AMD and Intel hardware, and the advanced section covers the telemetry swaps. Everything else below is free to download, with one exception covered in step 6.\n\n| Requirement | Version or spec | Why it is needed | \n|---|---|---|\n| GPU | Any NVIDIA GeForce RTX card (the GeForce RTX 5090 is the current desktop flagship on NVIDIA’s lineup page) | Runs the workloads | \n| Operating system | Windows 11 64-bit or Ubuntu 24.04 LTS | Both work, commands for each are shown | \n| NVIDIA driver | The current Game Ready or Studio driver for your card | Provides nvidia-smi and CUDA support | \n| Python | 3.11 or newer | Runs the harness | \n| llama.cpp | v1.0.0 (stable release) | Provides llama-bench | \n| CMake | 3.20 or newer | Builds llama.cpp from source | \n| Compiler | Visual Studio 2022 Build Tools (Windows) or GCC (Linux) | Builds llama.cpp from source | \n| CUDA Toolkit | A release your driver supports | Enables the CUDA backend | \n| Disk space | About 20 GB free | Source tree, build output and a model file | \n| Model file | One GGUF model in Q4_K_M quantization, 7B to 8B parameters | The AI workload | \n\nTwo version notes save trouble later. First, llama.cpp also publishes numbered development builds. A pre-release labeled b12445 appeared on November 16, 2026, and the project’s own materials treat those as development snapshots, not replacements for the stable v1.0.0 label. Pin the stable tag for benchmarking so every run uses the same code. Second, the 1.0 release removed a set of deprecated public API functions, so scripts written against pre-1.0 nightly builds may need changes. The harness below only calls the command-line tools, which keeps it insulated from library API changes.\n\nYou also need a model that fits entirely in VRAM. A 7B or 8B model at Q4_K_M typically needs a few gigabytes, so it fits on most cards sold in the last several years. Choose a model you would actually run, because a benchmark on a model you never use is trivia. Keep the exact filename in your notes, since swapping the quantization changes the numbers more than most hardware changes do.\n\n## Steps 1 to 3: Plan the Tests, Prepare the PC, Scaffold the Project\n\n### Step 1: Define the test plan before you touch a tool\n\nWrite the plan first. It takes five minutes and prevents the most common failure in GPU testing, which is changing three things at once and then not knowing what caused the result. Decide which questions the benchmark must answer. Typical ones are whether a new driver changed anything, whether an undervolt cost performance, whether a power limit saves energy at an acceptable speed cost, or whether a used card behaves like its listing.\n\n| Workload | Tool | Metric | Repeats | \n|---|---|---|---|\n| Graphics, short | 3DMark Steel Nomad | Score | 3 | \n| Graphics, sustained | Superposition 1080p Extreme | Score and average frame rate | 3 | \n| Prompt processing | llama-bench pp512 | Tokens per second | 3 rounds of 5 reps | \n| Token generation | llama-bench tg128 | Tokens per second | 3 rounds of 5 reps | \n| Telemetry | nvidia-smi | Power, temperature, clocks, VRAM | Whole session | \n\nName each run with a label that states the one variable you changed, such as `baseline`, `driver-update` or `pl-80`. The label becomes the folder name later, so keep it short and free of spaces.\n\n### Step 2: Prepare the PC for repeatable runs\n\nRepeatability comes from the machine state, not the software. Close browsers, launchers, chat apps and overlay tools. Set the OS power mode to its high-performance option. On a laptop, plug in the charger, because battery operation can cut GPU power limits sharply. Let the system idle for five minutes after boot so background indexing and update checks finish.\n\nInstall or confirm your driver before the first run and do not change it mid-session. Reboot once after any driver install. Then check that the GPU is visible:\n\n```\nnvidia-smi\n```\n\nYou should see a table listing the driver version, the GPU name, memory, temperature and power. If the command fails, jump to the troubleshooting list. Write down the room conditions too. A card that boosts at 21 degrees Celsius ambient will boost lower at 30, and you cannot compare runs across that gap without noting it.\n\n### Step 3: Scaffold the project folder and Python environment\n\nCreate a folder for the harness and a virtual environment. The scripts use only the Python standard library, so there is nothing to install beyond Python itself.\n\n```\nmkdir gpubench\ncd gpubench\nmkdir runs\npython -m venv .venv\n```\n\nOn Windows, activate the environment with `.venv\\Scripts\\Activate.ps1` in PowerShell. On Linux, run `source .venv/bin/activate`. The finished project holds five scripts and a `runs` folder, shown in full in the project layout section near the end. Each run writes its own subfolder under `runs`, so nothing is overwritten and you can diff any two sessions months apart.\n\n## Steps 4 and 5: Capture a System Inventory and Log GPU Telemetry\n\n### Step 4: Capture a system inventory\n\nA benchmark result without its context is almost worthless a month later. Did you run it on the old driver? Was the PCIe slot negotiating x16 or x8? Which llama.cpp build produced it? Save `sysinfo.py` in the project folder. It asks nvidia-smi for the card name, driver, VBIOS, memory size, power limit and PCIe link state, then adds the OS, Python version, power profile and llama.cpp version string.\n\n``` python\nimport json\nimport platform\nimport shutil\nimport subprocess\nimport sys\nfrom pathlib import Path\n\nGPU_FIELDS = [\n    \"name\", \"driver_version\", \"vbios_version\", \"memory.total\",\n    \"power.limit\", \"pcie.link.gen.current\", \"pcie.link.width.current\",\n]\n\ndef run(cmd):\n    try:\n        out = subprocess.run(cmd, capture_output=True, text=True, timeout=20)\n        return out.stdout.strip()\n    except (OSError, subprocess.TimeoutExpired):\n        return \"\"\n\ndef gpu_info(index=0):\n    if shutil.which(\"nvidia-smi\") is None:\n        return {\"error\": \"nvidia-smi not found\"}\n    raw = run([\"nvidia-smi\", f\"--id={index}\",\n               \"--query-gpu=\" + \",\".join(GPU_FIELDS),\n               \"--format=csv,noheader,nounits\"])\n    values = [v.strip() for v in raw.split(\",\")]\n    return dict(zip(GPU_FIELDS, values))\n\ndef power_profile():\n    if platform.system() == \"Windows\":\n        return run([\"powercfg\", \"/getactivescheme\"])\n    gov = Path(\"/sys/devices/system/cpu/cpu0/cpufreq/scaling_governor\")\n    return gov.read_text().strip() if gov.exists() else \"unknown\"\n\ndef collect(llama_cli=\"llama-cli\"):\n    return {\n        \"os\": platform.platform(),\n        \"python\": sys.version.split()[0],\n        \"cpu\": platform.processor() or platform.machine(),\n        \"gpu\": gpu_info(),\n        \"power_profile\": power_profile(),\n        \"llama_cpp\": run([llama_cli, \"--version\"]).splitlines()[-1:] or [\"unknown\"],\n    }\n\nif __name__ == \"__main__\":\n    print(json.dumps(collect(), indent=2))\n```\n\nRun it once to confirm it works:\n\n```\npython sysinfo.py\n```\n\nExpected output (the values below show the format, yours will differ):\n\n```\n{\n  \"os\": \"Windows-11-10.0.26100-SP0\",\n  \"python\": \"3.12.x\",\n  \"cpu\": \"Intel64 Family 6 Model 198 Stepping 2, GenuineIntel\",\n  \"gpu\": {\n    \"name\": \"NVIDIA GeForce RTX (your card)\",\n    \"driver_version\": \"(your driver)\",\n    \"vbios_version\": \"(your VBIOS)\",\n    \"memory.total\": \"(MiB)\",\n    \"power.limit\": \"(W)\",\n    \"pcie.link.gen.current\": \"5\",\n    \"pcie.link.width.current\": \"16\"\n  },\n  \"power_profile\": \"Power Scheme GUID: ... (High performance)\",\n  \"llama_cpp\": [\"unknown\"]\n}\n```\n\nThe `llama_cpp` field reads `unknown` until you build llama.cpp in step 7. Idle GPUs often report a lower PCIe generation to save power, so a Gen 1 reading at rest is normal. Read the link state again under load if you suspect a slot problem.\n\n### Step 5: Build the nvidia-smi telemetry logger\n\nThe logger starts nvidia-smi in loop mode and writes one CSV row every 500 milliseconds. It reads the child process output through a pipe and flushes each row to disk, which avoids the buffering surprise you get when you redirect nvidia-smi straight to a file. Fields that a card does not report, such as fan speed on some laptops, arrive as `[N/A]` and the report script skips them.\n\n``` python\nimport csv\nimport shutil\nimport subprocess\nimport threading\nfrom pathlib import Path\n\nFIELDS = [\n    \"timestamp\", \"power.draw\", \"temperature.gpu\", \"utilization.gpu\",\n    \"memory.used\", \"clocks.sm\", \"clocks.mem\", \"fan.speed\",\n]\n\nclass Telemetry:\n    def __init__(self, out_csv, interval_ms=500, gpu_index=0):\n        self.out_csv = Path(out_csv)\n        self.interval_ms = interval_ms\n        self.gpu_index = gpu_index\n        self.proc = None\n        self.thread = None\n\n    def _pump(self):\n        with open(self.out_csv, \"w\", newline=\"\") as fh:\n            writer = csv.writer(fh)\n            writer.writerow(FIELDS)\n            for line in self.proc.stdout:\n                row = next(csv.reader([line], skipinitialspace=True), None)\n                if row and len(row) == len(FIELDS):\n                    writer.writerow(row)\n                    fh.flush()\n\n    def start(self):\n        if shutil.which(\"nvidia-smi\") is None:\n            raise SystemExit(\"nvidia-smi is not on PATH\")\n        cmd = [\n            \"nvidia-smi\", f\"--id={self.gpu_index}\",\n            \"--query-gpu=\" + \",\".join(FIELDS),\n            \"--format=csv,noheader,nounits\",\n            f\"--loop-ms={self.interval_ms}\",\n        ]\n        self.proc = subprocess.Popen(cmd, stdout=subprocess.PIPE,\n                                     stderr=subprocess.DEVNULL, text=True)\n        self.thread = threading.Thread(target=self._pump, daemon=True)\n        self.thread.start()\n\n    def stop(self):\n        if self.proc:\n            self.proc.terminate()\n            self.proc.wait(timeout=10)\n        if self.thread:\n            self.thread.join(timeout=10)\n\n    def __enter__(self):\n        self.start()\n        return self\n\n    def __exit__(self, *exc):\n        self.stop()\n```\n\nTest it for ten seconds while the GPU does something, such as playing a video or running a game menu:\n\n``` python\npython -c \"import time; from telemetry import Telemetry; t = Telemetry('test.csv'); t.start(); time.sleep(10); t.stop()\"\n```\n\nThen open `test.csv`. Healthy output looks like this:\n\n```\ntimestamp,power.draw,temperature.gpu,utilization.gpu,memory.used,clocks.sm,clocks.mem,fan.speed\n2026/11/18 10:02:41.318,38.12,41,3,1210,375,405,30\n2026/11/18 10:02:41.820,38.40,41,4,1210,375,405,30\n```\n\nA 500 ms interval is a good default. Faster sampling adds load to the driver and rarely changes the conclusion, while slower sampling can miss short throttling dips. The sample values above are format examples from an idle card.\n\n## Step 6: Run the Graphics Benchmarks (3DMark and Superposition)\n\nTelemetry should run during the graphics tests too, so start the logger first in a second terminal and stop it after the last test finishes:\n\n``` python\npython -c \"import time; from telemetry import Telemetry; t = Telemetry('runs/gfx-baseline.csv'); t.start(); input('Logging. Press Enter when the graphics tests are done: '); t.stop()\"\n```\n\nRun each graphics test three times with a 90 second pause between runs. The pause lets temperatures settle so the third run starts from conditions close to the first.\n\nFor 3DMark, use the Steel Nomad test. UL Solutions reportedly sells 3DMark through Steam at $34.99, so confirm the current price and edition on the store page before buying. Run the test at default settings, wait for the result screen and write down the overall score, the graphics score and the frame-rate figure. Do not change the resolution or turn on custom settings if you want the result to be comparable with the public results database.\n\nFor Unigine Superposition, pick the 1080p Extreme preset. Its public results pages still label runs as Superposition 1.1 as of late October 2026, so that is the newest version you can confirm, though UNIGINE’s separate engine and SDK have moved on to version 2.22. Check the download page for the current license terms before you rely on it. The Extreme preset loads the card heavily for several minutes, which makes it a decent thermal soak test as well as a score generator. Record the final score and the average frames per second.\n\nTreat the first run of a session as a warm-up. The card starts cold, boost clocks are at their peak, and the score is slightly higher than the sustained figure. If the first score is the best of the three by a wide margin, that gap is a measure of how much your cooling limits sustained performance. That is useful information, not a flaw in the test.\n\nStore the scores in a text file or pass them to the harness later with the `--score` option. A line such as `steel_nomad=value` in the final command saves them into the run record.\n\n## Steps 7 and 8: Build llama.cpp v1.0.0 and Run llama-bench by Hand\n\n### Step 7: Build llama.cpp and get a model\n\nClone the repository, switch to the stable release, and build with the CUDA backend. On Linux:\n\n```\ngit clone https://github.com/ggml-org/llama.cpp\ncd llama.cpp\ngit fetch --tags\ngit tag --list \"v1*\"\ngit checkout v1.0.0\ncmake -B build -DGGML_CUDA=ON\ncmake --build build --config Release -j\n```\n\nThe `tag --list` line is there because release tags occasionally differ from the headline version, so confirm that `v1.0.0` exists before checking it out. On Windows, run the same commands from a Developer PowerShell for Visual Studio 2022. The binaries land in `build\\bin\\Release` on Windows and `build/bin` on Linux. Prebuilt release archives also exist for Windows, and that is a faster route if you do not want to install a compiler. Whichever route you pick, use the stable v1.0.0 tag and avoid the numbered pre-release builds, which the project’s [CI notes on Hugging Face](https://huggingface.co/blog/ggml-org/llama-cpp-ci-buckets) describe as automated outputs of a busy daily pipeline.\n\nFor AMD or Intel graphics, swap the CUDA flag for the Vulkan backend with `-DGGML_VULKAN=ON`. The project has kept adding Vulkan work in recent builds, and the Vulkan path is the practical cross-vendor option.\n\nDownload one GGUF model file from its Hugging Face page and place it in a `models` folder beside the project. Pick a 7B to 8B model in Q4_K_M. Record the exact file name. The [iacompa report on the 1.0 release](https://iacompa.com/news/ggml-llama-cpp-1-0-local-inference) lists the API removals if you maintain code that links against libllama, though the command-line workflow here is not affected.\n\nConfirm the build works by asking for the version:\n\n```\n./build/bin/llama-cli --version\n```\n\n### Step 8: Run llama-bench by hand first\n\nNever automate a command you have not seen work manually. Run llama-bench directly against your model, with all layers offloaded to the GPU, a 512-token prompt, 128 generated tokens and five repetitions:\n\n```\n./build/bin/llama-bench -m ../models/model-8b-q4_k_m.gguf -ngl 99 -p 512 -n 128 -r 5\n```\n\nThe default output is a Markdown table. The numbers below are placeholders that show the layout, not a measured result:\n\n```\n| model              |   size | params | backend | ngl |  test |         t/s |\n| ------------------ | -----: | -----: | ------- | --: | ----: | ----------: |\n| model 8B Q4_K_M    | x GiB  | 8 B    | CUDA    |  99 | pp512 | NNNN +- NN  |\n| model 8B Q4_K_M    | x GiB  | 8 B    | CUDA    |  99 | tg128 |  NNN +- N   |\n```\n\nCheck three things in that table. The backend column must say CUDA (or Vulkan if you built that), because CPU means the GPU was not used and every number is meaningless for your purpose. The `ngl` column should show 99 or the layer count you asked for. And the plus-minus figure after each rate shows the run-to-run spread inside the five repetitions. A spread above a few percent usually means something else is using the GPU or the card is throttling.\n\nWhy two rows? Prompt processing is a large batched calculation that stresses compute. Token generation produces one token at a time and mostly streams the model weights through memory, so it tracks memory bandwidth. When you compare two cards, the pp512 ratio often differs from the tg128 ratio, and that gap tells you which spec decides the matchup for your workload.\n\n## Step 9: Automate the Run With a Python Harness\n\nThe manual run proves the tools work. The harness makes the result repeatable. It creates a timestamped folder, saves the system inventory, runs one unrecorded warm-up pass, then starts the telemetry logger and runs the benchmark in several rounds with a cooldown between them. Each round holds its own set of repetitions, so the final summary can report both the median and the spread across rounds. That spread is the noise floor you will use in step 11 to decide whether a change was real.\n\nThe script shells out to llama-bench with JSON output and slices the text from the first opening bracket, which protects the parser if the tool prints a log line before the JSON. Save it as `runbench.py`:\n\n``` python\nimport argparse\nimport json\nimport subprocess\nimport time\nfrom datetime import datetime\nfrom pathlib import Path\n\nimport report\nimport sysinfo\nfrom telemetry import Telemetry\n\ndef llama_bench(binary, model, ngl, prompt, gen, reps):\n    cmd = [binary, \"-m\", model, \"-ngl\", str(ngl), \"-p\", str(prompt),\n           \"-n\", str(gen), \"-r\", str(reps), \"-o\", \"json\"]\n    out = subprocess.run(cmd, capture_output=True, text=True, check=True)\n    text = out.stdout\n    return json.loads(text[text.index(\"[\"):])\n\ndef main():\n    ap = argparse.ArgumentParser()\n    ap.add_argument(\"--label\", required=True)\n    ap.add_argument(\"--model\", required=True)\n    ap.add_argument(\"--bench-bin\", default=\"llama-bench\")\n    ap.add_argument(\"--ngl\", type=int, default=99)\n    ap.add_argument(\"--prompt\", type=int, default=512)\n    ap.add_argument(\"--gen\", type=int, default=128)\n    ap.add_argument(\"--reps\", type=int, default=5)\n    ap.add_argument(\"--rounds\", type=int, default=3)\n    ap.add_argument(\"--cooldown\", type=int, default=90)\n    ap.add_argument(\"--score\", action=\"append\", default=[],\n                    help=\"manual score, NAME=VALUE (repeatable)\")\n    args = ap.parse_args()\n\n    stamp = datetime.now().strftime(\"%Y%m%d-%H%M%S\")\n    run_dir = Path(\"runs\") / f\"{args.label}-{stamp}\"\n    run_dir.mkdir(parents=True)\n    (run_dir / \"sysinfo.json\").write_text(json.dumps(sysinfo.collect(), indent=2))\n\n    print(\"warm-up pass (not recorded)\")\n    llama_bench(args.bench_bin, args.model, args.ngl, args.prompt, args.gen, 1)\n\n    rounds = []\n    with Telemetry(run_dir / \"telemetry.csv\"):\n        for i in range(1, args.rounds + 1):\n            print(f\"round {i} of {args.rounds}\")\n            result = llama_bench(args.bench_bin, args.model, args.ngl,\n                                 args.prompt, args.gen, args.reps)\n            rounds.append({\"round\": i, \"llama\": result})\n            if i != args.rounds:\n                time.sleep(args.cooldown)\n\n    scores = dict(item.split(\"=\", 1) for item in args.score)\n    raw = {\"label\": args.label, \"args\": vars(args), \"rounds\": rounds,\n           \"scores\": scores}\n    (run_dir / \"raw.json\").write_text(json.dumps(raw, indent=2))\n    print(json.dumps(report.build_summary(run_dir), indent=2))\n\nif __name__ == \"__main__\":\n    main()\n```\n\nRun the baseline with the same parameters you will use for every later test. Pass the graphics scores you wrote down in step 6 so they travel with the run:\n\n```\npython runbench.py --label baseline --model ../models/model-8b-q4_k_m.gguf --bench-bin ../llama.cpp/build/bin/llama-bench --score steel_nomad=VALUE --score superposition_1080p_extreme=VALUE\n```\n\nReplace `VALUE` with the numbers from your result screens. With three rounds, five repetitions and a 90 second cooldown, the AI portion takes roughly 8 to 10 minutes on a typical card, mostly spent in cooldowns. The terminal prints progress, then the summary once the run finishes:\n\n```\nwarm-up pass (not recorded)\nround 1 of 3\nround 2 of 3\nround 3 of 3\n{\n  \"label\": \"baseline\",\n  \"rates\": {\n    \"pp512\": { \"median_tps\": NNNN.NN, \"spread_pct\": 0.6, \"rounds\": 3, \"tokens_per_joule\": N.NNNN },\n    \"tg128\": { \"median_tps\": NNN.NN, \"spread_pct\": 1.5, \"rounds\": 3, \"tokens_per_joule\": N.NNNN }\n  },\n  \"scores\": { \"steel_nomad\": \"VALUE\", \"superposition_1080p_extreme\": \"VALUE\" },\n  \"telemetry\": {\n    \"load_samples\": NNN,\n    \"power_avg_w\": NNN.N,\n    \"temp_max_c\": NN.N,\n    \"sm_clock_avg_mhz\": NNNN,\n    \"vram_peak_mib\": NNNN\n  }\n}\n```\n\nThe placeholders stand in for your own numbers. The structure is what matters here. Every run ends with the same fields, so a spreadsheet, a script or a person can read any run the same way.\n\n## Steps 10 to 12: Read the Results, Compare Runs, Archive Everything\n\n### Step 10: Read the results correctly\n\nThe summary logic lives in `report.py`, which `runbench.py` already imports. It does three things worth understanding. It filters telemetry to rows where GPU utilization is at least 50 percent, so idle time before and after the test does not drag the averages down. It takes the median of each rate across rounds instead of the mean, which keeps one unlucky round from skewing the answer. And it divides tokens per second by average power draw to give tokens per joule, a rough efficiency measure that makes undervolts and power limits easy to judge.\n\n``` python\nimport csv\nimport json\nimport statistics\nimport sys\nfrom pathlib import Path\n\ndef num(value):\n    try:\n        return float(value)\n    except (TypeError, ValueError):\n        return None\n\ndef summarize(csv_path, min_util=50.0):\n    rows = []\n    with open(csv_path, newline=\"\") as fh:\n        for row in csv.DictReader(fh):\n            util = num(row[\"utilization.gpu\"])\n            if util is not None and util >= min_util:\n                rows.append(row)\n    if not rows:\n        return {\"load_samples\": 0}\n\n    def col(name):\n        return [v for v in (num(r[name]) for r in rows) if v is not None]\n\n    power, temp = col(\"power.draw\"), col(\"temperature.gpu\")\n    clock, mem = col(\"clocks.sm\"), col(\"memory.used\")\n    return {\n        \"load_samples\": len(rows),\n        \"power_avg_w\": round(statistics.mean(power), 1) if power else None,\n        \"power_max_w\": max(power) if power else None,\n        \"temp_avg_c\": round(statistics.mean(temp), 1) if temp else None,\n        \"temp_max_c\": max(temp) if temp else None,\n        \"sm_clock_avg_mhz\": round(statistics.mean(clock)) if clock else None,\n        \"sm_clock_min_mhz\": min(clock) if clock else None,\n        \"vram_peak_mib\": max(mem) if mem else None,\n    }\n\ndef rate_table(rounds):\n    table = {}\n    for entry in rounds:\n        for item in entry[\"llama\"]:\n            kind = \"pp\" if item[\"n_prompt\"] and not item[\"n_gen\"] else \"tg\"\n            size = item[\"n_prompt\"] or item[\"n_gen\"]\n            table.setdefault(f\"{kind}{size}\", []).append(item[\"avg_ts\"])\n    result = {}\n    for name, values in table.items():\n        med = statistics.median(values)\n        spread = (max(values) - min(values)) / med * 100 if med else 0.0\n        result[name] = {\"median_tps\": round(med, 2),\n                        \"spread_pct\": round(spread, 2),\n                        \"rounds\": len(values)}\n    return result\n\ndef build_summary(run_dir):\n    run_dir = Path(run_dir)\n    data = json.loads((run_dir / \"raw.json\").read_text())\n    summary = {\n        \"label\": data[\"label\"],\n        \"rates\": rate_table(data[\"rounds\"]),\n        \"scores\": data.get(\"scores\", {}),\n        \"telemetry\": summarize(run_dir / \"telemetry.csv\"),\n    }\n    power = summary[\"telemetry\"].get(\"power_avg_w\")\n    if power:\n        for name, rate in summary[\"rates\"].items():\n            rate[\"tokens_per_joule\"] = round(rate[\"median_tps\"] / power, 4)\n    (run_dir / \"summary.json\").write_text(json.dumps(summary, indent=2))\n    return summary\n\nif __name__ == \"__main__\":\n    print(json.dumps(build_summary(sys.argv[1]), indent=2))\n```\n\nYou can rebuild a summary for any saved run, which is handy after you adjust the filter:\n\n```\npython report.py runs/baseline-20261118-101500\n```\n\nUse this table to read what you see:\n\n| What you see | Likely meaning | What to do | \n|---|---|---|\n| Spread across rounds under about 2% | Stable test conditions | Trust small differences only if they exceed the spread | \n| Round 1 much faster than rounds 2 and 3 | Thermal or power throttling as the card heats up | Check temp_max_c and the sm_clock_min_mhz value, improve cooling or lengthen cooldowns | \n| sm_clock_min_mhz far below sm_clock_avg_mhz | Clock drops during the test | Look at power draw against the power limit and at temperature | \n| vram_peak_mib near the card’s total memory | Model plus cache almost fills VRAM | Use a smaller quantization or context, or expect slowdowns | \n| tg128 low but pp512 normal | Memory-bandwidth limit | Compare memory clock and PCIe state, then check for VRAM spill | \n| Power average near the card’s power limit | Power-limited rather than temperature-limited | A higher limit might raise speed, at an efficiency cost | \n\n### Step 11: Compare two runs after one change\n\nNow change exactly one thing. Update the driver, lower the power limit, apply an undervolt, switch a cooling profile or move the card to a different slot. Rerun the harness with a new label and the same arguments:\n\n```\npython runbench.py --label driver-update --model ../models/model-8b-q4_k_m.gguf --bench-bin ../llama.cpp/build/bin/llama-bench\n```\n\nThen compare the two folders with `compare.py`, which uses the larger of the two spreads as the noise threshold:\n\n``` python\nimport json\nimport sys\nfrom pathlib import Path\n\ndef load(path):\n    return json.loads((Path(path) / \"summary.json\").read_text())\n\ndef pct(new, old):\n    return (new - old) / old * 100 if old else 0.0\n\ndef main(base_dir, test_dir):\n    base, test = load(base_dir), load(test_dir)\n    print(f\"{'metric':<12}{'base':>10}{'test':>10}{'delta %':>10}{'noise %':>10}  verdict\")\n    for name, b in base[\"rates\"].items():\n        t = test[\"rates\"].get(name)\n        if not t:\n            continue\n        delta = pct(t[\"median_tps\"], b[\"median_tps\"])\n        noise = max(b[\"spread_pct\"], t[\"spread_pct\"])\n        verdict = \"real\" if abs(delta) > noise else \"inside noise\"\n        print(f\"{name:<12}{b['median_tps']:>10}{t['median_tps']:>10}\"\n              f\"{delta:>10.1f}{noise:>10.1f}  {verdict}\")\n    bp = base[\"telemetry\"].get(\"power_avg_w\")\n    tp = test[\"telemetry\"].get(\"power_avg_w\")\n    if bp and tp:\n        print(f\"{'power W':<12}{bp:>10}{tp:>10}{pct(tp, bp):>10.1f}\")\n\nif __name__ == \"__main__\":\n    main(sys.argv[1], sys.argv[2])\npython compare.py runs/baseline-20261118-101500 runs/driver-update-20261118-113000\n```\n\nThe comparison prints a table like this one (example numbers from synthetic test data, not measurements from a real card):\n\n```\nmetric          base      test   delta %   noise %  verdict\npp512        3011.34   3130.67       4.0       0.6  real\ntg128         100.46    100.88       0.4       1.5  inside noise\npower W        300.1     280.1      -6.7\n```\n\nRead that as a story. Prompt processing rose by more than the noise, so the change helped compute-bound work. Token generation moved less than the noise, so call it unchanged. Average power fell. A change that keeps speed and trims power is a clear win. The same tool will show you the opposite case just as plainly, a change that looks good in one score and quietly hurts another.\n\nA single-digit gain on one test with a 3 percent spread is not evidence. Rerun both sides before you believe it. Treat the verdict column as a filter for what deserves a second look, not as a statistical proof.\n\n### Step 12: Archive and share the results\n\nEach run folder already holds the raw JSON, the summary, the telemetry CSV and the system inventory. Copy the whole `runs` folder to cloud storage or a Git repository after each session. Telemetry files are small, usually well under a megabyte per run, so size is not a concern. Add a one-line note to each folder’s name or a `notes.txt` that records room temperature and anything unusual, like a Windows update that landed during the session.\n\nWhen you post results to a forum, include the card, driver version, llama.cpp version, model file with its quantization, the exact llama-bench arguments and the ambient temperature. Posting a bare token rate invites arguments that these fields would settle in a sentence. Small teams that need a shared record of workstation performance can store the folders next to the rest of their [business software stack](https://shattered.io/small-business-software-stack-2026/) documentation, which keeps hardware decisions auditable when someone asks why a purchase was approved.\n\n## Common Pitfalls That Ruin GPU Benchmark Results\n\nExperienced testers still repeat the same handful of mistakes. Each one produces numbers that look fine and are wrong.\n\n**Comparing different quantizations or models.** A Q4_K_M file and a Q8_0 file of the same model differ in size and speed by a wide margin. If you swap files between runs, you are measuring the file. Pin the model name in your notes and in the label.\n\n**Testing on battery or a balanced power plan.** Laptops cut GPU power on battery, and desktop power plans can cap CPU boost, which affects prompt processing and the benchmark launcher. Plug in and use the high-performance mode for every run.\n\n**Reading only the first run.** A cold card boosts higher than a warm one. If you publish the first score of the day, you publish the best case. The harness uses a warm-up pass and rounds with cooldowns for this reason.\n\n**Leaving overlays and background apps running.** Game launchers, recording software, RGB control suites and browser tabs steal GPU time or CPU cycles. A background process using a few percent of the GPU is enough to put a spread of 5 percent into your data.\n\n**Changing the driver mid-comparison.** Driver updates can change both performance and clocks. If a driver auto-installs between your baseline and your test, you have two changes, not one. Turn off automatic driver updates for the duration of a test series.\n\n**Letting the model spill out of VRAM.** If the model and its cache do not fit, layers move to system memory and token generation collapses. The vram_peak_mib field in the summary exposes this. Pick a model and context size that fit with margin.\n\n**Using a development build for one run and the stable build for another.** Numbered b-series builds change daily. Mixing them produces a difference that has nothing to do with your hardware. Pin v1.0.0 and note the version string saved in the system inventory.\n\n**Trusting a score without its telemetry.** A higher score at much higher power draw is a different statement from a higher score at the same power. Always read the power and clock lines beside the score.\n\n## Troubleshooting: 10 Problems and Fixes\n\n**1. nvidia-smi is not recognized or not found.** On Windows the tool normally lives in the System32 folder and installs with the driver, so a missing command usually means a broken or partial driver install. Reinstall the driver with the clean-install option and reboot. Inside WSL, the tool can be absent from the normal path, and NVIDIA’s [NemoClaw release notes from November 6](https://docs.nvidia.com/nemoclaw/latest/user-guide/deepagents/release-notes/2026/11/6) describe tooling that reportedly searches for the WSL-provided copy for that reason. Call it by its full path if needed.\n\n**2. The telemetry CSV has a header and no rows.** The logger drops any line that does not split into exactly eight fields. Run `nvidia-smi --query-gpu=power.draw,fan.speed --format=csv` by hand to see what the card returns. Some cards reject a field name, and the output then contains an error instead of values. Remove the unsupported field from `FIELDS` in both the logger and the report.\n\n**3. power.draw or fan.speed shows [N/A].** Laptop GPUs and some workstation boards do not expose every sensor. The report script skips values it cannot convert, so the run still works. If power is missing entirely, the tokens-per-joule line will not appear and you should lean on clock and temperature data instead.\n\n**4. llama-bench says the backend is CPU.** The build did not include the GPU backend, or the CUDA toolkit was not found during configuration. Delete the `build` folder, confirm the toolkit is installed and visible, rerun the configure step with `-DGGML_CUDA=ON` and read the configure output for a CUDA detection line. For AMD and Intel cards, build with the Vulkan flag instead.\n\n**5. The harness crashes with a JSON error.** The parser slices from the first opening bracket, but a mismatched llama-bench binary or a crash inside it can leave nothing valid to parse. Run the same command by hand and read the raw output. If the tool prints an error about an unknown argument, you are probably using an older or newer build than the one your notes describe. Pin v1.0.0 and rebuild.\n\n**6. The model fails to load or the process runs out of memory.** The file does not fit in VRAM with its cache. Lower `--ngl` to offload fewer layers, pick a smaller quantization, or use a smaller model. Remember that a partial offload makes the result a CPU and GPU mix, so label it clearly and never compare it to a full-offload run.\n\n**7. nvidia-smi fails after a driver change.** NVIDIA documents that moving between its closed and open kernel module variants can cause nvidia-smi to fail in some configurations. Reboot, then reinstall the driver cleanly using one variant consistently. If the error message mentions that the GPU has fallen off the bus, suspect power delivery, a loose PCIe connection or an unstable overclock, and test with all tuning reset to defaults.\n\n**8. Scores swing by more than 5 percent between rounds.** That is almost always thermal or background load. Check the clock minimum and the maximum temperature in the summary, lengthen the cooldown to 150 seconds, close everything on the taskbar, and rerun. Also watch for antivirus scans, search indexing and cloud sync, all of which wake up at random.\n\n**9. Superposition or 3DMark crashes, freezes or shows a black screen.** Confirm the driver is stable on its own by running a normal game for ten minutes. Remove any unsaved overclock or undervolt, since graphics tests are good at finding unstable settings. Then update or reinstall the benchmark, and run it in fullscreen at the preset resolution rather than windowed.\n\n**10. Python reports ModuleNotFoundError for telemetry or report.** The scripts import each other, so they must sit in the same folder, and you must launch them from that folder. Check that `telemetry.py` and `report.py` are beside `runbench.py` and not in a subfolder, and that the virtual environment is active.\n\nTwo more checks solve a surprising share of cases. Look at the exact command printed in the terminal history, because a typo in a path produces confusing downstream errors. And confirm the clock on the machine, since the telemetry timestamps come from nvidia-smi and the folder names come from Python, so a wrong system clock makes runs hard to line up.\n\n## Advanced Tips for Serious GPU Benchmark Software Testing\n\nOnce the baseline works, a few extensions turn the harness into a proper testing bench.\n\n**Sweep context length.** llama-bench accepts comma-separated lists, so `-p 512,2048,8192` measures prompt processing at several sizes in one invocation. Long prompts show where the card slows as the cache grows, which matters for coding assistants and document work. The `rate_table` function already names each row by its prompt size, so the report handles the extra rows without edits.\n\n**Sweep power limits.** Setting a lower board power limit with `nvidia-smi -pl` (needs administrator rights) and rerunning the harness at several limits maps the efficiency curve of your card. Many cards give up a small share of speed for a much larger power saving. Use labels such as `pl-80` and `pl-70` so compare.py output stays readable. Reset the limit afterward.\n\n**Test flash attention and batch size.** Flags such as `-fa 1` and `-b` change both speed and memory use. Keep one flag change per run, same as with hardware changes.\n\n**Log throttle reasons.** Add the clocks event reasons field to the query list when your driver supports it (older drivers call it clocks_throttle_reasons.active) or read `nvidia-smi -q -d PERFORMANCE` during a long run. A power cap and a thermal cap call for different fixes, and the log will say which one is active.\n\n**Swap the logger for non-NVIDIA hardware.** On AMD cards under Linux, the ROCm tooling offers equivalent sensor queries. Intel Arc cards have their own monitoring utilities. On Apple Silicon, the built-in powermetrics tool reports GPU and package power. Only `telemetry.py` and the field names in `report.py` need to change, because llama-bench and the comparison logic are vendor neutral. Third-party Windows ROCm packages of llama.cpp reportedly appeared on November 14, 2026, which would give AMD owners another route, though those are downstream builds and not official releases.\n\n**Run it on a schedule.** A scheduled task that runs the harness weekly with the label `weekly` builds a history of how your card ages, how driver updates shift speed, and when dust or dried thermal paste starts to hurt. Plot the median rates and power from the summary files in any spreadsheet.\n\n**Use the CPU as a control.** Run the same model on CPU only with `-ngl 0` once. It confirms the GPU delivers a real speedup, and it flags a system where the GPU path is misconfigured.\n\n## Complete Project Layout and One-Command Run\n\nThe finished harness is five Python files and a results folder. Nothing outside the standard library is required.\n\n```\ngpubench/\n  sysinfo.py      system inventory (step 4)\n  telemetry.py    nvidia-smi sampler (step 5)\n  report.py       summaries and rate tables (step 10)\n  runbench.py     orchestrator (step 9)\n  compare.py      two-run comparison (step 11)\n  runs/           one subfolder per run, created automatically\n  models/         your GGUF file\n```\n\nA complete session looks like this. Do the graphics tests first, then the AI run, so the scores and the harness result share a label:\n\n```\npython sysinfo.py\npython runbench.py --label baseline --model models/model-8b-q4_k_m.gguf --bench-bin llama.cpp/build/bin/llama-bench --score steel_nomad=VALUE --score superposition_1080p_extreme=VALUE\npython runbench.py --label change-one --model models/model-8b-q4_k_m.gguf --bench-bin llama.cpp/build/bin/llama-bench\npython compare.py runs/baseline-STAMP runs/change-one-STAMP\n```\n\nReplace `STAMP` with the timestamp shown in the folder names. If you keep the same label for a series, sort the folders by name to find the latest. Every file written is plain text, so the results open in any editor, import into any spreadsheet and diff cleanly in version control.\n\nKeep the testing loop tight. Change one variable, rerun with a new label, compare against the baseline, and only keep changes that clear the noise floor. After a month you will hold a small dataset about your own hardware, which is worth more than any single score on a public leaderboard.\n\n## FAQ\n\n### What is the best GPU benchmark software for a beginner?\n\nStart with one graphics benchmark you can finish in a few minutes, such as Unigine Superposition or 3DMark Steel Nomad, then add llama-bench if you run local AI models. Pair either with nvidia-smi logging so you can see power, clocks and temperature. Using the same tools the same way each time matters more than which brand you pick.\n\n### How many times should I run a GPU benchmark?\n\nRun each test at least three times and treat the first run as a warm-up. Take the median, and keep the spread between runs. If the spread is larger than the difference you are trying to measure, the result is inside noise and you need more runs or better-controlled conditions.\n\n### Does a higher 3DMark score mean faster local AI?\n\nNot reliably. Graphics scores track rendering performance. Local language model speed depends on memory capacity, memory bandwidth and compute throughput in ways a graphics score does not isolate. Use llama-bench for AI decisions and a graphics tool for gaming decisions.\n\n### Why does llama-bench report two different speeds?\n\nThe first, pp512, measures prompt processing, which is compute-heavy. The second, tg128, measures token generation, which mostly streams model weights through memory. A card can lead one and trail the other, so read them separately.\n\n### Is nvidia-smi accurate enough for benchmarking?\n\nIt is good for trends and comparisons on the same machine, and NVIDIA’s documentation recommends it for testing, monitoring and benchmarking. Sampling is periodic, so very short spikes can slip between samples. Use the same interval for every run and compare like with like.\n\n### Can I use this harness on an AMD or Intel GPU?\n\nYes, with changes. Build llama.cpp with the Vulkan backend, run llama-bench the same way, and replace telemetry.py with the vendor’s monitoring command. The orchestration, report and comparison scripts stay the same once the CSV columns match.\n\n### Should I benchmark with the latest llama.cpp build or the stable release?\n\nUse the stable v1.0.0 release for comparisons, and upgrade only when you want to measure the new version itself. Numbered development builds change daily, which makes cross-run comparisons unreliable. When you do upgrade, rerun your baseline on the new version before comparing anything else.\n\n### How long should I wait between runs?\n\nNinety seconds works for most desktop cards. Extend it to two or three minutes in a warm room or on a small case. If round-to-round spread stays high after you lengthen the cooldown, look for background load rather than adding more time.", "url": "https://wpnews.pro/news/gpu-benchmark-software-12-steps-80-min-test-plan", "canonical_source": "https://shattered.io/gpu-benchmark-software-3dmark-superposition-llama-cpp-12-steps/", "published_at": "2026-11-18 09:19:25+00:00", "updated_at": "2026-11-18 09:44:43.599471+00:00", "lang": "en", "topics": ["large-language-models", "ai-infrastructure", "ai-tools", "developer-tools"], "entities": ["llama.cpp", "llama-bench", "nvidia-smi", "NVIDIA", "3DMark", "UL Solutions", "Unigine Superposition", "SmartScope"], "also_reported_by": [], "takeaway": "Run llama-bench's pp512 and tg128 separately alongside nvidia-smi telemetry, since prompt processing is compute-bound while token generation depends on memory bandwidth, so one score cannot explain a card's performance.", "recommendation": {"advice": "Benchmark local models with llama-bench's pp512 and tg128 rates and log power, clocks and temperature via nvidia-smi during the run, so a score can be traced to compute or memory-bandwidth limits.", "playbook": "inference-serving", "playbook_url": "https://wpnews.pro/playbooks/inference-serving"}, "alternates": {"html": "https://wpnews.pro/news/gpu-benchmark-software-12-steps-80-min-test-plan", "markdown": "https://wpnews.pro/news/gpu-benchmark-software-12-steps-80-min-test-plan.md", "text": "https://wpnews.pro/news/gpu-benchmark-software-12-steps-80-min-test-plan.txt", "jsonld": "https://wpnews.pro/news/gpu-benchmark-software-12-steps-80-min-test-plan.jsonld"}}