GPU Benchmark Software: 12 Steps, 80-Min Test Plan Llama.cpp reached version 1.0.0 on November 13, 2026, its first major stable release, and the project now follows semantic-versioning rules for its public API, according to a SmartScope write-up of the 1.0 changes. The release gives GPU testers a stable local-AI throughput yardstick via llama-bench, which reports prompt processing (pp512) and token generation (tg128) in tokens per second alongside graphics benchmarks such as 3DMark and Unigine Superposition and nvidia-smi telemetry. A 12-step test plan built on those three free tools is estimated at about 80 minutes for a first pass and under 20 minutes per re-test. llama.cpp reached version 1.0.0 on November 13, 2026, its first major stable release, and the project now follows semantic-versioning rules for its public API, according to a SmartScope write-up of the 1.0 changes https://smartscope.blog/en/blog/llama-cpp-1-0-changed-api-abi-handling-2026/ . That matters for anyone who tests graphics cards. Local AI workloads finally have a stable yardstick that sits next to the older graphics tests, and a fair run of GPU benchmark software in late 2026 needs both. Most people grab one tool, run it once and post a score. That number tells you very little. A single score hides the clock speed, power draw and temperature behind it, and it cannot show whether a driver update helped or the card simply ran cooler that afternoon. The plan below fixes that with 12 steps, a few hundred lines of Python and three free building blocks: a graphics benchmark, the llama-bench tool for AI throughput, and nvidia-smi for telemetry. Expect about 80 minutes for the first pass and under 20 minutes for every re-test after that. If you are still choosing a card for local models, the Nvidia vs AMD GPU for local AI comparison https://shattered.io/nvidia-vs-amd-gpu-local-ai-32gb-vs-24gb-2026/ covers the memory and price trade-offs. The steps here assume the GPU is already in the machine and you want proof of what it does. Don't miss new tech stories on Google Add Shattered once in the Google app and our stories show up in your news suggestions. Add now https://www.google.com/preferences/source?hl=en&q=https%3A%2F%2Fshattered.io%2F What GPU Benchmark Software Measures and What It Misses No single program answers “is my GPU fast?” Each tool isolates one slice of behavior, and the slices do not predict each other well. A card that tops a gaming score can fall behind in token generation if its memory bandwidth is lower. A card with huge memory can post a modest graphics score and still run larger models than anything else in its class. Graphics benchmarks such as 3DMark and Unigine Superposition render a fixed scene and report a score. They are repeatable, they are widely shared, and they let you compare your result against public databases. UL Solutions runs a public results service for 3DMark, and its GPU ranking pages https://benchmarks.ul.com/en/compare/best-gpus?amount=0&sortBy=VALUE FOR MONEY&reverseOrder=true&types=DESKTOP&minRating=0&search=nvidia list hardware by score, which gives you a sanity check on your own number. What these tools do not do is tell you anything about language-model speed. Based on the 3DMark pages available today, UL presents the suite as a graphics and processor benchmark, not an inference test. For AI work, llama-bench ships with llama.cpp and reports two rates in tokens per second. Prompt processing often written pp512 for a 512-token prompt is compute-heavy. Token generation tg128 for 128 generated tokens leans on memory bandwidth. Reading both numbers separately tells you which resource limits the card. Then there is telemetry. NVIDIA’s own documentation for the System Management Interface recommends nvidia-smi for testing, monitoring and benchmarking. It does not render anything and it does not run a model. It records utilization, memory, temperature, power and clocks while another workload runs, which is exactly what turns a bare score into an explanation. | Tool | What it measures | Output | Best use | |---|---|---|---| | 3DMark Steel Nomad and CPU Profile | Graphics and processor performance in a fixed scene | A score plus public ranking | Comparing against other owners of the same card | | Unigine Superposition | Sustained GPU rendering load | A score and frame rate for a chosen preset | Stability and thermal soak runs | | llama-bench llama.cpp | Local AI inference speed | Tokens per second for prompt and generation | Judging a card for local models | | nvidia-smi | Power, clocks, temperature, VRAM, utilization | Sampled CSV rows | Explaining why a score moved | | A game’s built-in benchmark | Real engine behavior with your settings | Average and low frame rates | Final check on the games you actually play | The takeaway: pick one graphics tool, one AI tool and one logger, then use them the same way every time. Consistency beats variety. Readers who want the wider buying picture can find it in the AMD vs Nvidia GPU buyer’s guide for AI https://shattered.io/amd-vs-nvidia-gpu-ai-buyers-guide-97-percent-growth-vs-30x-claim-2026/ , and the hardware pillar page collects more testing guides under Hardware and Chips coverage https://shattered.io/category/hardware/ . Prerequisites and Versions for the GPU Benchmark Software Setup The harness uses an NVIDIA card because nvidia-smi supplies the telemetry. The graphics tools and llama.cpp also run on AMD and Intel hardware, and the advanced section covers the telemetry swaps. Everything else below is free to download, with one exception covered in step 6. | Requirement | Version or spec | Why it is needed | |---|---|---| | GPU | Any NVIDIA GeForce RTX card the GeForce RTX 5090 is the current desktop flagship on NVIDIA’s lineup page | Runs the workloads | | Operating system | Windows 11 64-bit or Ubuntu 24.04 LTS | Both work, commands for each are shown | | NVIDIA driver | The current Game Ready or Studio driver for your card | Provides nvidia-smi and CUDA support | | Python | 3.11 or newer | Runs the harness | | llama.cpp | v1.0.0 stable release | Provides llama-bench | | CMake | 3.20 or newer | Builds llama.cpp from source | | Compiler | Visual Studio 2022 Build Tools Windows or GCC Linux | Builds llama.cpp from source | | CUDA Toolkit | A release your driver supports | Enables the CUDA backend | | Disk space | About 20 GB free | Source tree, build output and a model file | | Model file | One GGUF model in Q4 K M quantization, 7B to 8B parameters | The AI workload | Two version notes save trouble later. First, llama.cpp also publishes numbered development builds. A pre-release labeled b12445 appeared on November 16, 2026, and the project’s own materials treat those as development snapshots, not replacements for the stable v1.0.0 label. Pin the stable tag for benchmarking so every run uses the same code. Second, the 1.0 release removed a set of deprecated public API functions, so scripts written against pre-1.0 nightly builds may need changes. The harness below only calls the command-line tools, which keeps it insulated from library API changes. You also need a model that fits entirely in VRAM. A 7B or 8B model at Q4 K M typically needs a few gigabytes, so it fits on most cards sold in the last several years. Choose a model you would actually run, because a benchmark on a model you never use is trivia. Keep the exact filename in your notes, since swapping the quantization changes the numbers more than most hardware changes do. Steps 1 to 3: Plan the Tests, Prepare the PC, Scaffold the Project Step 1: Define the test plan before you touch a tool Write the plan first. It takes five minutes and prevents the most common failure in GPU testing, which is changing three things at once and then not knowing what caused the result. Decide which questions the benchmark must answer. Typical ones are whether a new driver changed anything, whether an undervolt cost performance, whether a power limit saves energy at an acceptable speed cost, or whether a used card behaves like its listing. | Workload | Tool | Metric | Repeats | |---|---|---|---| | Graphics, short | 3DMark Steel Nomad | Score | 3 | | Graphics, sustained | Superposition 1080p Extreme | Score and average frame rate | 3 | | Prompt processing | llama-bench pp512 | Tokens per second | 3 rounds of 5 reps | | Token generation | llama-bench tg128 | Tokens per second | 3 rounds of 5 reps | | Telemetry | nvidia-smi | Power, temperature, clocks, VRAM | Whole session | Name each run with a label that states the one variable you changed, such as baseline , driver-update or pl-80 . The label becomes the folder name later, so keep it short and free of spaces. Step 2: Prepare the PC for repeatable runs Repeatability comes from the machine state, not the software. Close browsers, launchers, chat apps and overlay tools. Set the OS power mode to its high-performance option. On a laptop, plug in the charger, because battery operation can cut GPU power limits sharply. Let the system idle for five minutes after boot so background indexing and update checks finish. Install or confirm your driver before the first run and do not change it mid-session. Reboot once after any driver install. Then check that the GPU is visible: nvidia-smi You should see a table listing the driver version, the GPU name, memory, temperature and power. If the command fails, jump to the troubleshooting list. Write down the room conditions too. A card that boosts at 21 degrees Celsius ambient will boost lower at 30, and you cannot compare runs across that gap without noting it. Step 3: Scaffold the project folder and Python environment Create a folder for the harness and a virtual environment. The scripts use only the Python standard library, so there is nothing to install beyond Python itself. mkdir gpubench cd gpubench mkdir runs python -m venv .venv On Windows, activate the environment with .venv\Scripts\Activate.ps1 in PowerShell. On Linux, run source .venv/bin/activate . The finished project holds five scripts and a runs folder, shown in full in the project layout section near the end. Each run writes its own subfolder under runs , so nothing is overwritten and you can diff any two sessions months apart. Steps 4 and 5: Capture a System Inventory and Log GPU Telemetry Step 4: Capture a system inventory A benchmark result without its context is almost worthless a month later. Did you run it on the old driver? Was the PCIe slot negotiating x16 or x8? Which llama.cpp build produced it? Save sysinfo.py in the project folder. It asks nvidia-smi for the card name, driver, VBIOS, memory size, power limit and PCIe link state, then adds the OS, Python version, power profile and llama.cpp version string. python import json import platform import shutil import subprocess import sys from pathlib import Path GPU FIELDS = "name", "driver version", "vbios version", "memory.total", "power.limit", "pcie.link.gen.current", "pcie.link.width.current", def run cmd : try: out = subprocess.run cmd, capture output=True, text=True, timeout=20 return out.stdout.strip except OSError, subprocess.TimeoutExpired : return "" def gpu info index=0 : if shutil.which "nvidia-smi" is None: return {"error": "nvidia-smi not found"} raw = run "nvidia-smi", f"--id={index}", "--query-gpu=" + ",".join GPU FIELDS , "--format=csv,noheader,nounits" values = v.strip for v in raw.split "," return dict zip GPU FIELDS, values def power profile : if platform.system == "Windows": return run "powercfg", "/getactivescheme" gov = Path "/sys/devices/system/cpu/cpu0/cpufreq/scaling governor" return gov.read text .strip if gov.exists else "unknown" def collect llama cli="llama-cli" : return { "os": platform.platform , "python": sys.version.split 0 , "cpu": platform.processor or platform.machine , "gpu": gpu info , "power profile": power profile , "llama cpp": run llama cli, "--version" .splitlines -1: or "unknown" , } if name == " main ": print json.dumps collect , indent=2 Run it once to confirm it works: python sysinfo.py Expected output the values below show the format, yours will differ : { "os": "Windows-11-10.0.26100-SP0", "python": "3.12.x", "cpu": "Intel64 Family 6 Model 198 Stepping 2, GenuineIntel", "gpu": { "name": "NVIDIA GeForce RTX your card ", "driver version": " your driver ", "vbios version": " your VBIOS ", "memory.total": " MiB ", "power.limit": " W ", "pcie.link.gen.current": "5", "pcie.link.width.current": "16" }, "power profile": "Power Scheme GUID: ... High performance ", "llama cpp": "unknown" } The llama cpp field reads unknown until you build llama.cpp in step 7. Idle GPUs often report a lower PCIe generation to save power, so a Gen 1 reading at rest is normal. Read the link state again under load if you suspect a slot problem. Step 5: Build the nvidia-smi telemetry logger The logger starts nvidia-smi in loop mode and writes one CSV row every 500 milliseconds. It reads the child process output through a pipe and flushes each row to disk, which avoids the buffering surprise you get when you redirect nvidia-smi straight to a file. Fields that a card does not report, such as fan speed on some laptops, arrive as N/A and the report script skips them. python import csv import shutil import subprocess import threading from pathlib import Path FIELDS = "timestamp", "power.draw", "temperature.gpu", "utilization.gpu", "memory.used", "clocks.sm", "clocks.mem", "fan.speed", class Telemetry: def init self, out csv, interval ms=500, gpu index=0 : self.out csv = Path out csv self.interval ms = interval ms self.gpu index = gpu index self.proc = None self.thread = None def pump self : with open self.out csv, "w", newline="" as fh: writer = csv.writer fh writer.writerow FIELDS for line in self.proc.stdout: row = next csv.reader line , skipinitialspace=True , None if row and len row == len FIELDS : writer.writerow row fh.flush def start self : if shutil.which "nvidia-smi" is None: raise SystemExit "nvidia-smi is not on PATH" cmd = "nvidia-smi", f"--id={self.gpu index}", "--query-gpu=" + ",".join FIELDS , "--format=csv,noheader,nounits", f"--loop-ms={self.interval ms}", self.proc = subprocess.Popen cmd, stdout=subprocess.PIPE, stderr=subprocess.DEVNULL, text=True self.thread = threading.Thread target=self. pump, daemon=True self.thread.start def stop self : if self.proc: self.proc.terminate self.proc.wait timeout=10 if self.thread: self.thread.join timeout=10 def enter self : self.start return self def exit self, exc : self.stop Test it for ten seconds while the GPU does something, such as playing a video or running a game menu: python python -c "import time; from telemetry import Telemetry; t = Telemetry 'test.csv' ; t.start ; time.sleep 10 ; t.stop " Then open test.csv . Healthy output looks like this: timestamp,power.draw,temperature.gpu,utilization.gpu,memory.used,clocks.sm,clocks.mem,fan.speed 2026/11/18 10:02:41.318,38.12,41,3,1210,375,405,30 2026/11/18 10:02:41.820,38.40,41,4,1210,375,405,30 A 500 ms interval is a good default. Faster sampling adds load to the driver and rarely changes the conclusion, while slower sampling can miss short throttling dips. The sample values above are format examples from an idle card. Step 6: Run the Graphics Benchmarks 3DMark and Superposition Telemetry should run during the graphics tests too, so start the logger first in a second terminal and stop it after the last test finishes: python python -c "import time; from telemetry import Telemetry; t = Telemetry 'runs/gfx-baseline.csv' ; t.start ; input 'Logging. Press Enter when the graphics tests are done: ' ; t.stop " Run each graphics test three times with a 90 second pause between runs. The pause lets temperatures settle so the third run starts from conditions close to the first. For 3DMark, use the Steel Nomad test. UL Solutions reportedly sells 3DMark through Steam at $34.99, so confirm the current price and edition on the store page before buying. Run the test at default settings, wait for the result screen and write down the overall score, the graphics score and the frame-rate figure. Do not change the resolution or turn on custom settings if you want the result to be comparable with the public results database. For Unigine Superposition, pick the 1080p Extreme preset. Its public results pages still label runs as Superposition 1.1 as of late October 2026, so that is the newest version you can confirm, though UNIGINE’s separate engine and SDK have moved on to version 2.22. Check the download page for the current license terms before you rely on it. The Extreme preset loads the card heavily for several minutes, which makes it a decent thermal soak test as well as a score generator. Record the final score and the average frames per second. Treat the first run of a session as a warm-up. The card starts cold, boost clocks are at their peak, and the score is slightly higher than the sustained figure. If the first score is the best of the three by a wide margin, that gap is a measure of how much your cooling limits sustained performance. That is useful information, not a flaw in the test. Store the scores in a text file or pass them to the harness later with the --score option. A line such as steel nomad=value in the final command saves them into the run record. Steps 7 and 8: Build llama.cpp v1.0.0 and Run llama-bench by Hand Step 7: Build llama.cpp and get a model Clone the repository, switch to the stable release, and build with the CUDA backend. On Linux: git clone https://github.com/ggml-org/llama.cpp cd llama.cpp git fetch --tags git tag --list "v1 " git checkout v1.0.0 cmake -B build -DGGML CUDA=ON cmake --build build --config Release -j The tag --list line is there because release tags occasionally differ from the headline version, so confirm that v1.0.0 exists before checking it out. On Windows, run the same commands from a Developer PowerShell for Visual Studio 2022. The binaries land in build\bin\Release on Windows and build/bin on Linux. Prebuilt release archives also exist for Windows, and that is a faster route if you do not want to install a compiler. Whichever route you pick, use the stable v1.0.0 tag and avoid the numbered pre-release builds, which the project’s CI notes on Hugging Face https://huggingface.co/blog/ggml-org/llama-cpp-ci-buckets describe as automated outputs of a busy daily pipeline. For AMD or Intel graphics, swap the CUDA flag for the Vulkan backend with -DGGML VULKAN=ON . The project has kept adding Vulkan work in recent builds, and the Vulkan path is the practical cross-vendor option. Download one GGUF model file from its Hugging Face page and place it in a models folder beside the project. Pick a 7B to 8B model in Q4 K M. Record the exact file name. The iacompa report on the 1.0 release https://iacompa.com/news/ggml-llama-cpp-1-0-local-inference lists the API removals if you maintain code that links against libllama, though the command-line workflow here is not affected. Confirm the build works by asking for the version: ./build/bin/llama-cli --version Step 8: Run llama-bench by hand first Never automate a command you have not seen work manually. Run llama-bench directly against your model, with all layers offloaded to the GPU, a 512-token prompt, 128 generated tokens and five repetitions: ./build/bin/llama-bench -m ../models/model-8b-q4 k m.gguf -ngl 99 -p 512 -n 128 -r 5 The default output is a Markdown table. The numbers below are placeholders that show the layout, not a measured result: | model | size | params | backend | ngl | test | t/s | | ------------------ | -----: | -----: | ------- | --: | ----: | ----------: | | model 8B Q4 K M | x GiB | 8 B | CUDA | 99 | pp512 | NNNN +- NN | | model 8B Q4 K M | x GiB | 8 B | CUDA | 99 | tg128 | NNN +- N | Check three things in that table. The backend column must say CUDA or Vulkan if you built that , because CPU means the GPU was not used and every number is meaningless for your purpose. The ngl column should show 99 or the layer count you asked for. And the plus-minus figure after each rate shows the run-to-run spread inside the five repetitions. A spread above a few percent usually means something else is using the GPU or the card is throttling. Why two rows? Prompt processing is a large batched calculation that stresses compute. Token generation produces one token at a time and mostly streams the model weights through memory, so it tracks memory bandwidth. When you compare two cards, the pp512 ratio often differs from the tg128 ratio, and that gap tells you which spec decides the matchup for your workload. Step 9: Automate the Run With a Python Harness The manual run proves the tools work. The harness makes the result repeatable. It creates a timestamped folder, saves the system inventory, runs one unrecorded warm-up pass, then starts the telemetry logger and runs the benchmark in several rounds with a cooldown between them. Each round holds its own set of repetitions, so the final summary can report both the median and the spread across rounds. That spread is the noise floor you will use in step 11 to decide whether a change was real. The script shells out to llama-bench with JSON output and slices the text from the first opening bracket, which protects the parser if the tool prints a log line before the JSON. Save it as runbench.py : python import argparse import json import subprocess import time from datetime import datetime from pathlib import Path import report import sysinfo from telemetry import Telemetry def llama bench binary, model, ngl, prompt, gen, reps : cmd = binary, "-m", model, "-ngl", str ngl , "-p", str prompt , "-n", str gen , "-r", str reps , "-o", "json" out = subprocess.run cmd, capture output=True, text=True, check=True text = out.stdout return json.loads text text.index " " : def main : ap = argparse.ArgumentParser ap.add argument "--label", required=True ap.add argument "--model", required=True ap.add argument "--bench-bin", default="llama-bench" ap.add argument "--ngl", type=int, default=99 ap.add argument "--prompt", type=int, default=512 ap.add argument "--gen", type=int, default=128 ap.add argument "--reps", type=int, default=5 ap.add argument "--rounds", type=int, default=3 ap.add argument "--cooldown", type=int, default=90 ap.add argument "--score", action="append", default= , help="manual score, NAME=VALUE repeatable " args = ap.parse args stamp = datetime.now .strftime "%Y%m%d-%H%M%S" run dir = Path "runs" / f"{args.label}-{stamp}" run dir.mkdir parents=True run dir / "sysinfo.json" .write text json.dumps sysinfo.collect , indent=2 print "warm-up pass not recorded " llama bench args.bench bin, args.model, args.ngl, args.prompt, args.gen, 1 rounds = with Telemetry run dir / "telemetry.csv" : for i in range 1, args.rounds + 1 : print f"round {i} of {args.rounds}" result = llama bench args.bench bin, args.model, args.ngl, args.prompt, args.gen, args.reps rounds.append {"round": i, "llama": result} if i = args.rounds: time.sleep args.cooldown scores = dict item.split "=", 1 for item in args.score raw = {"label": args.label, "args": vars args , "rounds": rounds, "scores": scores} run dir / "raw.json" .write text json.dumps raw, indent=2 print json.dumps report.build summary run dir , indent=2 if name == " main ": main Run the baseline with the same parameters you will use for every later test. Pass the graphics scores you wrote down in step 6 so they travel with the run: python runbench.py --label baseline --model ../models/model-8b-q4 k m.gguf --bench-bin ../llama.cpp/build/bin/llama-bench --score steel nomad=VALUE --score superposition 1080p extreme=VALUE Replace VALUE with the numbers from your result screens. With three rounds, five repetitions and a 90 second cooldown, the AI portion takes roughly 8 to 10 minutes on a typical card, mostly spent in cooldowns. The terminal prints progress, then the summary once the run finishes: warm-up pass not recorded round 1 of 3 round 2 of 3 round 3 of 3 { "label": "baseline", "rates": { "pp512": { "median tps": NNNN.NN, "spread pct": 0.6, "rounds": 3, "tokens per joule": N.NNNN }, "tg128": { "median tps": NNN.NN, "spread pct": 1.5, "rounds": 3, "tokens per joule": N.NNNN } }, "scores": { "steel nomad": "VALUE", "superposition 1080p extreme": "VALUE" }, "telemetry": { "load samples": NNN, "power avg w": NNN.N, "temp max c": NN.N, "sm clock avg mhz": NNNN, "vram peak mib": NNNN } } The placeholders stand in for your own numbers. The structure is what matters here. Every run ends with the same fields, so a spreadsheet, a script or a person can read any run the same way. Steps 10 to 12: Read the Results, Compare Runs, Archive Everything Step 10: Read the results correctly The summary logic lives in report.py , which runbench.py already imports. It does three things worth understanding. It filters telemetry to rows where GPU utilization is at least 50 percent, so idle time before and after the test does not drag the averages down. It takes the median of each rate across rounds instead of the mean, which keeps one unlucky round from skewing the answer. And it divides tokens per second by average power draw to give tokens per joule, a rough efficiency measure that makes undervolts and power limits easy to judge. python import csv import json import statistics import sys from pathlib import Path def num value : try: return float value except TypeError, ValueError : return None def summarize csv path, min util=50.0 : rows = with open csv path, newline="" as fh: for row in csv.DictReader fh : util = num row "utilization.gpu" if util is not None and util = min util: rows.append row if not rows: return {"load samples": 0} def col name : return v for v in num r name for r in rows if v is not None power, temp = col "power.draw" , col "temperature.gpu" clock, mem = col "clocks.sm" , col "memory.used" return { "load samples": len rows , "power avg w": round statistics.mean power , 1 if power else None, "power max w": max power if power else None, "temp avg c": round statistics.mean temp , 1 if temp else None, "temp max c": max temp if temp else None, "sm clock avg mhz": round statistics.mean clock if clock else None, "sm clock min mhz": min clock if clock else None, "vram peak mib": max mem if mem else None, } def rate table rounds : table = {} for entry in rounds: for item in entry "llama" : kind = "pp" if item "n prompt" and not item "n gen" else "tg" size = item "n prompt" or item "n gen" table.setdefault f"{kind}{size}", .append item "avg ts" result = {} for name, values in table.items : med = statistics.median values spread = max values - min values / med 100 if med else 0.0 result name = {"median tps": round med, 2 , "spread pct": round spread, 2 , "rounds": len values } return result def build summary run dir : run dir = Path run dir data = json.loads run dir / "raw.json" .read text summary = { "label": data "label" , "rates": rate table data "rounds" , "scores": data.get "scores", {} , "telemetry": summarize run dir / "telemetry.csv" , } power = summary "telemetry" .get "power avg w" if power: for name, rate in summary "rates" .items : rate "tokens per joule" = round rate "median tps" / power, 4 run dir / "summary.json" .write text json.dumps summary, indent=2 return summary if name == " main ": print json.dumps build summary sys.argv 1 , indent=2 You can rebuild a summary for any saved run, which is handy after you adjust the filter: python report.py runs/baseline-20261118-101500 Use this table to read what you see: | What you see | Likely meaning | What to do | |---|---|---| | Spread across rounds under about 2% | Stable test conditions | Trust small differences only if they exceed the spread | | Round 1 much faster than rounds 2 and 3 | Thermal or power throttling as the card heats up | Check temp max c and the sm clock min mhz value, improve cooling or lengthen cooldowns | | sm clock min mhz far below sm clock avg mhz | Clock drops during the test | Look at power draw against the power limit and at temperature | | vram peak mib near the card’s total memory | Model plus cache almost fills VRAM | Use a smaller quantization or context, or expect slowdowns | | tg128 low but pp512 normal | Memory-bandwidth limit | Compare memory clock and PCIe state, then check for VRAM spill | | Power average near the card’s power limit | Power-limited rather than temperature-limited | A higher limit might raise speed, at an efficiency cost | Step 11: Compare two runs after one change Now change exactly one thing. Update the driver, lower the power limit, apply an undervolt, switch a cooling profile or move the card to a different slot. Rerun the harness with a new label and the same arguments: python runbench.py --label driver-update --model ../models/model-8b-q4 k m.gguf --bench-bin ../llama.cpp/build/bin/llama-bench Then compare the two folders with compare.py , which uses the larger of the two spreads as the noise threshold: python import json import sys from pathlib import Path def load path : return json.loads Path path / "summary.json" .read text def pct new, old : return new - old / old 100 if old else 0.0 def main base dir, test dir : base, test = load base dir , load test dir print f"{'metric':<12}{'base': 10}{'test': 10}{'delta %': 10}{'noise %': 10} verdict" for name, b in base "rates" .items : t = test "rates" .get name if not t: continue delta = pct t "median tps" , b "median tps" noise = max b "spread pct" , t "spread pct" verdict = "real" if abs delta noise else "inside noise" print f"{name:<12}{b 'median tps' : 10}{t 'median tps' : 10}" f"{delta: 10.1f}{noise: 10.1f} {verdict}" bp = base "telemetry" .get "power avg w" tp = test "telemetry" .get "power avg w" if bp and tp: print f"{'power W':<12}{bp: 10}{tp: 10}{pct tp, bp : 10.1f}" if name == " main ": main sys.argv 1 , sys.argv 2 python compare.py runs/baseline-20261118-101500 runs/driver-update-20261118-113000 The comparison prints a table like this one example numbers from synthetic test data, not measurements from a real card : metric base test delta % noise % verdict pp512 3011.34 3130.67 4.0 0.6 real tg128 100.46 100.88 0.4 1.5 inside noise power W 300.1 280.1 -6.7 Read that as a story. Prompt processing rose by more than the noise, so the change helped compute-bound work. Token generation moved less than the noise, so call it unchanged. Average power fell. A change that keeps speed and trims power is a clear win. The same tool will show you the opposite case just as plainly, a change that looks good in one score and quietly hurts another. A single-digit gain on one test with a 3 percent spread is not evidence. Rerun both sides before you believe it. Treat the verdict column as a filter for what deserves a second look, not as a statistical proof. Step 12: Archive and share the results Each run folder already holds the raw JSON, the summary, the telemetry CSV and the system inventory. Copy the whole runs folder to cloud storage or a Git repository after each session. Telemetry files are small, usually well under a megabyte per run, so size is not a concern. Add a one-line note to each folder’s name or a notes.txt that records room temperature and anything unusual, like a Windows update that landed during the session. When you post results to a forum, include the card, driver version, llama.cpp version, model file with its quantization, the exact llama-bench arguments and the ambient temperature. Posting a bare token rate invites arguments that these fields would settle in a sentence. Small teams that need a shared record of workstation performance can store the folders next to the rest of their business software stack https://shattered.io/small-business-software-stack-2026/ documentation, which keeps hardware decisions auditable when someone asks why a purchase was approved. Common Pitfalls That Ruin GPU Benchmark Results Experienced testers still repeat the same handful of mistakes. Each one produces numbers that look fine and are wrong. Comparing different quantizations or models. A Q4 K M file and a Q8 0 file of the same model differ in size and speed by a wide margin. If you swap files between runs, you are measuring the file. Pin the model name in your notes and in the label. Testing on battery or a balanced power plan. Laptops cut GPU power on battery, and desktop power plans can cap CPU boost, which affects prompt processing and the benchmark launcher. Plug in and use the high-performance mode for every run. Reading only the first run. A cold card boosts higher than a warm one. If you publish the first score of the day, you publish the best case. The harness uses a warm-up pass and rounds with cooldowns for this reason. Leaving overlays and background apps running. Game launchers, recording software, RGB control suites and browser tabs steal GPU time or CPU cycles. A background process using a few percent of the GPU is enough to put a spread of 5 percent into your data. Changing the driver mid-comparison. Driver updates can change both performance and clocks. If a driver auto-installs between your baseline and your test, you have two changes, not one. Turn off automatic driver updates for the duration of a test series. Letting the model spill out of VRAM. If the model and its cache do not fit, layers move to system memory and token generation collapses. The vram peak mib field in the summary exposes this. Pick a model and context size that fit with margin. Using a development build for one run and the stable build for another. Numbered b-series builds change daily. Mixing them produces a difference that has nothing to do with your hardware. Pin v1.0.0 and note the version string saved in the system inventory. Trusting a score without its telemetry. A higher score at much higher power draw is a different statement from a higher score at the same power. Always read the power and clock lines beside the score. Troubleshooting: 10 Problems and Fixes 1. nvidia-smi is not recognized or not found. On Windows the tool normally lives in the System32 folder and installs with the driver, so a missing command usually means a broken or partial driver install. Reinstall the driver with the clean-install option and reboot. Inside WSL, the tool can be absent from the normal path, and NVIDIA’s NemoClaw release notes from November 6 https://docs.nvidia.com/nemoclaw/latest/user-guide/deepagents/release-notes/2026/11/6 describe tooling that reportedly searches for the WSL-provided copy for that reason. Call it by its full path if needed. 2. The telemetry CSV has a header and no rows. The logger drops any line that does not split into exactly eight fields. Run nvidia-smi --query-gpu=power.draw,fan.speed --format=csv by hand to see what the card returns. Some cards reject a field name, and the output then contains an error instead of values. Remove the unsupported field from FIELDS in both the logger and the report. 3. power.draw or fan.speed shows N/A . Laptop GPUs and some workstation boards do not expose every sensor. The report script skips values it cannot convert, so the run still works. If power is missing entirely, the tokens-per-joule line will not appear and you should lean on clock and temperature data instead. 4. llama-bench says the backend is CPU. The build did not include the GPU backend, or the CUDA toolkit was not found during configuration. Delete the build folder, confirm the toolkit is installed and visible, rerun the configure step with -DGGML CUDA=ON and read the configure output for a CUDA detection line. For AMD and Intel cards, build with the Vulkan flag instead. 5. The harness crashes with a JSON error. The parser slices from the first opening bracket, but a mismatched llama-bench binary or a crash inside it can leave nothing valid to parse. Run the same command by hand and read the raw output. If the tool prints an error about an unknown argument, you are probably using an older or newer build than the one your notes describe. Pin v1.0.0 and rebuild. 6. The model fails to load or the process runs out of memory. The file does not fit in VRAM with its cache. Lower --ngl to offload fewer layers, pick a smaller quantization, or use a smaller model. Remember that a partial offload makes the result a CPU and GPU mix, so label it clearly and never compare it to a full-offload run. 7. nvidia-smi fails after a driver change. NVIDIA documents that moving between its closed and open kernel module variants can cause nvidia-smi to fail in some configurations. Reboot, then reinstall the driver cleanly using one variant consistently. If the error message mentions that the GPU has fallen off the bus, suspect power delivery, a loose PCIe connection or an unstable overclock, and test with all tuning reset to defaults. 8. Scores swing by more than 5 percent between rounds. That is almost always thermal or background load. Check the clock minimum and the maximum temperature in the summary, lengthen the cooldown to 150 seconds, close everything on the taskbar, and rerun. Also watch for antivirus scans, search indexing and cloud sync, all of which wake up at random. 9. Superposition or 3DMark crashes, freezes or shows a black screen. Confirm the driver is stable on its own by running a normal game for ten minutes. Remove any unsaved overclock or undervolt, since graphics tests are good at finding unstable settings. Then update or reinstall the benchmark, and run it in fullscreen at the preset resolution rather than windowed. 10. Python reports ModuleNotFoundError for telemetry or report. The scripts import each other, so they must sit in the same folder, and you must launch them from that folder. Check that telemetry.py and report.py are beside runbench.py and not in a subfolder, and that the virtual environment is active. Two more checks solve a surprising share of cases. Look at the exact command printed in the terminal history, because a typo in a path produces confusing downstream errors. And confirm the clock on the machine, since the telemetry timestamps come from nvidia-smi and the folder names come from Python, so a wrong system clock makes runs hard to line up. Advanced Tips for Serious GPU Benchmark Software Testing Once the baseline works, a few extensions turn the harness into a proper testing bench. Sweep context length. llama-bench accepts comma-separated lists, so -p 512,2048,8192 measures prompt processing at several sizes in one invocation. Long prompts show where the card slows as the cache grows, which matters for coding assistants and document work. The rate table function already names each row by its prompt size, so the report handles the extra rows without edits. Sweep power limits. Setting a lower board power limit with nvidia-smi -pl needs administrator rights and rerunning the harness at several limits maps the efficiency curve of your card. Many cards give up a small share of speed for a much larger power saving. Use labels such as pl-80 and pl-70 so compare.py output stays readable. Reset the limit afterward. Test flash attention and batch size. Flags such as -fa 1 and -b change both speed and memory use. Keep one flag change per run, same as with hardware changes. Log throttle reasons. Add the clocks event reasons field to the query list when your driver supports it older drivers call it clocks throttle reasons.active or read nvidia-smi -q -d PERFORMANCE during a long run. A power cap and a thermal cap call for different fixes, and the log will say which one is active. Swap the logger for non-NVIDIA hardware. On AMD cards under Linux, the ROCm tooling offers equivalent sensor queries. Intel Arc cards have their own monitoring utilities. On Apple Silicon, the built-in powermetrics tool reports GPU and package power. Only telemetry.py and the field names in report.py need to change, because llama-bench and the comparison logic are vendor neutral. Third-party Windows ROCm packages of llama.cpp reportedly appeared on November 14, 2026, which would give AMD owners another route, though those are downstream builds and not official releases. Run it on a schedule. A scheduled task that runs the harness weekly with the label weekly builds a history of how your card ages, how driver updates shift speed, and when dust or dried thermal paste starts to hurt. Plot the median rates and power from the summary files in any spreadsheet. Use the CPU as a control. Run the same model on CPU only with -ngl 0 once. It confirms the GPU delivers a real speedup, and it flags a system where the GPU path is misconfigured. Complete Project Layout and One-Command Run The finished harness is five Python files and a results folder. Nothing outside the standard library is required. gpubench/ sysinfo.py system inventory step 4 telemetry.py nvidia-smi sampler step 5 report.py summaries and rate tables step 10 runbench.py orchestrator step 9 compare.py two-run comparison step 11 runs/ one subfolder per run, created automatically models/ your GGUF file A complete session looks like this. Do the graphics tests first, then the AI run, so the scores and the harness result share a label: python sysinfo.py python runbench.py --label baseline --model models/model-8b-q4 k m.gguf --bench-bin llama.cpp/build/bin/llama-bench --score steel nomad=VALUE --score superposition 1080p extreme=VALUE python runbench.py --label change-one --model models/model-8b-q4 k m.gguf --bench-bin llama.cpp/build/bin/llama-bench python compare.py runs/baseline-STAMP runs/change-one-STAMP Replace STAMP with the timestamp shown in the folder names. If you keep the same label for a series, sort the folders by name to find the latest. Every file written is plain text, so the results open in any editor, import into any spreadsheet and diff cleanly in version control. Keep the testing loop tight. Change one variable, rerun with a new label, compare against the baseline, and only keep changes that clear the noise floor. After a month you will hold a small dataset about your own hardware, which is worth more than any single score on a public leaderboard. FAQ What is the best GPU benchmark software for a beginner? Start with one graphics benchmark you can finish in a few minutes, such as Unigine Superposition or 3DMark Steel Nomad, then add llama-bench if you run local AI models. Pair either with nvidia-smi logging so you can see power, clocks and temperature. Using the same tools the same way each time matters more than which brand you pick. How many times should I run a GPU benchmark? Run each test at least three times and treat the first run as a warm-up. Take the median, and keep the spread between runs. If the spread is larger than the difference you are trying to measure, the result is inside noise and you need more runs or better-controlled conditions. Does a higher 3DMark score mean faster local AI? Not reliably. Graphics scores track rendering performance. Local language model speed depends on memory capacity, memory bandwidth and compute throughput in ways a graphics score does not isolate. Use llama-bench for AI decisions and a graphics tool for gaming decisions. Why does llama-bench report two different speeds? The first, pp512, measures prompt processing, which is compute-heavy. The second, tg128, measures token generation, which mostly streams model weights through memory. A card can lead one and trail the other, so read them separately. Is nvidia-smi accurate enough for benchmarking? It is good for trends and comparisons on the same machine, and NVIDIA’s documentation recommends it for testing, monitoring and benchmarking. Sampling is periodic, so very short spikes can slip between samples. Use the same interval for every run and compare like with like. Can I use this harness on an AMD or Intel GPU? Yes, with changes. Build llama.cpp with the Vulkan backend, run llama-bench the same way, and replace telemetry.py with the vendor’s monitoring command. The orchestration, report and comparison scripts stay the same once the CSV columns match. Should I benchmark with the latest llama.cpp build or the stable release? Use the stable v1.0.0 release for comparisons, and upgrade only when you want to measure the new version itself. Numbered development builds change daily, which makes cross-run comparisons unreliable. When you do upgrade, rerun your baseline on the new version before comparing anything else. How long should I wait between runs? Ninety seconds works for most desktop cards. Extend it to two or three minutes in a warm room or on a small case. If round-to-round spread stays high after you lengthen the cooldown, look for background load rather than adding more time.