{"slug": "why-my-first-cpu-benchmark-was-too-noisy-to-use-and-how-i-fixed-it", "title": "Why my first CPU benchmark was too noisy to use, and how I fixed it", "summary": "A developer's first CPU inference benchmark of ResNet-18 on PyTorch, ONNX Runtime and OpenVINO was unusable because 12 of 18 repeated rows differed by more than 5% between identical runs, with one PyTorch row at 12 threads and batch 1 varying by 48.27%, according to the cpu-inference-bench repo's README. The developer fixed the setup by pinning the 6 P-cores of an Intel i7-12650H at a sustained 2600 MHz with SMT siblings offline, stopping background services, waiting for package temperature below 60 °C before each repeat, interleaving rounds, and only printing speedup ratios between rows whose spread over 5 repeats was under 5% with no thermal-throttle counter increase. Under those controls, ResNet-18 INT8 versus FP32 at batch 1 measured 2.34× on PyTorch, 2.74× on ONNX Runtime and 3.48× on OpenVINO at 2.6 ms, though 5 of 36 BERT rows and 2 of 18 ResNet rows remained unstable and are marked rather than dropped.", "body_md": "# Why my first CPU benchmark was too noisy to use, and how I fixed it\n\nMy first benchmark numbers looked reasonable, until I ran the same thing twice and got different answers. This post is about how I found out why, and what I changed before trusting any number. All numbers come from the repo's README (including its \"First attempt (archived)\" section) and results/arch\n\nive_summary.json in cpu-inference-bench. I started by running ResNet-18 on PyTorch, ONNX Runtime and OpenVINO with default settings on a laptop CPU (Intel i7-12650H). To check repeatability I ran each configuration twice. In 12 of 18 repeated rows, the two identical runs differed by more than 5%, up to 48.27% for one row (PyTorch, 12 threads, batch 1).\n\nThe results were not usable as a comparison, so I archived them. A laptop has package power and thermal limits, so the clock changes during a run. Without control over that, a difference of a few percent between frameworks would be within the noise. The first attempt used default frequency scaling, so the clock was free to move.\n\nThis CPU has a sustained package power limit of 45 W and a short-term limit of 115 W, and power and thermal limits like these can change the clock during a run. Background services and other processes sharing the cores can add noise as well. The data from that attempt is archived locally and is not in the public repo, so I can't show its per-run frequencies or temperatures here. Instead of guessing which factor dominated, I changed the setup so that each one is either controlled or visible: every repeat now records the package temperature, the mean and minimum P-core frequency, and the kernel's thermal-throttle counters.\n\nI capped the P-cores at 3000 MHz and ran a small pilot. Two of four rows still raised the kernel's thermal-throttle counters, so I chose 2600 MHz as a clock the cores could sustain. All reported runs use: a fixed 2600 MHz clock on the 6 P-cores, with SMT siblings offline, a sustained clock the cores can hold rather than the peak clock; pinned cores, AC power and background services stopped; a cooldown wait until the package temperature is below 60 °C before every repeat; interleaved rounds, so drift affects all frameworks equally; a stability rule: a row is stable only if its spread over 5 repeats is below 5% and no thermal-throttle counter increased. Speedup ratios are printed only between two stable rows.\n\nUnstable rows are shown and marked, not dropped. With these controls, ResNet-18 INT8 vs FP32 at batch 1 came out at 2.34× (PyTorch), 2.74× (ONNX Runtime) and 3.48× (OpenVINO, 2.6 ms). Five of 36 BERT rows and 2 of 18 ResNet rows (both FP32) were still unstable, and the README says so. Thermal counters do not detect power-limit throttling, so I also checked the frequency columns.\n\nA VPN client was running during all runs and I did not measure its effect. This is one laptop and one campaign. Run the same configuration twice before comparing anything. Report unstable results instead of hiding them.\n\nWrite down what you did not control (here: power limits, VPN). The code was written with Claude Code; the methodology, decisions and verification are mine. Originally published at diyor.dev.\n\n## Key Takeaways\n\n- •I started by running ResNet-18 on PyTorch, ONNX Runtime and OpenVINO with default settings on a laptop CPU (Intel i7-12650H).\n- •This story was reported by **Dev.to** , covering developments in the**dev** space.\n- •AI advancements continue to reshape industries — read the full article on Dev.to for complete coverage.\n\n📖 Continue reading the full article:\n\n[Read Full Article on Dev.to →](https://dev.to/diyor-ai/why-my-first-cpu-benchmark-was-too-noisy-to-use-and-how-i-fixed-it-4m14)", "url": "https://wpnews.pro/news/why-my-first-cpu-benchmark-was-too-noisy-to-use-and-how-i-fixed-it", "canonical_source": "https://ainexusdaily.vercel.app/article/2026-11-18-why-my-first-cpu-benchmark-was-too-noisy-to-use-and-how-i-fixed-it", "published_at": "2026-11-18 14:52:42+00:00", "updated_at": "2026-11-18 16:13:58.645548+00:00", "lang": "en", "topics": ["ai-infrastructure", "mlops"], "entities": ["PyTorch", "ONNX Runtime", "OpenVINO", "Intel i7-12650H", "ResNet-18", "BERT", "Claude Code", "cpu-inference-bench"], "also_reported_by": [], "takeaway": "Run each benchmark configuration twice before comparing: with P-cores pinned at a sustained 2600 MHz, SMT off, background services stopped and a cooldown below 60 °C, ResNet-18 INT8 vs FP32 at batch 1 measured 2.34× on PyTorch, 2.74× on ONNX Runtime and 3.48× on OpenVINO, while 7 of 54 rows stayed u", "recommendation": {"advice": "Repeat every benchmark configuration twice and only report speedup ratios between rows with under 5% spread over 5 repeats and no thermal-throttle counter increase; pin clocks and cool below 60 °C first.", "playbook": "evals-testing", "playbook_url": "https://wpnews.pro/playbooks/evals-testing"}, "alternates": {"html": "https://wpnews.pro/news/why-my-first-cpu-benchmark-was-too-noisy-to-use-and-how-i-fixed-it", "markdown": "https://wpnews.pro/news/why-my-first-cpu-benchmark-was-too-noisy-to-use-and-how-i-fixed-it.md", "text": "https://wpnews.pro/news/why-my-first-cpu-benchmark-was-too-noisy-to-use-and-how-i-fixed-it.txt", "jsonld": "https://wpnews.pro/news/why-my-first-cpu-benchmark-was-too-noisy-to-use-and-how-i-fixed-it.jsonld"}}