{"slug": "homebench-local-llm-benchmark-results-across-speed-memory", "title": "Homebench: Local LLM Benchmark Results Across Speed, Memory", "summary": "Homebench, a local LLM benchmarking tool, tested five models on an RTX 3060 12GB, finding Qwen 2.5 7B fastest at ~58 tokens/sec and LLaMA 3 8B highest quality at 8.2/10, while Mistral Nemo 12B underperformed. The tool reports hardware context and uses GPT-4o-mini for quality scoring.", "body_md": "# Homebench: Local LLM Benchmark Results Across Speed, Memory\n\n## What I tested\n\nI stuck to models that run comfortably on a 16GB VRAM card, since that's the most common setup I see people asking about:\n\n**LLaMA 3 8B**(GGUF Q4_K_M)** Qwen 2.5 7B**(GGUF Q4_K_M)** Mistral Nemo 12B**(GGUF Q4_K_M)** Gemma 2 9B**(GGUF Q4_K_M)(GGUF Q4_K_M)[DeepSeek](/en/tags/deepseek/)7B\n\nAll runs were on an RTX 3060 12GB, Ubuntu 22.04, using\n\n`llama.cpp`\n\nwith CUDA offload. Homebench's three pillars it measures are throughput (tokens/sec), peak memory usage, and an LLM-judged quality score on a fixed prompt set.## The numbers\n\n**Throughput (tokens/sec):** Qwen 2.5 7B leads at ~58 t/s, followed by Gemma 2 9B at ~49 t/s. LLaMA 3 8B sits at ~42 t/s, which surprised me — it felt faster in practice but the benchmark doesn't lie. Mistral Nemo drags at ~28 t/s, likely because it's the only 12B model here and the memory bandwidth gets saturated.**Peak memory:** Gemma 2 9B uses the least at 6.1GB, Qwen 2.5 7B at 6.8GB. LLaMA 3 8B peaks at 7.4GB. Mistral Nemo 12B eats 9.2GB, and DeepSeek 7B hits 8.1GB — odd, since it's smaller on paper.**Quality score:** LLaMA 3 8B wins here, scoring 8.2/10. Qwen 2.5 7B is close behind at 7.9/10. Gemma 2 9B drops to 7.1/10, DeepSeek 7B to 6.8/10, and Mistral Nemo 12B surprisingly comes in last at 6.5/10 despite being the biggest.\n\n## What this tells me\n\nIf you're chasing raw speed and your workload is mostly short responses or code completion, **Qwen 2.5 7B** is the sweet spot. It's fast, light on memory, and the quality dip from LLaMA 3 is small enough that you probably won't notice in most contexts.\n\nIf you care about getting the best possible answer from a 7-9B model and can tolerate a ~30% speed hit, **LLaMA 3 8B** is still the benchmark to beat. The quality gap is real, especially on reasoning and multi-turn prompts.\n\nGemma 2 9B is interesting — it's the most memory-efficient and nearly as fast as Qwen, but the quality score makes me hesitant unless you're severely VRAM-constrained.\n\nMistral Nemo 12B is the disappointment. It's bigger, slower, hungrier for memory, and scored lowest on quality. I'm not sure what niche it's carving out, honestly.\n\n## A practical note on Homebench itself\n\nThe tool is straightforward — install via pip, point it at your model, and it runs through its preset suite. What I appreciate is that it doesn't try to hide the hardware context. It reports your GPU model, driver version, and llama.cpp commit, so you can actually compare apples to apples across different machines. Fair warning: the quality scoring uses a reference LLM (currently GPT-4o-mini via API), so you need an OpenAI key handy. That's fine for me, but worth flagging if you're going fully offline.\n\nIf you're tuning your local LLM setup for a specific use case, I'd recommend running Homebench with your own prompt templates rather than relying on the defaults. The stock prompts favor general-chat scenarios, which may not reflect your actual workload.\n\n```\npip install homebench\nhomebench run --model ./models/qwen2.5-7b-q4km.gguf --prompts custom_prompts.json\n```\n\nThe `custom_prompts.json`\n\nformat is just a list of strings — simple enough to script up your own test set from real user queries.\n\n[Next evalgate: Fail CI on Prompt Regression →](/en/threads/4850/)", "url": "https://wpnews.pro/news/homebench-local-llm-benchmark-results-across-speed-memory", "canonical_source": "https://promptcube3.com/en/threads/4935/", "published_at": "2026-08-04 13:45:44+00:00", "updated_at": "2026-08-04 13:55:00.643137+00:00", "lang": "en", "topics": ["large-language-models", "ai-tools", "ai-research"], "entities": ["Homebench", "Qwen 2.5 7B", "LLaMA 3 8B", "Mistral Nemo 12B", "Gemma 2 9B", "DeepSeek 7B", "RTX 3060", "llama.cpp"], "alternates": {"html": "https://wpnews.pro/news/homebench-local-llm-benchmark-results-across-speed-memory", "markdown": "https://wpnews.pro/news/homebench-local-llm-benchmark-results-across-speed-memory.md", "text": "https://wpnews.pro/news/homebench-local-llm-benchmark-results-across-speed-memory.txt", "jsonld": "https://wpnews.pro/news/homebench-local-llm-benchmark-results-across-speed-memory.jsonld"}}