A tool that tells you which local LLM your PC can actually run

A developer built Jaull, an open-source Python tool with a terminal UI that detects a machine's CPU, RAM, GPU and backends and recommends up to five concrete local LLM plans from Hugging Face metadata, each specifying the exact file, runtime, backend and GPU offload. Jaull estimates memory per component and treats RAM and VRAM as separate pools, and it only reorders recommendations from benchmark results tied to a specific file's SHA-256. On an RTX 2060, the developer measured LFM2.5-1.2B at 80.2% IFEval versus 41.0% for Qwen2.5-1.5B, and found a corrected GPU offload policy raised Qwen2.5-7B throughput from 21.5 to 44.9 tok/s.

For my final degree project I wanted to know what hardware a small company would need to run language models locally. My method was to download a model, watch it crash or crawl, and try another quantization. So I built Jaull https://github.com/Ton-Llop/Jaull to do that work properly. Jaull is an open-source Python tool with a terminal UI. It detects your CPU, RAM, GPU and backends, asks a few plain questions what you'll use it for, what matters most, languages, commercial use and searches Hugging Face using metadata only. You get up to five concrete plans, each with an exact file, runtime, backend and GPU offload, and an explanation of its position. Jaull estimates memory by component weights, KV cache, overhead, margins and treats RAM and VRAM as two pools, so it can tell you which one you'll run out of. If you want to check, you can run, benchmark or score any plan on your machine. Each result is saved with the file's SHA-256, the machine and the protocol, and only results that match a plan can reorder recommendations. A benchmark of a Q5 K M file says nothing about the Q6 K next to it. Some numbers from my RTX 2060: | Model Q5 K M | IFEval | Generation speed | |---|---|---| | Qwen2.5-0.5B | 25.9% | 246 tok/s | | LFM2.5-1.2B | 80.2% | 231 tok/s | | Qwen2.5-1.5B | 41.0% | 151 tok/s | The 1.2B model beat the 1.5B one by a wide margin on instruction following, which is a good reason not to rank by parameter count. My first offload policy left most of the GPU unused. On Qwen2.5-7B it gave 21.5 tok/s, and putting every layer on the GPU gave 59.4 for 133 MB more VRAM. After fixing the formula, the same run reached 44.9. I also built a tokens/second predictor. A simple version estimated 170 tok/s for a model that measured 18. I removed predictions from the UI, and Jaull now shows a speed only after measuring it. Some numbers can't be measured at all. On a GeForce that drives your display, Windows won't report VRAM per process, so Jaull says "not comparable" there. git clone https://github.com/Ton-Llop/Jaull.git cd Jaull uv sync --locked uv run jaull You don't need llama.cpp to get recommendations, only to run plans. I haven't tested Intel Vulkan or AMD support on real hardware yet. If you try it, I'd like to hear what it recommends for your machine. Jaull is not a finished product.