# LLMario > Source: > Published: 2026-10-02 04:55:41+00:00 [New Now on Windows (preview) →](https://github.com/mannysinghx/LLMario/releases/tag/v0.2.0) # Run open AI models privately on your Mac or PC LLMario is a free, open-source app for chatting with open models like Qwen, Gemma, Llama and gpt-oss on your own computer. It picks the right engine for your hardware, checks the model fits in memory before loading it, and never sends your messages anywhere. Free · Apache-2.0 · macOS 13+ (Apple Silicon and Intel) · Windows 10/11 x64 (preview) · command line and local API included · [all downloads and checksums](https://github.com/mannysinghx/LLMario/releases/latest) ## How it works LLMario doesn't reinvent AI engines. It runs proven open-source engines for you, correctly configured for your machine. ### You ask Chat in the LLMario window, use the `llmario` command, or connect your own apps through the built-in local API. ### LLMario plans It detects your chip and memory, estimates what the model needs, and refuses with a clear reason if it won't fit, instead of freezing your computer. ### The right engine runs it **MLX** (Apple's engine, usually fastest on Apple Silicon) or **llama.cpp** (runs everywhere, including Windows). LLMario starts, watches and stops it for you. ### Answers stay on your computer Replies stream back as the model writes them. No accounts, no cloud, no telemetry. Your messages are never logged. ## Get started in 3 steps 1. 1### Install an engineEngines do the actual AI work. Install one (or both): Recommended: install both. MLX is usually fastest; llama.cpp runs the widest range of models. ``` brew install llama.cpp pip3 install mlx-lm ``` No Homebrew? Get it at [brew.sh](https://brew.sh) .Windows uses llama.cpp. Run this in PowerShell or Command Prompt, then restart LLMario so it finds the engine: ``` winget install ggml.llamacpp ``` No winget? Download a Windows zip from [llama.cpp releases](https://github.com/ggml-org/llama.cpp/releases) and add its folder to your PATH. 2. 2### Install LLMarioDownload **LLMario for macOS** (.dmg, Apple Silicon and Intel), open it, and drag LLMario into Applications. It is signed and notarized by Apple, so it opens without warnings.[Download for macOS](https://github.com/mannysinghx/LLMario/releases/download/v0.2.0/LLMario-0.2.0-macos-universal.dmg)Download **LLMario for Windows** (10 and 11, 64-bit) and run the installer. It installs for your account only, with no admin rights needed. Windows 10 may also install Microsoft's WebView2 component; Windows 11 already has it.[Download for Windows](https://github.com/mannysinghx/LLMario/releases/download/v0.2.0/LLMario-0.2.0-windows-x64-setup.exe)**Preview:** the installer isn't code-signed yet, so Windows may show "Windows protected your PC". Click**More info** →**Run anyway** .Just the command-line tool? Download [llmario for Windows (.zip)](https://github.com/mannysinghx/LLMario/releases/download/v0.2.0/llmario-0.2.0-windows-x64.zip) , unzip it anywhere and run`llmario.exe` .Builds and installs `LLMario.app` and the`llmario` command. Needs[Rust](https://rustup.rs) and Xcode Command Line Tools (`xcode-select --install` ). The script checks both first. ``` git clone https://github.com/mannysinghx/LLMario.git && cd LLMario && ./scripts/install-macos.sh ``` Just the command-line tool: add `--cli-only` . Check prerequisites without installing:`--check` .On Windows, build the command-line tool with Rust and the Visual Studio Build Tools: `cargo install --path crates/cli --locked` in the cloned folder. 3. 3### Pick a model and chat 1. Open **LLMario** : from Launchpad or Spotlight on a Mac, or the Start menu on Windows. 2. Click **Models** →**Library** . Every model shows what it's good at, its size, and whether it fits your computer. 3. Click **Download** on one marked recommended. A small model like*Qwen3.5 4B* or*Gemma 4 E4B* is a good first choice. 4. Select it in the top bar and start typing. The first message loads the model, which takes a few seconds. Already have a model file? Drag a `.gguf` file (or, on a Mac, an MLX model folder) onto the window. It's used where it is, never copied. 4. Open ## What you get ### 🧭 Model library 37 current open-model families with exact Hugging Face names. Search by task (chat, reasoning, coding, agents, multilingual, long context, small) and see what fits *your* computer before downloading. ### 🧠 Memory check first LLMario estimates weights plus conversation memory before loading and explains the limit, with a smaller setting that would fit. ### ⚙️ Right engine, automatically MLX on Apple Silicon, llama.cpp on Intel Macs and Windows. LLMario also knows which model types your installed engine version can load. ### 🔒 Private by default Runs offline once a model is downloaded. Local-only network access, no accounts, no telemetry, and messages are never written to logs. ### ✅ Verified downloads Every file is checked against Hugging Face checksums and pinned to an exact version. Models already on your computer are reused. ### 🔌 Works with your apps An OpenAI-compatible API on your own computer, so tools and scripts that speak OpenAI can use local models instead. ## Models you can run A curated library of current open models, verified against Hugging Face. The app shows the full list with sizes and licenses. Full list with exact file names: [docs/MODELS.md](https://github.com/mannysinghx/LLMario/blob/main/docs/MODELS.md). Check each model's license before commercial use. ## Configure ### In the app: Settings | Setting | What it does | |---|---| | Performance profile | **Latency** : one chat, fastest replies (default).**Balanced** : up to 4 requests at once, e.g. your chat plus other apps.**Throughput** : many parallel requests. | | Context length | How much conversation the model remembers. Longer uses more memory. | | Max reply length | Upper limit on tokens per answer. Raise it for "thinking" models. | | Temperature | 0 = predictable, higher = more creative. | | System prompt | Instructions applied to every chat. | | Chat history | Stored only on this computer. Turn it off or clear it anytime. | ### Advanced: config file Optional. Lives at `~/.llmario/config.toml` (Windows: `%USERPROFILE%\.llmario\config.toml`) and applies to the app, the command line and the API. ``` [server] port = 11500 # local API port [runtime] profile = "latency" # latency | balanced | throughput context = 16384 # tokens per conversation memory_limit_gb = 24 # cap LLMario's memory use [backends] prefer = "mlx" # mlx | llamacpp ``` See the effective settings with `llmario config`. Models are stored in `~/.llmario/models`. ## For developers ### Command line ``` llmario doctor # your hardware, engines, what fits llmario model catalog coding # browse the library llmario model pull qwen3.5-9b # best version for your machine llmario run qwen3.5-9b "Hello!" # chat in the terminal llmario serve # start the local API llmario bench -m qwen3.5-9b # measure speed and memory ``` ### OpenAI-compatible API `llmario serve` listens on `http://127.0.0.1:11500/v1`, on your computer only unless you enable remote access with an API key. ``` python from openai import OpenAI client = OpenAI(base_url="http://127.0.0.1:11500/v1", api_key="local") r = client.chat.completions.create( model="qwen3.5-9b", messages=[{"role": "user", "content": "Hello!"}]) print(r.choices[0].message.content) ``` ## Questions ## Is it really private? Yes. Models run on your own computer. After a model is downloaded, LLMario works offline. The API listens only on your computer by default, messages are never logged, and there is no telemetry. ## Which computers work? **Mac:** macOS 13 or later. Apple Silicon (M1 and newer) is fastest and can use both engines; Intel Macs use llama.cpp. **Windows (preview):** Windows 10 or 11, 64-bit, with llama.cpp. LLMario uses an NVIDIA graphics card when it finds one; otherwise models run on the processor, which is slower. Memory decides model size: 8 GB suits small models (1–4B), 16 GB mid-size (up to ~9B), and 32 GB+ larger ones. The app shows exactly what fits. ## A model says "too large" or "needs a newer engine" **Too large**: pick a smaller model, or lower the context length in Settings. **Needs a newer engine**: the model uses a new architecture your installed engine doesn't support yet. Update it with `brew upgrade llama.cpp` or `pip3 install -U mlx-lm` on a Mac, or `winget upgrade ggml.llamacpp` on Windows, or choose the model's other engine version. ## "Engine not installed" / nothing happens when I chat Install an engine (step 1), then reopen LLMario. **Models → Engines** shows what LLMario found. `llmario doctor` gives full details. ## Windows says "Windows protected your PC" The Windows preview isn't code-signed yet, so Microsoft SmartScreen shows this for new apps. Click **More info** → **Run anyway**. Only do this for the installer downloaded from this site or the [GitHub releases page](https://github.com/mannysinghx/LLMario/releases). You can check the download against its `.sha256` file with `Get-FileHash` in PowerShell. ## macOS says the app can't be opened Official releases are notarized by Apple and open normally. If you received a build from someone else, prefer the GitHub release or install from source. ## Does it use Ollama or send data to a server? No. LLMario runs the open-source engines llama.cpp and MLX directly. It only goes online to download a model you choose from Hugging Face. ## How do I remove it? **Mac:** delete LLMario from Applications. **Windows:** Settings → Apps → LLMario → Uninstall. Models and settings are in `~/.llmario` (Windows: `%USERPROFILE%\.llmario`); delete that folder to free the space. Remove a source-built command-line tool with `cargo uninstall llmario`.