Show HN: mlxsh A lightweight CLI to serve multiple local LLMs on Apple

Developer ansnadeem released mlxsh, a lightweight command-line tool that serves multiple local MLX models on Apple silicon at OpenAI-compatible endpoints, supporting text-only and multimodal modes, and browsing Hugging Face Hub. The tool, available on GitHub, keeps several models loaded simultaneously, switches engines on the same port, and requires Python 3.10 or newer on macOS. It installs via a curl command and manages state in ~/.mlxsh/.

Run local MLX https://github.com/ml-explore/mlx models on Apple silicon. mlxsh serves a model at an OpenAI-compatible endpoint, keeps several loaded at once, switches a model between text-only and multimodal, and browses and downloads from the Hugging Face Hub. One file, standard library only; it drives mlx-lm and mlx-vlm . mlxsh lm gemma-4-26B serve it at http://127.0.0.1:41277/v1 mlxsh lm qwen3.6 a second model, the first keeps running mlxsh ask --on qwen "hello" no extra copy is loaded mlxsh stop all macOS on Apple silicon, Python 3.10 or newer. Driving mlxsh from a script or a coding agent: AGENTS.md /ansnadeem/mlxsh/blob/main/AGENTS.md . curl -LsSf https://raw.githubusercontent.com/ansnadeem/mlxsh/main/install.sh | sh mlxsh setup The first line installs uv https://github.com/astral-sh/uv if it is missing, then mlxsh with both engines. The second is only needed if something is missing: it installs mlx-lm , mlx-vlm and huggingface hub into whichever environment mlxsh runs from, about 300 MB. mlxsh doctor shows what it found. State lives in ~/.mlxsh/ : the registry, one file per running server, logs and history. Override with MLXSH HOME . | mode | engine | what loads | |---|---|---| lm | mlx lm | text weights only, no vision tower | vision | mlx vlm | the full multimodal stack | Most MLX repos are vision-capable, so the mode picks the engine, not the model. Loading a model that is already running in the other mode swaps the engine on the same port. One model per port. When the configured port is busy the next model takes the next free port, so nothing running is stopped by accident. mlxsh status lm gemma-4-26B-A4B-it-qat-4bit 15.0 GB up 01:15 http://127.0.0.1:41277/v1 vision gemma-4-31b-it-4bit 18.4 GB up 00:59 http://127.0.0.1:41278/v1 2 servers, 33.4 GB resident now, about 33.4 GB of weights, machine has 68.7 GB stop , bench , ask , chat and log take a port, a model substring or a mode: stop 41278 , bench gemma , ask --on vision "..." , stop all . Leave the argument off in the shell and you get a picker. --new forces a new port, --replace reuses the configured one, and config when busy sets the default. Before adding a model mlxsh adds up the weights already loaded and asks if the new one will not fit. -y skips the question. Any OpenAI client works. There is no API key. curl -s http://127.0.0.1:41277/v1/chat/completions \ -H 'Content-Type: application/json' \ -d '{"model":"default model","messages": {"role":"user","content":"hello"} }' default model means whatever that port was started with, so a client pinned to a port needs no repo id. An exact repo id works too, and mlxsh status --json reports it. GET /v1/models on a port lists that one model. mlx lm builds that list by scanning the Hugging Face cache, so a server started against the real cache advertises every model on the machine and clients cannot tell which is loaded; mlxsh gives each server a cache view holding only its own model. That also stops a stray request from swapping the model out or downloading another one. config pin model off restores mlx lm's behaviour. The servers have no authentication and bind to 127.0.0.1 ; config host 0.0.0.0 exposes them to your network. mlxsh this list mlxsh shell the list, then the interactive shell mlxsh <command ... run one command and exit lm model serve text only vision model serve multimodal stop port|model|all stop a server and free the memory status --json what is running bench port|model n measure tok/s, saved to the registry log port|model n tail one server's log models pick a model, then pick an action ls --json the same list as plain text use model mode set the default for lm or vision rm model -y delete from disk browse filters live list from hugging face, space marks downloads get n|repo -y download from that list, or by repo id ask <text single prompt, --image PATH sends a picture chat a chat loop --on <port|model which running model answers --load load a private copy instead of using a server config settings, and where each value comes from doctor versions, paths, machine setup install the MLX packages edit open the registry in $EDITOR help this list --port N, --host ADDR override the address for one run --foreground run a server attached to this terminal model is a repo id, a number from ls , or a unique substring: lm 3 , lm qwen3.6 , vision gemma-4-31b . A command that prints error: exits non-zero. mlxsh <command -h explains one command with examples. Errors go to stderr and exit non-zero, so a command composes in a pipeline. Every command works both one-shot and in the shell. The shell adds what only makes sense while you are sitting in it: a picker whenever you leave an argument off, a command list on an empty line, tab completion, history, a chat loop, and a line pinned to the top showing what is loaded config status bar off hides it . One-shot runs never open a picker, so scripts stay predictable. In a picker: up/down or j/k move, type to filter, enter selects, space marks several, esc cancels. browse estimates each repo's memory from its safetensors dtypes, so 4-bit packing is counted correctly, and hides what will not fit unless you ask for all . Filters: vision , text , trending , popular , new , all , any other word searches, and org/ scopes to an org. config list settings and their source config port 8080 change one config reset port back to the default Flag beats environment beats registry beats default. | setting | env | default | |---|---|---| host | MLXSH HOST | 127.0.0.1 | port | MLXSH PORT | 41277 | org | MLXSH ORG | mlx-community | mem comfy | MLXSH MEM COMFY | 0.7 | mem tight | MLXSH MEM TIGHT | 0.85 | browse limit | MLXSH BROWSE LIMIT | 25 | start timeout | MLXSH START TIMEOUT | 900 | reply timeout | MLXSH REPLY TIMEOUT | 600 | when busy | MLXSH WHEN BUSY | new | pin model | MLXSH PIN MODEL | on | status bar | MLXSH STATUS BAR | on | bar interval | MLXSH BAR INTERVAL | 2.0 | Also read: MLXSH HOME , MLXSH PYTHON , MLXSH DEBUG , HF TOKEN , NO COLOR . | symptom | what to do | |---|---| port 41277 is held by pid N, not an mlx server | config port <n , or stop that process | | a model exits during startup | log shows the last lines: usually an unsupported architecture or memory | | everything is slow with several models loaded | they compete for bandwidth, stop one | | a thinking model answers nothing | it spent its budget reasoning, raise config ask args | | downloads are rate limited | set HF TOKEN | doctor is the first thing to include in a bug report. MLXSH DEBUG=1 turns an unexpected error into a traceback. uv tool uninstall mlxsh rm -rf ~/.mlxsh Models live in the Hugging Face cache, not in ~/.mlxsh . python3 -m unittest discover -s tests No network, no models, no MLX packages needed. AGENTS.md /ansnadeem/mlxsh/blob/main/AGENTS.md for scripts and coding agents CHANGELOG.md /ansnadeem/mlxsh/blob/main/CHANGELOG.md , CONTRIBUTING.md /ansnadeem/mlxsh/blob/main/CONTRIBUTING.md - MIT, see LICENSE /ansnadeem/mlxsh/blob/main/LICENSE