{"slug": "show-hn-mlxsh-a-lightweight-cli-to-serve-multiple-local-llms-on-apple", "title": "Show HN: mlxsh A lightweight CLI to serve multiple local LLMs on Apple", "summary": "Developer ansnadeem released mlxsh, a lightweight command-line tool that serves multiple local MLX models on Apple silicon at OpenAI-compatible endpoints, supporting text-only and multimodal modes, and browsing Hugging Face Hub. The tool, available on GitHub, keeps several models loaded simultaneously, switches engines on the same port, and requires Python 3.10 or newer on macOS. It installs via a curl command and manages state in ~/.mlxsh/.", "body_md": "Run local [MLX](https://github.com/ml-explore/mlx) models on Apple silicon.\nmlxsh serves a model at an OpenAI-compatible endpoint, keeps several loaded at\nonce, switches a model between text-only and multimodal, and browses and\ndownloads from the Hugging Face Hub. One file, standard library only; it drives\n`mlx-lm`\n\nand `mlx-vlm`\n\n.\n\n```\nmlxsh> lm gemma-4-26B             serve it at http://127.0.0.1:41277/v1\nmlxsh> lm qwen3.6                 a second model, the first keeps running\nmlxsh> ask --on qwen \"hello\"      no extra copy is loaded\nmlxsh> stop all\n```\n\nmacOS on Apple silicon, Python 3.10 or newer. Driving mlxsh from a script or a\ncoding agent: [AGENTS.md](/ansnadeem/mlxsh/blob/main/AGENTS.md).\n\n```\ncurl -LsSf https://raw.githubusercontent.com/ansnadeem/mlxsh/main/install.sh | sh\nmlxsh setup\n```\n\nThe first line installs [uv](https://github.com/astral-sh/uv) if it is missing,\nthen mlxsh with both engines. The second is only needed if something is\nmissing: it installs `mlx-lm`\n\n, `mlx-vlm`\n\nand `huggingface_hub`\n\ninto whichever\nenvironment mlxsh runs from, about 300 MB. `mlxsh doctor`\n\nshows what it found.\n\nState lives in `~/.mlxsh/`\n\n: the registry, one file per running server, logs and\nhistory. Override with `MLXSH_HOME`\n\n.\n\n| mode | engine | what loads |\n|---|---|---|\n`lm` |\n`mlx_lm` |\ntext weights only, no vision tower |\n`vision` |\n`mlx_vlm` |\nthe full multimodal stack |\n\nMost MLX repos are vision-capable, so the mode picks the engine, not the model. Loading a model that is already running in the other mode swaps the engine on the same port.\n\nOne model per port. When the configured port is busy the next model takes the next free port, so nothing running is stopped by accident.\n\n```\nmlxsh> status\n  lm      gemma-4-26B-A4B-it-qat-4bit    15.0 GB   up 01:15   http://127.0.0.1:41277/v1\n  vision  gemma-4-31b-it-4bit            18.4 GB   up 00:59   http://127.0.0.1:41278/v1\n  2 servers, 33.4 GB resident now, about 33.4 GB of weights, machine has 68.7 GB\n```\n\n`stop`\n\n, `bench`\n\n, `ask`\n\n, `chat`\n\nand `log`\n\ntake a port, a model substring or a\nmode: `stop 41278`\n\n, `bench gemma`\n\n, `ask --on vision \"...\"`\n\n, `stop all`\n\n. Leave\nthe argument off in the shell and you get a picker. `--new`\n\nforces a new port,\n`--replace`\n\nreuses the configured one, and `config when_busy`\n\nsets the default.\n\nBefore adding a model mlxsh adds up the weights already loaded and asks if the\nnew one will not fit. `-y`\n\nskips the question.\n\nAny OpenAI client works. There is no API key.\n\n```\ncurl -s http://127.0.0.1:41277/v1/chat/completions \\\n  -H 'Content-Type: application/json' \\\n  -d '{\"model\":\"default_model\",\"messages\":[{\"role\":\"user\",\"content\":\"hello\"}]}'\n```\n\n`default_model`\n\nmeans whatever that port was started with, so a client pinned\nto a port needs no repo id. An exact repo id works too, and `mlxsh status --json`\n\nreports it.\n\n`GET /v1/models`\n\non a port lists that one model. mlx_lm builds that list by\nscanning the Hugging Face cache, so a server started against the real cache\nadvertises every model on the machine and clients cannot tell which is loaded;\nmlxsh gives each server a cache view holding only its own model. That also\nstops a stray request from swapping the model out or downloading another one.\n`config pin_model off`\n\nrestores mlx_lm's behaviour.\n\nThe servers have no authentication and bind to `127.0.0.1`\n\n;\n`config host 0.0.0.0`\n\nexposes them to your network.\n\n```\nmlxsh                    this list\nmlxsh shell              the list, then the interactive shell\nmlxsh <command> ...      run one command and exit\n\nlm [model]               serve text only\nvision [model]           serve multimodal\nstop [port|model|all]    stop a server and free the memory\nstatus [--json]          what is running\nbench [port|model] [n]   measure tok/s, saved to the registry\nlog [port|model] [n]     tail one server's log\n\nmodels                   pick a model, then pick an action\nls [--json]              the same list as plain text\nuse [model] [mode]       set the default for lm or vision\nrm [model] [-y]          delete from disk\n\nbrowse [filters]         live list from hugging face, space marks downloads\nget [n|repo] [-y]        download from that list, or by repo id\n\nask <text>               single prompt, --image PATH sends a picture\nchat                     a chat loop\n--on <port|model>        which running model answers\n--load                   load a private copy instead of using a server\n\nconfig                   settings, and where each value comes from\ndoctor                   versions, paths, machine\nsetup                    install the MLX packages\nedit                     open the registry in $EDITOR\nhelp                     this list\n\n--port N, --host ADDR    override the address for one run\n--foreground             run a server attached to this terminal\n```\n\n`model`\n\nis a repo id, a number from `ls`\n\n, or a unique substring: `lm 3`\n\n,\n`lm qwen3.6`\n\n, `vision gemma-4-31b`\n\n. A command that prints `error:`\n\nexits\nnon-zero.\n\n`mlxsh <command> -h`\n\nexplains one command with examples. Errors go to stderr\nand exit non-zero, so a command composes in a pipeline.\n\nEvery command works both one-shot and in the shell. The shell adds what only\nmakes sense while you are sitting in it: a picker whenever you leave an\nargument off, a command list on an empty line, tab completion, history, a chat\nloop, and a line pinned to the top showing what is loaded (`config status_bar off`\n\nhides it). One-shot runs never open a picker, so scripts stay\npredictable.\n\nIn a picker: up/down or j/k move, type to filter, enter selects, space marks several, esc cancels.\n\n`browse`\n\nestimates each repo's memory from its safetensors dtypes, so 4-bit\npacking is counted correctly, and hides what will not fit unless you ask for\n`all`\n\n. Filters: `vision`\n\n, `text`\n\n, `trending`\n\n, `popular`\n\n, `new`\n\n, `all`\n\n, any\nother word searches, and `org/`\n\nscopes to an org.\n\n```\nconfig                   list settings and their source\nconfig port 8080         change one\nconfig reset port        back to the default\n```\n\nFlag beats environment beats registry beats default.\n\n| setting | env | default |\n|---|---|---|\n`host` |\n`MLXSH_HOST` |\n`127.0.0.1` |\n`port` |\n`MLXSH_PORT` |\n`41277` |\n`org` |\n`MLXSH_ORG` |\n`mlx-community` |\n`mem_comfy` |\n`MLXSH_MEM_COMFY` |\n`0.7` |\n`mem_tight` |\n`MLXSH_MEM_TIGHT` |\n`0.85` |\n`browse_limit` |\n`MLXSH_BROWSE_LIMIT` |\n`25` |\n`start_timeout` |\n`MLXSH_START_TIMEOUT` |\n`900` |\n`reply_timeout` |\n`MLXSH_REPLY_TIMEOUT` |\n`600` |\n`when_busy` |\n`MLXSH_WHEN_BUSY` |\n`new` |\n`pin_model` |\n`MLXSH_PIN_MODEL` |\n`on` |\n`status_bar` |\n`MLXSH_STATUS_BAR` |\n`on` |\n`bar_interval` |\n`MLXSH_BAR_INTERVAL` |\n`2.0` |\n\nAlso read: `MLXSH_HOME`\n\n, `MLXSH_PYTHON`\n\n, `MLXSH_DEBUG`\n\n, `HF_TOKEN`\n\n, `NO_COLOR`\n\n.\n\n| symptom | what to do |\n|---|---|\n`port 41277 is held by pid N, not an mlx server` |\n`config port <n>` , or stop that process |\n| a model exits during startup | `log` shows the last lines: usually an unsupported architecture or memory |\n| everything is slow with several models loaded | they compete for bandwidth, `stop` one |\n| a thinking model answers nothing | it spent its budget reasoning, raise `config ask_args` |\n| downloads are rate limited | set `HF_TOKEN` |\n\n`doctor`\n\nis the first thing to include in a bug report. `MLXSH_DEBUG=1`\n\nturns\nan unexpected error into a traceback.\n\n```\nuv tool uninstall mlxsh\nrm -rf ~/.mlxsh\n```\n\nModels live in the Hugging Face cache, not in `~/.mlxsh`\n\n.\n\n```\npython3 -m unittest discover -s tests\n```\n\nNo network, no models, no MLX packages needed.\n\n[AGENTS.md](/ansnadeem/mlxsh/blob/main/AGENTS.md)for scripts and coding agents[CHANGELOG.md](/ansnadeem/mlxsh/blob/main/CHANGELOG.md),[CONTRIBUTING.md](/ansnadeem/mlxsh/blob/main/CONTRIBUTING.md)- MIT, see\n[LICENSE](/ansnadeem/mlxsh/blob/main/LICENSE)", "url": "https://wpnews.pro/news/show-hn-mlxsh-a-lightweight-cli-to-serve-multiple-local-llms-on-apple", "canonical_source": "https://github.com/ansnadeem/mlxsh", "published_at": "2026-08-03 00:03:17+00:00", "updated_at": "2026-08-03 00:52:53.034363+00:00", "lang": "en", "topics": ["developer-tools", "ai-tools", "machine-learning", "large-language-models"], "entities": ["ansnadeem", "mlxsh", "MLX", "Hugging Face Hub", "mlx-lm", "mlx-vlm", "uv", "Apple"], "alternates": {"html": "https://wpnews.pro/news/show-hn-mlxsh-a-lightweight-cli-to-serve-multiple-local-llms-on-apple", "markdown": "https://wpnews.pro/news/show-hn-mlxsh-a-lightweight-cli-to-serve-multiple-local-llms-on-apple.md", "text": "https://wpnews.pro/news/show-hn-mlxsh-a-lightweight-cli-to-serve-multiple-local-llms-on-apple.txt", "jsonld": "https://wpnews.pro/news/show-hn-mlxsh-a-lightweight-cli-to-serve-multiple-local-llms-on-apple.jsonld"}}