DGXMM is a self-hosted web app for serving local LLMs on an NVIDIA DGX Spark (GB10, 128 GB unified memory) or a similar single-GPU Linux box (see supported hardware). It deploys the models on your disk as vLLM or llama.cpp containers, checks before every start that a model fits next to everything already running, and serves all of them through one OpenAI-compatible LiteLLM endpoint with virtual keys. An optional small local decision model adds advice (why a start failed, which config, engine, parser or draft model to use, whether a deployment is healthy, when it could sleep), always after DGXMM's own rules and never without your confirmation. It runs as a single container and installs with one command.
Documentation: https://nikghodki.github.io/dgxmodelmanager/
- Memory-safe deploys: every deploy is previewed against a unified-memory budget; when a model does not fit, DGXMM offers plans and does nothing until you confirm one.
- vLLM and llama.cpp , with switch and reconfigure that roll back automatically, and engines that start one at a time.
- One LiteLLM endpoint for every deployment, with virtual keys (your LiteLLM or a bundled one).
- On-demand models: a model can start on its first request and stop again after a few idle minutes, making room or queueing when memory is short. It works through LiteLLM on DGXMM's Docker network, so new deployments are always on unless you choose it (the deploy wizard preselects it when LiteLLM is set up). Seeon-demand models .
- An optional decision model for 16 advisory features,insights from background checks,schedules proposed from traffic, and**⌘K Ask** in the command palette.
- Observability and administration: live metrics, tuning advice, a benchmark, the process list,
adminandviewerroles and an audit log.
See all features and the screenshots.
On Linux with an NVIDIA GPU, Docker with the Compose plugin and the NVIDIA runtime (requirements):
git clone https://github.com/nikghodki/dgxmodelmanager.git && cd dgxmodelmanager && ./install.sh
Open the URL it prints and sign in as admin with the printed password. Add --with-litellm for a
bundled LiteLLM endpoint, --with-decider for the decision model, or --dry-run to see first what it
would do. Next: install options,
first login and
upgrade.
Cloned before 2026-10-18? The repository history was rewritten, so ./install.sh upgrade refuses
to fast-forward an older clone. Move it once with git fetch && git reset --hard origin/main, then run
./install.sh (your .env, config.yaml, secrets/ and data/ stay as they are; see
clones from before the history rewrite).
| Section | What it covers |
|---|---|
| Getting started | Features, requirements and supported hardware, install, first login, upgrade, uninstall |
| User guide | The UI page by page: dashboard, models and catalog, deploying, deployments, insights, schedules, ⌘K, the decision model, LiteLLM, processes, users, settings |
| Architecture | Components, request and data flow, memory budget, deployer, decision model, job engine, schedule runner, data model, frontend |
| Configuration | Every setting with its default |
| Operations | Backups, logs, admin command line, troubleshooting |
| Security | Threat model, hardening, security policy |
| Evaluation | The decision model's 16 features measured on real data |
The site is built from docs/ (see docs/README.md).
DGXMM mounts the Docker socket so it can start engine containers: access to it is equivalent to root
on the host, so only give the admin role to people you would trust with root, and keep the UI off
the public internet. See the security overview.
Reporting a vulnerability: please do not open a public issue. Use GitHub's private vulnerability reporting as described in SECURITY.md, which also lists what is in scope.
Bug reports and feature requests are welcome as
issues: the forms ask for what is needed
to reproduce a problem (version, host and GPU, ./install.sh status, logs). The development setup, the
test suites and the code layout are in CONTRIBUTING.md; the changelog is in
CHANGELOG.md.
MIT. See LICENSE.