Triton Control – an open-source control plane for Nvidia Triton on Kubernetes

Operate NVIDIA Triton on Kubernetes from one open-source control plane.

Triton Control brings deployment, model repositories, inference testing, performance analysis, development workspaces, MLflow, and Argo Workflows into one open-source UI for NVIDIA Triton on Kubernetes.

Documentation · Quickstart · Examples · Roadmap

The primary deployment target is Kubernetes through the Helm chart in charts/triton-control

. The same application can also run with Docker Compose or Podman Compose for local evaluation, with reduced Kubernetes-specific functionality such as no self-deployed Triton deployment workflows.

Published container image: ailabtechtriton/triton-control:v1.2.2

NVIDIA Triton is a powerful inference server, but the workflow around it is often fragmented. Teams coordinate model repositories, S3 credentials, Kubernetes resources, model configuration, endpoint tests, performance runs, development environments, and access control across separate tools. Triton Control connects those steps without replacing Triton or hiding the underlying model engineering.

Core capabilities include:

• existing Triton instance registration and management
• self-deployed Triton serving workflows when Triton Control runs in Kubernetes
• per-user browser-based development workspaces backed by code-server
• user management and instance access control
• model inference workflows with model configuration inspection
• S3-backed model repository integration with reusable S3 profiles and an integrated S3 Browser
• Perf Analyzer workflows when Triton Control runs in Kubernetes
• Kubernetes-managed MLflow tracking with persistent storage and an embedded, authenticated MLflow UI
• embedded Argo Workflows UI and API through an authenticated backend proxy

With Docker installed, start the published image:

docker pull ailabtechtriton/triton-control:v1.2.2
docker tag ailabtechtriton/triton-control:v1.2.2 triton-control:compose
docker compose up --no-build

After the stack starts, open http://localhost:8080

in your browser.

To build from source instead:

docker compose up --build

The backend API is available at http://localhost:8000

and PostgreSQL at 127.0.0.1:5433

.

Docker Compose does not provide Kubernetes. Deployments, Development workspaces, managed MLflow, and Argo Workflows are therefore disabled.

For Triton running on the host, use http://host.docker.internal:<published-http-port>

instead of 127.0.0.1

. Before a non-development deployment, replace the secrets and database password defined in compose.yaml

.

With Podman and podman-compose

installed, start the published image:

podman pull docker.io/ailabtechtriton/triton-control:v1.2.2
podman tag docker.io/ailabtechtriton/triton-control:v1.2.2 \
  localhost/triton-control:compose
podman-compose -f podman-compose.yaml up --no-build

After the stack starts, open http://localhost:8080

in your browser.

To build from source instead:

podman-compose -f podman-compose.yaml up --build

The backend API is available at http://localhost:8000

and PostgreSQL at 127.0.0.1:5433

.

Podman Compose does not provide Kubernetes. Deployments, Development workspaces, managed MLflow, and Argo Workflows are therefore disabled.

The Helm chart deploys:

• one combined app image with Nginx, Angular, and FastAPI
• one optional PostgreSQL Deployment
• one optional Argo Workflows installation
• one Service for frontend and backend ports
• optional Ingress routes

OIDC login has been tested with Microsoft Entra ID, Keycloak, and Dex. The Kubernetes deployment has also been tested with Argo CD managing the Helm release in a GitOps workflow.

The Helm deployment can use the published image directly; users do not need to build, pull, or push it on the machine running Helm. Kubernetes pulls the image from Docker Hub when it creates the application pod.

Create a values file, for example values-prod.yaml

:

app:
  image:
    repository: ailabtechtriton/triton-control
    tag: "v1.2.2"
  secretEnv:
    SESSION_SECRET: "replace-me"
    JWT_SECRET: "replace-me"
    S3_SECRET_ENCRYPTION_KEY: "replace-me"

postgresql:
  enabled: true
  auth:
    database: triton_backend
    username: triton
    password: "replace-me"
  persistence:
    enabled: true
    size: 20Gi

ingress:
  enabled: true
  className: nginx
  hosts:
    - host: triton-control.example.com
      paths:
        frontend:
          - path: /
            pathType: Prefix
        backend:
          - path: /api
            pathType: Prefix
          - path: /auth
            pathType: Prefix
          - path: /login
            pathType: Prefix
          - path: /logout
            pathType: Prefix
          - path: /whoami
            pathType: Prefix

Install or upgrade:

helm upgrade --install triton-control ./charts/triton-control \
  --namespace triton-control \
  --create-namespace \
  -f values-prod.yaml

If postgresql.enabled=true

, the chart generates and injects DATABASE_URL

. For an external database, set postgresql.enabled=false

and provide DATABASE_URL

through app.existingSecret

, app.env

, or app.envFrom

.

When you use Triton Control to install a self-deployed Triton instance or Perf Analyzer, namespace behavior depends on backend runtime context:

• Triton Control backend running in Kubernetes (in-cluster detection): self-deployed Triton and Perf Analyzer are created in the same namespace as the Triton Control pod.
• Triton Control backend running outside Kubernetes (for example local dev with KUBERNETES_KUBECONFIG_PATH

): self-deployed Triton remains name-based, while Perf Analyzer defaults to the sharedtriton-control

namespace.

In-cluster detection is automatic and uses Kubernetes runtime signals (ServiceAccount files and Kubernetes service environment).

KUBERNETES_KUBECONFIG_PATH

is intended as a local development/testing override for Triton Control running outside Kubernetes. In-cluster production deployments should use ServiceAccount-based in-cluster configuration.

This mode is useful when working in VS Code or another IDE.

Local development prerequisites:

• Python 3.12

. - Node.js and npm for the Angular frontend.

• Java, Bash, curl, and Python on the frontend host if you run npm run generate:api

; that command downloads and runsswagger-codegen-cli.jar

.

The backend has a local PostgreSQL Compose file with TLS support:

cd triton-backend/postgresql
docker compose up -d

It exposes PostgreSQL on:

127.0.0.1:5433
cd triton-backend
cp .env.example .env

On Windows PowerShell:

cd triton-backend
Copy-Item .env.example .env

The default local database URL is:

DATABASE_URL=postgresql://triton:tritonpw@localhost:5433/triton_backend

macOS/Linux:

cd triton-backend
python -m venv .venv
source .venv/bin/activate
pip install -e ".[dev]"
python main.py

Windows PowerShell:

cd triton-backend
python -m venv .venv
.\.venv\Scripts\activate
pip install -e ".[dev]"
python main.py

Backend API:

https://localhost:8000
https://localhost:8000/docs

If SERVER_HTTPS_ENABLED=false

in .env

, use http://127.0.0.1:8000

instead.

Open a second terminal:

cd triton-frontend
npm ci
npm run generate:api
npm run start:http

Frontend:

http://localhost:4200

The default frontend environment points API calls to:

http://127.0.0.1:8000

HTTPS frontend mode is also available:

npm run start:https

Certificate paths are configured in triton-frontend/angular.json

.

The repository includes .vscode/launch.json

with a generic Python current-file debug configuration. For backend debugging, open triton-backend/main.py

and start the Python debugger from VS Code.

For frontend work, use the integrated terminal:

cd triton-frontend
npm run start:http

Run backend and frontend in separate terminals. The backend must be running for most frontend API workflows.

Backend:

docker run --rm -v "$PWD:/repo" -w /repo ghcr.io/gitleaks/gitleaks:latest detect --no-git --source . --redact --verbose
cd triton-backend
pip install -e ".[dev]"
pip install pip-audit
pip-audit
coverage run -m unittest discover -s tests -p "test_*.py" -v
coverage report --fail-under=75
mypy app/ tests/ scripts/ main.py
ruff check app/ main.py
lint-imports
bandit -r app/ main.py

Frontend:

cd triton-frontend
npm ci
npm audit --audit-level=moderate
npm run generate:api
npm run lint
npm run format:check
npm test -- --watch=false --browsers=ChromeHeadless --code-coverage
npm run test:smoke

triton-frontend/

• Angular Material frontend.triton-backend/

• Python FastAPI backend.charts/triton-control/

• Helm chart for Kubernetes deployment.compose.yaml

• Docker Compose stack for Triton Control and PostgreSQL.podman-compose.yaml

• Podman Compose equivalent of the Docker Compose stack.Dockerfile

• Builds a combined runtime image with frontend, backend, and Nginx.

• Documentation site source: docs/

• Getting started: docs/getting-started.md

• User guide: docs/user-guide.md

• User management: docs/user-management.md

• API documentation: docs/api.md

• Architecture overview: docs/architecture-overview.md

• Architecture backend components: docs/architecture-backend-components.md

• Deployment: docs/deployment.md

• Development: docs/development.md

• Security: docs/security.md

• Troubleshooting: docs/troubleshooting.md

• Backend details: triton-backend/README.md

• Frontend details: triton-frontend/README.md

• Helm chart details: charts/triton-control/README.md

• Backend TLS setup: triton-backend/tls/README.md

source & further reading

github.com — original article