How to Self-Host Nex-N2.5 with SGLang and Docker

Nex-AGI released its Nex-N2.5 family of open-weight agentic models in three sizes — mini, Pro, and Max — with a prebuilt Docker image running a customized SGLang fork (nexagi/sglang:v0.5.18-nex-patch) as the supported deployment path. Nex-N2.5-mini requires 2x H100 GPUs, Pro requires 8x H100, and Max, built on a 1.6-trillion-parameter text-only Mixture-of-Experts model, requires a 16-GPU H200 cluster across two nodes. All three models use temperature 0.7, top_p 0.95, and top_k 40 sampling defaults, with mini and Pro on the qwen3 reasoning parser and Max on deepseek-r1.

How to Self-Host Nex-N2.5 with SGLang and Docker A hands-on guide to self-hosting Nex-N2.5-mini, Pro, and Max using Docker and SGLang, with GPU requirements for each model size. What is Nex-N2.5 and why self-host it? Nex-N2.5 is Nex-AGI’s family of open-weight agentic models, released in three sizes: mini, Pro, and Max. All three ship with a prebuilt Docker image running a customized SGLang fork nexagi/sglang:v0.5.18-nex-patch , so self-hosting means pulling that image, mounting your model weights, and launching an SGLang server with the right flags for your GPU setup. Mini runs on 2x H100, Pro needs 8x H100, and Max requires a 16-GPU H200 cluster across two nodes. TL;DR - Nex-N2.5 comes in three sizes, mini and Pro built on the Nex-N2 multimodal foundation, and Max built on a 1.6-trillion-parameter text-only Mixture-of-Experts model, Nex-AGI’s first full post-training run at trillion-parameter scale. - Docker is the supported deployment path : Nex-AGI publishes a ready-made image, nexagi/sglang:v0.5.18-nex-patch , with their SGLang fork already installed, so you don’t build SGLang from source. - Hardware requirements scale steeply : Nex-N2.5-mini needs 2x H100 GPUs, Pro needs 8x H100, and Max needs 16x H200 GPUs split across two nodes with fast interconnect. - Each model uses a different reasoning parser and chat template : mini and Pro use the qwen3 reasoning parser, while Max uses deepseek-r1 , and all three need the qwen3 coder tool-call parser plus a model-specific Jinja chat template. - Max’s launch command is meaningfully more complex , adding tensor parallelism across nodes, expert parallelism --ep-size 16 , FP8 KV-cache quantization, and DeepEP/DeepGEMM MoE backends that mini and Pro don’t need. - Sampling defaults are consistent across the family : temperature 0.7, top p 0.95, top k 40, and a reasoning effort parameter to dial thinking behavior up or down. - Benchmark tables show a clear size-to-capability curve , with Max closing much of the gap to frontier closed models like Claude Opus 5 and GPT-5.6 on agentic and browsing tasks, while mini trades accuracy for a dramatically smaller footprint. One coffee. One working app. You bring the idea. Remy manages the project. How do you deploy Nex-N2.5-mini with Docker? Mini is the accessible entry point, designed to run on a single node with 2x H100 GPUs. The deployment pattern follows standard Docker practice: mount your local model directory into the container, expose a port, and pass SGLang launch flags. The core command structure is: docker run --gpus all --shm-size 32g --ipc=host \ -p 30000:30000 \ -v /path/to/your/model:/model \ nexagi/sglang:v0.5.18-nex-patch \ python3 -m sglang.launch server \ --model-path /model \ --tp 2 \ --host 0.0.0.0 --port 30000 \ --reasoning-parser qwen3 \ --tool-call-parser qwen3 coder \ --chat-template /path/to/nex-N2.5-mini/chat-template.jinja \ --mamba-scheduler-strategy extra buffer The --tp 2 flag sets tensor parallelism across your two GPUs. The --shm-size 32g flag matters because SGLang and multi-GPU tensor parallelism rely on shared memory for inter-process communication, and the default Docker shared memory allocation is too small for this workload. You’ll also need to supply your own copy of the chat template file, referenced by local path, since it isn’t baked into the image. How do you deploy Nex-N2.5-Pro? Pro scales up to a single node with 8x H100 GPUs. The command is nearly identical to mini’s, just with --tp 8 instead of --tp 2 : docker run --gpus all --shm-size 32g --ipc=host \ -p 30000:30000 \ -v /path/to/your/model:/model \ nexagi/sglang:v0.5.18-nex-patch \ python3 -m sglang.launch server \ --model-path /model \ --tp 8 \ --host 0.0.0.0 --port 30000 \ --reasoning-parser qwen3 \ --tool-call-parser qwen3 coder \ --chat-template /path/to/nex-N2.5-Pro/chat-template.jinja \ --mamba-scheduler-strategy extra buffer Because Pro still shares the Nex-N2 multimodal architecture with mini, the same reasoning parser qwen3 and tool-call parser qwen3 coder apply. The only structural difference in the launch command is the tensor-parallel degree, which needs to match your GPU count. An 8-GPU single node is a substantial but not exotic setup, roughly what you’d expect for a mid-size open model with strong agentic benchmark scores. How do you deploy Nex-N2.5-Max across multiple nodes? Max is a different scale of problem entirely. It’s built on a 1.6-trillion-parameter Mixture-of-Experts foundation, and the model card specifies a two-node, 16x H200 GPU cluster as the reference deployment. The launch command reflects that complexity: docker run --gpus all --shm-size 32g --network host \ -v /path/to/your/model:/model \ nexagi/sglang:v0.5.18-nex-patch \ python3 -m sglang.launch server \ --model-path /path/to/your/model \ --trust-remote-code \ --host 0.0.0.0 \ --port 8000 \ --nnodes 2 \ --node-rank "${NODE RANK}" \ --dist-init-addr "${MASTER ADDR}:5000" \ --tp 16 \ --pp-size 1 \ --dp 1 \ --ep-size 16 \ --attention-backend dsv4 \ --kv-cache-dtype fp8 e4m3 \ --page-size 256 \ --moe-a2a-backend deepep \ --moe-runner-backend deep gemm \ --moe-dense-tp-size 1 \ --deepep-mode auto \ --context-length 262144 \ --mem-fraction-static 0.84 \ --chunked-prefill-size 8192 \ --enable-mixed-chunk \ --disable-overlap-schedule \ --max-running-requests 64 \ --cuda-graph-max-bs-decode 64 \ --cuda-graph-backend-decode full \ --cuda-graph-backend-prefill disabled \ --chat-template /path/to/nex-n2.5-max/chat template.jinja \ --reasoning-parser deepseek-r1 \ --tool-call-parser qwen3 coder You run this exact command on both nodes, changing only NODE RANK 0 for the head node, 1 for the other and pointing MASTER ADDR at the head node’s IP, which needs to be reachable from every other node in the cluster. Several flags here don’t appear in the mini or Pro commands at all: --ep-size 16 sets expert parallelism, appropriate for a MoE model where different GPUs handle different experts. --moe-a2a-backend deepep and --moe-runner-backend deep gemm configure specialized MoE communication and kernel backends. --kv-cache-dtype fp8 e4m3 quantizes the KV cache to FP8 to save memory at this scale, and --context-length 262144 sets a 256K token context window. --network host replaces the port mapping used for mini and Pro, since multi-node communication needs direct host networking rather than Docker’s bridge network. What GPU and infrastructure requirements should you plan for? The three model sizes map to three distinct infrastructure tiers. Mini fits on 2 H100s, which is achievable for a small team or an individual with cloud GPU access. Pro needs 8 H100s on one node, a jump that typically means renting a dedicated multi-GPU instance from a cloud provider rather than assembling hardware yourself. Max needs 16 H200 GPUs across two networked nodes, which puts it firmly in data-center territory: you need low-latency interconnect between nodes the --dist-init-addr and --network host settings assume this , enough shared memory --shm-size 32g , and careful attention to the --mem-fraction-static setting that controls how much GPU memory SGLang reserves for KV cache versus model weights. None of the three commands specify quantization for the model weights themselves only Max’s KV cache uses FP8 , so plan storage and GPU memory around the published parameter counts, keeping in mind that mini and Pro are smaller checkpoints from the Nex-N2 lineage, while Max is the 1.6T-parameter MoE. Is self-hosting Nex-N2.5 worth it over using a hosted endpoint? Nex-N2.5-Pro and Nex-N2.5-mini are both available through OpenRouter, which removes the GPU procurement problem entirely if you just want API access. Self-hosting makes sense when you need data locality, want to fine-tune or modify the serving stack, or are running high enough volume that dedicated GPUs beat per-token API pricing. Given that Max requires a 16-GPU H200 cluster, self-hosting that tier is realistically limited to organizations with existing multi-node GPU infrastructure or a specific reason to avoid a hosted API. Mini is the more approachable self-hosting option for teams testing agentic workflows, computer use, or web-browsing tasks without data-center-scale hardware. Frequently Asked Questions What Docker image do I need to run Nex-N2.5? Use nexagi/sglang:v0.5.18-nex-patch , a prebuilt image with Nex-AGI’s customized SGLang fork already installed. All three model sizes mini, Pro, Max use this same image, with different launch flags. How many GPUs does each Nex-N2.5 size need? Nex-N2.5-mini needs 2x H100, Nex-N2.5-Pro needs 8x H100 on a single node, and Nex-N2.5-Max needs 16x H200 GPUs split across two nodes. Do mini, Pro, and Max use the same reasoning parser? No. Mini and Pro use the qwen3 reasoning parser since they share the Nex-N2 multimodal foundation. Max uses deepseek-r1 since it’s built on a separate 1.6-trillion-parameter MoE architecture. All three use qwen3 coder as the tool-call parser. Can I run Nex-N2.5-Max on a single node? No. The documented deployment for Max spans two nodes with 16 H200 GPUs total, using --nnodes 2 and matching --node-rank and --dist-init-addr settings on each machine so they can coordinate over the network. What sampling settings does Nex-N2.5 recommend? Across all three sizes, the recommended defaults are temperature 0.7, top p 0.95, and top k 40. You can also adjust the reasoning effort parameter to control how much the model thinks before responding.