A practical guide to deploying Nanbeige4.3-33B locally with Transformers, SGLang, or vLLM, including thinking mode and tool-use setup.
What is Nanbeige4.3-33B and how do you run it locally? #
Nanbeige4.3-33B is a 33-billion-parameter open-weight model built as the flagship “teacher” model in the Nanbeige4.3 series, designed for agentic coding and professional cowork tasks. You can run it locally through Transformers for quick testing, or through SGLang and vLLM for production-grade serving with tensor parallelism, tool calling, and reasoning parsing. All three paths use the same chat template, which exposes settings for thinking mode, multi-turn reasoning retention, and structured tool calls.
TL;DR #
- Nanbeige4.3-33B is positioned as a teacher model meant to anchor distillation into smaller models like a planned Nanbeige4.3-3B, rather than a standalone chat model only.
- The model supports a 512K token context window (524,288 tokens), making it suitable for long agentic sessions and large codebases.
- You can deploy it three ways: Transformers for local experimentation,SGLang for optimized serving, orvLLM for high-throughput production inference.
- The chat template has three key switches:
enable_thinkingfor reasoning generation,preserve_thinkingfor carrying reasoning across turns, and atoolsparameter that activates structured tool-call formatting. - Nanbeige recommends XML-format tool calls over JSON for best tool-calling accuracy, though JSON remains supported for compatibility with existing agent frameworks.
- Inference settings differ by task: agentic and tool-use work calls for temperature 1.0 and up to 65,536 new tokens, whilereasoning and chat tasks call for temperature 0.6 and up to 262,144 new tokens.
- On benchmarks reported by the model’s own card, Nanbeige4.3-33B leads comparably sized open models (around 30B parameters) on most agentic coding and cowork tasks, though it trails on a few individual benchmarks like SWE-Bench Pro and GDPval.
Other agents ship a demo. Remy ships an app. #
Real backend. Real database. Real auth. Real plumbing. Remy has it all.
Why would you run this model locally instead of a hosted API? #
Local deployment matters for agentic workloads because of context length, cost, and control. Nanbeige4.3-33B’s 512K context window lets it hold entire codebases, long tool-call histories, or multi-document reference material in a single session, something that gets expensive fast on metered APIs. Running it yourself also means you control the inference harness: which tool-call parser you use, how aggressively you cache reasoning tokens across turns, and how you tune temperature for different workloads.
The model card frames Nanbeige4.3-33B specifically as a foundation for distillation work, meaning teams building smaller, cheaper fine-tuned agents may want local access to the full model to generate training data or evaluate distilled checkpoints against it. That kind of workflow isn’t practical through a hosted inference endpoint with rate limits.
How do you set up Nanbeige4.3-33B with Transformers? #
The simplest path is the Hugging Face Transformers library, useful for quick tests or single-GPU experimentation rather than production traffic. You load the tokenizer with trust_remote_code=True and use_fast=False, then load the model with torch_dtype='auto' and device_map='auto' to let Transformers handle device placement automatically.
A minimal example:
from transformers import AutoModelForCausalLM, AutoTokenizer
model_id = 'Nanbeige/Nanbeige4.3-33B'
tokenizer = AutoTokenizer.from_pretrained(
model_id,
use_fast=False,
trust_remote_code=True
)
model = AutoModelForCausalLM.from_pretrained(
model_id,
torch_dtype='auto',
device_map='auto',
trust_remote_code=True
)
messages = [{'role': 'user', 'content': 'Which number is bigger, 9.11 or 9.8?'}]
inputs = tokenizer.apply_chat_template(
messages,
add_generation_prompt=True,
return_tensors='pt'
).to(model.device)
output_ids = model.generate(inputs, max_new_tokens=1024)
response = tokenizer.decode(output_ids[0][inputs.shape[-1]:], skip_special_tokens=True)
print(response)
This gives you a baseline chat interaction but doesn’t take advantage of the model’s agentic strengths. For tool use and long-context agent work, SGLang or vLLM are the better choice.
How do you serve it with SGLang for agentic workloads? #
SGLang is built for high-throughput serving and includes built-in parsers for reasoning traces and tool calls, both of which Nanbeige4.3-33B relies on. The launch command sets tensor parallelism, memory allocation, and the specific parsers needed to interpret the model’s output format correctly:
MODEL_PATH=/path/to/your/Nanbeige4.3-33B
python -m sglang.launch_server \
--model-path ${MODEL_PATH} \
--host 0.0.0.0 \
--port 8000 \
--tp-size 4 \
--mem-fraction-static 0.8 \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder
Note that --tp-size 4 splits the model across four GPUs. The --reasoning-parser qwen3 and --tool-call-parser qwen3_coder flags tell SGLang to use parsing logic compatible with Nanbeige’s chat template conventions, since the model shares formatting conventions with the Qwen3 family. --mem-fraction-static 0.8 reserves 80% of GPU memory for the model and KV cache, leaving headroom for SGLang’s runtime overhead.
How do you serve it with vLLM instead? #
vLLM offers a near-identical setup path with its own flags for tensor parallelism and tool calling. The equivalent vLLM launch command:
MODEL_PATH=/path/to/your/Nanbeige4.3-33B
vllm serve ${MODEL_PATH} \
--host 0.0.0.0 \
--port 8000 \
--tensor-parallel-size 4 \
--gpu-memory-utilization 0.8 \
--enable-auto-tool-choice \
--reasoning-parser qwen3 \
--tool-call-parser qwen3_coder
- ✕a coding agent
- ✕no-code
- ✕vibe coding
- ✕a faster Cursor
The one that tells the coding agents what to build.
The key difference from SGLang is the --enable-auto-tool-choice flag, which lets vLLM automatically decide when to invoke a tool call based on the model’s output rather than requiring every request to force tool use. Both servers expose an OpenAI-compatible API once running, so existing agent frameworks that already speak the OpenAI chat completions format should connect with minimal changes.
How should you configure thinking mode and tool calls? #
Nanbeige4.3-33B’s chat template exposes three settings worth understanding before deploying it in an agent pipeline:
enable_thinking turns the model’s reasoning generation on or off. It’s on by default. Turn it off for latency-sensitive tasks where you don’t need visible chain-of-thought, such as simple classification or short-form chat.
preserve_thinking decides whether reasoning from earlier turns stays in context for later turns in a multi-turn conversation. Nanbeige recommends setting this to False for general chat and Q&A (to save tokens and avoid stale reasoning influencing new answers) but True for multi-turn tool use, office tasks, and code-agent workflows, where the model benefits from remembering its own prior reasoning steps. Notably, the benchmark numbers on the model card were generated with preserve_thinking=true, so replicating those scores requires that setting.
tools, when passed into the chat template, switches the model into tool-use mode. Nanbeige recommends tool_call_format="xml" for the best tool-calling accuracy, with JSON available as a fallback for compatibility with frameworks that expect standard JSON-formatted function calls.
For inference hyperparameters, the model card gives two distinct profiles. Agentic and tool-use tasks should run at temperature 1.0, top_p 0.95, with up to 65,536 new tokens. Reasoning and chat tasks should run cooler, at temperature 0.6, top_p 0.95, with up to 262,144 new tokens, reflecting the longer, more deliberate outputs expected from extended reasoning chains.
How does it compare to other models at this scale? #
Nanbeige positions 4.3-33B against three other roughly 30B-parameter open models: Qwen3.8-27B, Muse-Glimmer-30B, and Gemma4-31B, all evaluated at their highest thinking effort setting. Across agentic coding benchmarks like Terminal-Bench, DeepSWE, and SkillsBench, Nanbeige4.3-33B leads in most categories, sometimes by wide margins (for example, 65.5 versus 42.5 on DeepSWE against Qwen3.8-27B). It also scores highest on most agentic cowork benchmarks, including JobBench and Automation Bench.
It doesn’t win everywhere. Qwen3.8-27B edges it out on SWE-Bench Pro (61.7 vs 60.3) and GDPval (86.4 vs 84.8), and Gemma4-31B leads on GPQA Diamond, a pure reasoning benchmark. These are the model’s own reported comparisons, so treat them as a starting point rather than independent verification, but they do suggest Nanbeige4.3-33B’s strengths skew toward agentic and coding tasks specifically rather than general reasoning benchmarks.
Frequently Asked Questions #
What hardware do you need to run Nanbeige4.3-33B locally?
The official examples use tensor parallelism across 4 GPUs (--tp-size 4 in SGLang, --tensor-parallel-size 4 in vLLM) with 80% GPU memory utilization. Exact VRAM requirements depend on your quantization format and context length, but a 33B-parameter model at full precision across 4 GPUs is a realistic baseline configuration from the model’s own documentation.
What’s the difference between SGLang and vLLM for this model?
Both support the same core features: tensor parallelism, the qwen3 reasoning parser, and the qwen3_coder tool-call parser. The main functional difference is vLLM’s --enable-auto-tool-choice flag, which automates tool invocation decisions. Choice between the two often comes down to which serving stack your existing infrastructure already uses.
Remy doesn't build the plumbing. It inherits it. #
Other agents wire up auth, databases, models, and integrations from scratch every time you ask them to build something.
Remy ships with all of it from MindStudio — so every cycle goes into the app you actually want.
Should I use XML or JSON for tool calls?
Nanbeige recommends XML-format tool calls (tool_call_format="xml") for the best tool-calling performance. JSON is supported for compatibility with agent frameworks that expect standard JSON function-calling schemas, but expect a performance tradeoff if you choose it over XML.
Does Nanbeige4.3-33B support long context out of the box?
Yes. It supports up to 524,288 tokens (512K) of context, which is handled natively by the chat template and inference servers without extra configuration beyond ensuring your serving setup has enough memory for the KV cache at that length.
Is Nanbeige4.3-33B meant to be used standalone or as a base for smaller models?
Both are supported use cases, but the model card frames it primarily as a teacher model intended to anchor distillation into smaller models, with a 3B-parameter version mentioned as a planned follow-up. Teams wanting a smaller, cheaper deployment may eventually prefer a distilled version once released, while Nanbeige4.3-33B itself serves as the reference model.