Customization is what enables developers to take a general model and tailor it to use cases, domains, languages, and more.
However, customization comes with a few challenges. It requires infrastructure, technical expertise, and software specific to the workflow, as well as resources such as GPUs and the ability to use them effectively. It also depends on specialized domain knowledge: What algorithm should I use? What environment should I use? How do I know whether the model actually learned?
Customization can appear daunting.
With open models such as the NVIDIA Nemotron 3 family—including Nano, Super, and Ultra—combined with platforms such as Prime Intellect Lab that offer training as a service, the complete customization loop is much more accessible. In this tutorial, you’ll use Prime Intellect Lab to customize NVIDIA Nemotron 3 Nano with hosted reinforcement learning and produce a downloadable LoRA adapter.
The local setup takes about five minutes—that’s all.
By the end of this blog, you’ll learn two things:
- Customization is more accessible than ever.
- Getting started with open models is easier than ever.
What you’ll build #
We’re going to follow a simple flow for this tutorial: Baseline, train, and reevaluate—all using Nemotron 3 Nano—and resulting in a customized model for our task. To show how easy it is to get started with the robust customization tools we have at our disposal, we’ll be using a standard “hello world” example: Python Math.
This tutorial applies to Nemotron 3 Super and Nemotron 3 Ultra. The video will also walk you through the more involved coding task that’s part of a companion notebook to this tutorial:
What customization means #
Customization means taking a model and changing or adapting its trainable parameters so that it becomes more likely to do the kinds of tasks you want it to do successfully.
There are many ways to do this. Depending on the problem, you might use supervised fine-tuning, parameter-efficient fine-tuning, preference optimization, reinforcement learning, or a combination of techniques. For a broader treatment, see the Selecting Large Language Model Customization Techniques post.
This tutorial focuses on reinforcement learning with verifiable rewards (RLVR) on the Python Math environment. The basic idea of the environment is simple: Use Python tools (Python, numpy
, sympy
and scipy
) to do some math. Here’s an example of a prompt:
Use Python for all calculations. Give your answer inside \boxed{}.
In addition to the Python standard library, you have access to: numpy sympy scipy.
If $2^8=4^x$, what is the value of $x$?
NOTE: The environment has a default of 100 turns. For this experiment, we cap each rollout at five assistant turns. A direct response is one turn, and so a tool call with the final response is two of the 5 turns. This penalizes the model heavily for repeatedly calling tools without producing a final response.
For a deeper introduction to reinforcement learning for agents, see the post Mastering Agentic Techniques: AI Agent Reinforcement Learning.
Why openness matters #
Open weights are an important part of the equation, but they aren’t the entire equation. Data, evaluation code, and training recipes also determine what you can understand, reproduce, and deploy.
The NVIDIA Nemotron 3 family is published with open model resources, including weights, data, and recipes. You can explore the NVIDIA Nemotron developer resources, read Inside NVIDIA Nemotron 3, and inspect the NVIDIA Nemotron repository.
These resources give a better view into how Nemotron was built and post-trained, the kinds of data and environments that were used, and where your own customization should differ.
Prerequisites #
For this tutorial, you’ll need:
- A development environment with
curl
and a supported Python 3 installation. - A Prime Intellect account with access to Hosted Training and billing configured.
- Internet access for the Prime CLI, environment package, hosted model, and adapter deployment.
- About 5-15 minutes.
Prime Intellect handles the hosted rollout, training, and inference infrastructure; you don’t need to manage a local GPU cluster.
Note: Model availability and pricing change over time. Treat the live CLI catalog as the source of truth rather than copying an identifier or price from a static article.
Customize Nemotron 3 Nano in three steps #
Let’s get into the simple steps required to customize your very own Nemotron 3 Nano.
Step 1: Get a Baseline
First, install the Prime CLI, authenticate, and create a Prime Intellect Lab workspace:
curl -LsSf https://astral.sh/uv/install.sh | sh
uv python install 3.13
uv tool install --python 3.13 --force "prime==0.6.17"
uv --version
prime --version
prime login
mkdir -p nemotron-customization
cd nemotron-customization
prime lab setup
uv run python --version
Prime Intellect Lab creates the workspace structure and installs the verifiers tooling. The longer-running evaluation and training steps come next.
Inspect the environment before using it:
prime env info primeintellect/math-python --version 0.1.10
prime env inspect primeintellect/math-python@0.1.10 README.md
Pin the exact environment package in the workspace:
uv add "math_python==0.1.10" \
--index https://hub.primeintellect.ai/primeintellect/simple/
uv lock --check
Now check the live hosted training catalog:
prime train models
prime inference models
Checkpoint: Continue only if the output contains nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16
and shows the current training, input, and output prices.
Use the following command to grab a quick baseline on our task so we can show if the model improved after training:
export NANO="nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
export MATH_EVAL_ARGS='{"dataset_name":"math","dataset_split":"test","num_train_examples":-1,"max_turns":5,"sandbox_client_max_workers":8}'
export MATH_SAMPLING='{"max_tokens":2048,"temperature":1.0,"extra_body":{"chat_template_kwargs":{"enable_thinking":false}}}'
prime eval run math-python \
--provider prime \
--model "$NANO" \
--num-examples 32 \
--rollouts-per-example 1 \
--max-concurrent 8 \
--env-args "$MATH_EVAL_ARGS" \
--sampling-args "$MATH_SAMPLING" \
--timeout 300 \
--max-retries 2 \
--save-results \
--disable-tui \
--abbreviated-summary
Your results should look similar to the following, which showcases that the model starts off doing rather poorly and has a fairly low accuracy.
...
--- All ---
Rewards:
reward: avg - 0.219, std - 0.413 ...
This is an example of a failing prompt for our baseline model, in which the model fails to produce a boxed response because it exhausted five turns calling unsupported tools:
Use Python for all calculations. Give your answer inside \boxed{}.
In addition to the Python standard library, you have access to: numpy sympy scipy.
If $2^8=4^x$, what is the value of $x$?
[[EMPTY RESPONSE]]
Figure 1 is an example of our reward distribution on this initial evaluation. The reward is binary: 1 means the model produced a correct response, and 0 means the model produced an incorrect or missing response.
Step 2: Take Nemotron 3 Nano to school
To get started on customizing Nemotron 3 Nano, the first thing we’ll do is create configs/rl/nemotron-3-nano-math-python-100step.toml
:
name = "nemotron-3-nano-math-python-100step"
model = "nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16"
loss = "rl"
max_steps = 100
batch_size = 32
rollouts_per_example = 8
max_inflight_rollouts = 32
learning_rate = 2e-5
lora_alpha = 16
[sampling]
max_tokens = 2048
temperature = 1.0
enable_thinking = false
[adapters]
interval = 0
keep_last = 1
[[env]]
id = "primeintellect/math-python@0.1.10"
args = { dataset_name = "math", dataset_split = "train", num_train_examples = -1, max_turns = 5 }
This config outlines how we want to train our model. The trainer samples eight attempts for each selected task, giving it a comparative signal inside each rollout group. Four groups form a batch of 32, and caps the number of rollouts being assembled concurrently for this small run.
Launch the run:
prime train configs/rl/nemotron-3-nano-math-python-100step.toml
Prime prints the parsed configuration, current price, and environment action status before asking for confirmation. After you confirm, the CLI returns a run ID and a dashboard URL. Evaluation and training continue after that point.
Checkpoint: Save the run ID. The launch output should show the exact Nano model, the pinned Math Python environment, live pricing, and a successful environment action check.
The dashboard shows reward curves, reward distributions, and individual rollouts. The CLI exposes the same evidence:
export RUN_ID="<run-id>"
export STEP="<step>"
prime train logs "$RUN_ID" --follow
prime train progress "$RUN_ID"
prime train distributions "$RUN_ID" --type rewards
prime train usage "$RUN_ID" --watch
prime train rollouts "$RUN_ID" --step "$STEP"
prime train checkpoints "$RUN_ID"
When experimenting, begin with small variations. If every attempt receives the same score, the trainer has little signal to work with. Next, check a high-, middle-, and low-reward rollout and ask whether the score reflects genuinely better math problem-solving. Finally, look at factors like token use and truncation to ensure the configuration is not interfering with the training signal.
If the reward curve tells us if the model got more or less reward; the rollouts help tell you why.
Step 3: Check how much Nemotron 3 Nano learned
Training reward is like measuring the model’s performance on homework. It’s not real until you give it a test.
When the run completes successfully, Prime Intellect Lab uploads the final LoRA adapter. Confirm that the training artifact is ready, and then inspect its deployment state:
prime deployments list
A ready adapter that isn’t currently serving can appear as NOT_DEPLOYED
. After deployment is requested, its serving state moves through DEPLOYING
to DEPLOYED
.
Checkpoint: Continue only after the final adapter exists and is ready to deploy. A completed trainer process without a usable adapter isn’t the artifact this tutorial promises.
You can download the adapter from the training dashboard. To query it through hosted inference, deploy it:
prime deployments list
prime train checkpoints "$RUN_ID" --status READY
export ADAPTER_ID="<adapter-id>"
export ADAPTED_MODEL="$NANO:$ADAPTER_ID"
prime deployments create "$ADAPTER_ID" --plain
COLUMNS=300 NO_COLOR=1 prime deployments list --plain
Deployment creates a billable inference endpoint. Once its status is DEPLOYED
, the model identifier combines Nano with the adapter ID:
nvidia/NVIDIA-Nemotron-3-Nano-30B-A3B-BF16:<adapter-id>
Now repeat the baseline command, changing only the model identifier
Keep the environment, task count, rollout count, sampling settings, and reward unchanged. Changing more than the model makes the comparison harder to interpret.
prime eval run math-python \
--provider prime \
--model "$ADAPTED_MODEL" \
--num-examples 32 \
--rollouts-per-example 1 \
--max-concurrent 8 \
--env-args "$MATH_EVAL_ARGS" \
--sampling-args "$MATH_SAMPLING" \
--timeout 300 \
--max-retries 2 \
--save-results \
--disable-tui \
--abbreviated-summary
Your results should contain the following, which showcases the same evaluation configuration used above—showing an increase in the average reward as compared to our baseline model on the same example set—this result shows that our model has learned to be meaningfully better at our task:
...
--- All ---
Rewards:
reward: avg - 0.906, std - 0.291
...
Here’s the same prompt the initial model was unable to correctly solve:
Use Python for all calculations. Give your answer inside \boxed{}.
In addition to the Python standard library, you have access to: numpy sympy scipy.
If $2^8=4^x$, what is the value of $x$?
sp.solve(sp.Eq(2**8, 4** x), x)
... # Assistant:
We can rewrite the equation using the same base.
\[
2^8 = 4^x \quad\Longrightarrow\quad 2^8 = (2^2)^x = 2^{2x}.
\]
Since the bases are equal, the exponents must be equal:
\[
8 = 2x \;\Longrightarrow\; x = \frac{8}{2}=4.
\]
\[
\boxed{4}
\]
Figure 3 shows the reward distribution on the same set of evaluation prompts we saw before, and as you can see, the model can get a significantly higher number of them correct:
On the same 32 held-out problems, accuracy increased from 21.9% (7/32) to 90.6% (29/32), an absolute increase of 68.75 percentage points. Of the 24 prompts that had different outcomes between baseline and final, 23 improved, and one regressed. Additionally, Nemotron 3 Nano got better at this task with a total spend of less than $5.
To remove the adapter deployment, you can issue the following command as cleanup:
prime deployments delete "$ADAPTER_ID" --plain
See the Prime Intellect guide to deploying LoRA adapters for inference for deployment states and OpenAI-compatible inference examples.
Training larger models with Prime Intellect Lab: #
Now that we’ve gone through an example of how we can use Prime Intellect Lab to run a short training of Nemotron 3 Nano, you may be asking yourself: “What if I wanted to train larger models? Is it much harder?” The answer is: not at all. With the convenience of hosted training, we can simply modify the model identifier and run the exact same workflow using the following:
Train Nemotron 3 Super
Run the live catalog check again:
prime train models
Confirm availability and current pricing, then replace the model line with:
model = "nvidia/NVIDIA-Nemotron-3-Super-120B-A12B-BF16"
Keep the same experimental discipline: baseline first, inspect reward variation, train, and retest under unchanged settings.
Train Nemotron 3 Ultra
Nemotron 3 Ultra is the high-capacity option in the family, and Prime Intellect has support for hosted reinforcement learning on NVIDIA Blackwell infrastructure.
Verify you have access to managed training of Ultra using the following command:
prime train models
If you see it, replace the model line with:
model = "nvidia/NVIDIA-Nemotron-3-Ultra-550B-A55B-NVFP4"
Begin customizing Nemotron 3 #
Customization is how to custom-tailor AI for our use cases. Without customization, we’re limited to whatever behavior a model has out of the box, and we may overpay for a more broadly capable model when a lower-cost model could be adapted to the task we need.
Open models such as NVIDIA Nemotron 3, combined with hosted training platforms such as Prime Intellect Lab, make that process much easier to start. The most valuable part isn’t simply that the weights are available or that the infrastructure is managed. It’s that the entire improvement loop can be inspected and repeated.
Visit the Nemotron developer page for resources to get started. Explore open Nemotron models and datasets on Hugging Face and Blueprints on build.nvidia.com.
Engage with Nemotron livestreams, tutorials, and the developer community on the NVIDIA forum and the Nemotron channel on Discord.
Stay up to date on NVIDIA Nemotron by subscribing to NVIDIA news and following NVIDIA AI on LinkedIn, X, Discord, and YouTube.