# Post-Training NVIDIA Nemotron to Increase Accuracy in Your Scientific Domain

> Source: <https://developer.nvidia.com/blog/post-training-nvidia-nemotron-to-increase-accuracy-in-your-scientific-domain/>
> Published: 2026-11-16 16:00:00+00:00

Agentic AI is reshaping scientific research and innovation. Researchers communicate complex scientific hypotheses in natural language and let autonomous agents independently plan tasks, execute simulations, evaluate results, and self-correct across multiple iterations. Scientific agents are taking on longer, more complex, and increasingly visual workflows.

Accuracy and inference efficiency matter at every step. Without verification, errors compound across the workflow, while latency and compute costs multiply with continued operation.

NVIDIA Nemotron for Science, a collection of models, datasets, tools, and recipes for accelerating agentic scientific workloads, addresses these hurdles. It includes [Nemotron 3.5 Super VL](https://developer.nvidia.com/blog/nvidia-nemotron-3-5-super-vl-powers-specialized-agents-with-multimodal-reasoning/)—an open, [customizable, multimodal mode](https://huggingface.co/nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A12B-BF16)l for these specialized workflows—and the NVIDIA Nemotron [Litmus-Bench V2.0](https://huggingface.co/datasets/nvidia/Nemotron-RL-litmus-bench-v0.2), an open source chemistry-reasoning dataset and benchmark that was designed to train and evaluate LLMs and agentic AI models on molecular properties. Out of the box, Nemotron delivers frontier level accuracy for scientific benchmarks (see Figure 1 below), along with the fastest time to solution for scientific coding.

An important benefit of Nemotron models is NVIDIA post-training support. You can customize models for your specific research domain to increase accuracy.

This blog provides a practical recipe for post-training Nemotron 3.5 Super VL for scientific use cases. Starting from a raw experimental dataset, you’ll walk through how to build a custom reinforcement learning (RL) environment using NVIDIA [NeMo Gym](https://github.com/NVIDIA-NeMo/Gym), apply RL, and measure improvement using molecular similarity scoring. NeMo Gym supplies the environment infrastructure, and NVIDIA NeMo RL supplies the post-training framework. 

This guide uses drug-metabolism prediction as a practical example. Given a parent compound, the starting molecule, and its transforming enzyme, the model predicts the resulting metabolite. The examples below use records from MetxBioDB, which is a comprehensive database of experimentally confirmed biotransformation reactions.

The blog first describes direct RL, then an optional distillation step that can be used before RL when a teacher model provides useful examples.

## Build the reinforcement learning environment from experimental data

The starting point is a CSV file of existing experimental records such as sequences, Simplified Molecular Input Line Entry System (SMILES) strings, reaction conditions, and any other metadata paired with measured properties or outcomes. In this example, molecular structures are represented as SMILES strings, a text notation for describing a molecule’s atoms and bonds.

These input-output pairs are reformatted as question-answer or user/assistant prompt pairs, with the experimental ground truth serving as a verifier to score the model’s responses. This setup constitutes a basic reinforcement learning with verifiable rewards (RLVR) environment and requires no human preference labelling: The reward signal comes directly from the data.

The fidelity of that reward signal can be tuned to the task. A simple sequence-similarity function works as a lightweight verifier, while more sophisticated tools like cheminformatics libraries, molecular simulators, or even a live autonomous lab can be a tool call and can all be wired in as the environment’s scoring layer. The richer the tool, the more precisely the model is steered toward chemically or biologically meaningful outputs.

For the metabolism example, the model receives the parent molecule and enzyme, then submits a predicted metabolite. The verifier compares the predicted and reference molecular fingerprints—numerical representations of structural features—using Tanimoto similarity. Tanimoto similarity is a metric used to measure the overlap between two sets or binary vectors and is calculated by dividing the size of their intersection by the size of their union. For the bit fingerprints used here, the score ranges from 0 to 1; higher scores indicate more similar fingerprints.

## Direct RL post-training on the custom environment

With the environment in place, fine-tune the model using group relative policy optimization (GRPO) with low rank adapters (LoRA). The environment presents the model with an experimental input and asks it to reason through the problem step-by-step before submitting a predicted answer via a tool call. Each submission is scored automatically by the verifier, giving a dense reward signal that requires no further annotation. The training loop, reward server, harness, and rollout workers are fully self-contained, so the same pipeline generalizes to any domain that can return a scalar reward.

At a 16,384-token output budget on the drug-metabolism RLVR environment, supervised fine tuning (SFT) improved the accuracy of Nemotron for Science by 12% and RL post-trained lifted it a further 17%. It demonstrates that even a highly capable instruction-tuned model continues to improve meaningfully when given targeted RL feedback on a domain-specific task.

## Distillation from stronger models as a “warm-start” for RL

If the base model needs a capability boost before RL training or if RL convergence is slow, a distillation step can provide significant lift. A stronger teacher model (e.g. a larger frontier model or a specialist) is run against the same environment to generate high-quality reasoning traces. A reasoning trace is high quality if the teacher model’s step-by-step logic for identifying the enzyme, predicting the reaction site, and deriving the metabolite structure leads to a SMILES that matches the ground truth metabolite. It is a low-quality reasoning trace even if the reasoning looks fluent and plausible but has some flawed logic that resulted in the wrong prediction.

The high-quality reasoning traces are used to fine-tune the student model via SFT or direct on-policy distillation, giving it a much better starting point before the RL loop begins. This is known as a “warm-start” and it typically accelerates convergence and raises the performance floor, making the subsequent RL training more efficient and more likely to find a strong policy.

### Data distillation for SFT

The distillation pipeline runs Anthropic’s Claude Sonnet 5.5 across the training set, scores each rollout by Tanimoto similarity (tool call) to the ground truth metabolite, and keeps only the (question, reasoning trace) pairs where the model got it right. Those become the SFT corpus for the smaller model.

In this environment 1,123 high-quality reasoning traces were filtered from 1,934 training prompts at Tanimoto threshold ≥ 0.7. Here is a distilled reasoning trace for a question-answer pair from Claude Sonnet 5.5. Note that a fixed system prompt is included with every request, which can be varied across environments. LLM requests are formatted as chat messages containing both the question and the system prompt.

#### MetxBioDB environment (CYP1A2 N-demethylation)

Question:

```
Parent compound SMILES: Cc1nn(C)c(Cl)c1C(O)=Nc1cccc2c1C(C)OC2(C)C Metabolising enzyme: CYP1A2 Reason through this biotransformation step by step, then call submit_metabolite with the predicted SMILES of the primary metabolite.
```

Ground truth:

```
Cc1[nH]nc(Cl)c1C(O)=Nc1cccc2c1C(C)OC2(C)C — Tanimoto similarity = 1.0 (exact match)
```

System prompt:

```
You are an expert computational chemist specializing in drug metabolism. Your task is to predict the SMILES string of the metabolite produced when a given parent compound is biotransformed by a specified enzyme or enzyme family. Response format: 1. Provide step-by-step scientific reasoning: a. Identify the functional groups in the parent compound relevant to the enzyme. b. Describe the enzyme's known reaction mechanism and regioselectivity. c. Identify the specific site of biotransformation and explain why. d. Describe the chemical transformation applied (e.g. hydroxylation, glucuronidation). e. State the resulting structural change to the parent SMILES. 2. Finish by calling the submit_metabolite tool with your predicted metabolite SMILES. Rules: - The SMILES passed to submit_metabolite must be valid, canonical, and represent the chemically correct metabolic product — no extra text in the argument. - Apply knowledge of the named enzyme's typical reaction mechanism (e.g. CYP450 hydroxylation/oxidation, UGT glucuronidation, SULT sulfation, MAO deamination, NAT acetylation, etc.). - If multiple enzymes are listed (semicolon-separated), use their combined or most likely reaction on the substrate.
```

Reasoning trace (Claude Sonnet 5.5):

```
a. Functional groups relevant to CYP1A2: The parent contains an N-methyl pyrazole (N–CH3 on the ring nitrogen). CYP1A2 is well known for N-demethylation of aromatic amines and N-alkyl heterocycles via alpha-carbon hydroxylation followed by hemi-aminal collapse. b. Enzyme mechanism: CYP1A2 abstracts a hydrogen from the N-methyl carbon (N–CH3 → N–CH2•), then oxygen rebound gives N–CH2OH; the hemi-aminal spontaneously collapses to the free N–H and formaldehyde. c. Site of biotransformation: The 1-methyl (N1-CH3) on the pyrazole is the primary target. The C3-methyl is on a carbon, not nitrogen, and is far less reactive. d. Chemical transformation: N-Demethylation: –N(CH3)– → –NH–, loss of CH2O. e. Structural change to SMILES: Original N-methyl: Cc1nn(C)c(Cl)c1… ← the nn(C) nitrogen loses its CH3 Product: Cc1[nH]nc(Cl)c1… ← becomes [nH]
```

Tool call:

```
submit_metabolite({"smiles": "Cc1[nH]nc(Cl)c1C(O)=Nc1cccc2c1C(C)OC2(C)C"})
```

### Getting started with code

The recipes below are drawn from the [nemo-rl](https://github.com/NVIDIA-NeMo/RL) and [nemo-gym](https://github.com/NVIDIA-NeMo/Gym) training modules. Each snippet is self-contained and can be adapted to a new domain by replacing the corresponding configs.

#### NemoGym environment registration

The following snippets register a NemoGym RLVR environment from an experimental CSV. Registering the environment takes four pieces.

1. CSV to Gym JSONL

Each CSV row becomes one JSONL row. The extra keys, `ground_truth_smiles` here, are passed to the server’s `seed_session` and verified.

``` python
import csv
import json

SYSTEM = open("system_prompt.txt").read()
TOOLS = [{
    "type": "function",
    "name": "submit_metabolite",
    "strict": True,
    "description": (
        "Submit your predicted metabolite SMILES. Returns validity and Tanimoto"
        " similarity (0-1)."
    ),
    "parameters": {
        "type": "object",
        "additionalProperties": False,
        "required": ["smiles"],
        "properties": {
            "smiles": {
                "type": "string",
                "description": "Predicted metabolite SMILES (no extra text).",
            }
        },
    },
}]

with open("train.jsonl", "w") as f:
    for r in csv.DictReader(
        open("metxbiodb.csv")
    ):  # columns: parent_smiles, enzyme, reaction_type, metabolite_smiles
        user = (
            f"Parent compound SMILES: {r['parent_smiles']}\nMetabolising"
            f" enzyme: {r['enzyme']}\n\nReason through this biotransformation"
            " step by step, then call submit_metabolite with the predicted"
            " SMILES of the primary metabolite."
        )
        f.write(
            json.dumps({
                "responses_create_params": {
                    "input": [
                        {"role": "system", "content": SYSTEM},
                        {"role": "user", "content": user},
                    ],
                    "tools": TOOLS,
                },
                "agent_ref": {"name": "metxbiodb_simple_agent"},
                "parent_smiles": r["parent_smiles"],
                "enzyme": r["enzyme"],
                "reaction_type": r["reaction_type"],
                "ground_truth_smiles": r["metabolite_smiles"],
            })
            + "\n"
        )
```

2. Resources server: resources_servers/metxbiodb/app.py

This is the only domain-specific code, and it has three parts:

- `Seed_session` stores the ground truth per session.
- The `submit_metabolite` tool endpoint scores the first submission with Morgan-fingerprint Tanimoto (radius 2, 2048 bits).
- `Verify` returns that score as the reward.

It also needs a `pyproject.toml` with `rdkit`. To adapt it to a new domain, replace `tanimoto_similarity` and the tool schema.

3. Registration [yaml](http://resources_servers/metxbiodb/configs/metxbiodb.yaml)

```
metxbiodb:
  resources_servers:
    metxbiodb:
      entrypoint: app.py
      domain: knowledge
metxbiodb_simple_agent:
  responses_api_agents:
    simple_agent: 
      entrypoint: app.py    
      max_steps: 2  # submit, then one closing turn; only the first submission is scored
      resources_server: {type: resources_servers, name: metxbiodb}
      model_server:     {type: responses_api_models, name: policy_model}
      datasets:
        - {name: train,      type: example, jsonl_fpath: data/train.jsonl}
        - {name: validation, type: example, jsonl_fpath: data/validation.jsonl}
```

The agent name must equal `agent_ref.name` in the JSONL rows.

4. Wiring in the trainer config

```
data:
  train:      {dataset_name: NemoGymDataset, data_path: data/train.jsonl}
  validation: {data_path: data/validation.jsonl}
  default:    {dataset_name: NemoGymDataset, env_name: nemo_gym, processor: nemo_gym_data_processor,
               prompt_file: null, system_prompt_file: null}
  nemo_gym:    
    metxbiodb_simple_agent:
      datasets:
        train:      {jsonl_fpath: data/train.jsonl}
        validation: {jsonl_fpath: data/validation.jsonl}
env:    
  should_use_nemo_gym: true
  nemo_gym:
    config_paths:
      - responses_api_models/vllm_model/configs/vllm_model_for_training.yaml
      - resources_servers/metxbiodb/configs/metxbiodb.yaml
```

#### GRPO training launch

Use the following command to launch GRPO training:

```
# Multi-node GRPO run (16 GPUs)
python examples/nemo_gym/run_grpo_nemo_gym.py \
    --config configs/grpo_metxbiodb_super_vl.yaml \
    cluster.num_nodes=4 cluster.gpus_per_node=4 \
    grpo.max_num_steps=100
```

Note: `grpo_metxbiodb_super_vl.yaml` is the complete GRPO trainer config for the environment in this blog.

#### GRPO policy configuration

The following shows the policy portion of the GRPO trainer config. The full config includes rollout, reward server, and logging settings.

```
policy:
  model_name: nvidia/NVIDIA-Nemotron-3.5-Super-VL-120B-A128-BF16
  tokenizer:
    - name: $(policy.model_name)
  chat_template_kwargs: null
  hf_config_overrides:
    num_next_predict_layers: 0
  train_micro_batch_size: 1
  logprob_batch_size: 1
  logprob_chunk_size: 2048
  max_total_sequence_length: 5120
  make_sequence_length_divisible_by: 256
  precision: bfloat16
  offload_optimizer_for_logprob: true
  optimizer:
    - name: transformer_engine.pytorch.optimizers.fused_adam.FusedAdam
      kwargs:
        lr: 1.0e-06
        weight_decay: 0.1
        betas: [0.9, 0.999]
        eps: 1.0e-08
        master_weights: true
        store_param_remainders: true
        exp_avg_dtype: torch.bfloat16
        exp_avg_sq_dtype: torch.bfloat16
  scheduler:
    - name: torch.optim.lr_scheduler.ConstantLR
      kwargs:
        factor: 1.0
        total_iters: 1000000000
    - milestones: []
  automodel_cfg:
    enabled: true
    tensor_parallel_size: 1
    context_parallel_size: 1
    expert_parallel_size: 4
    sequence_parallel: false
    activation_checkpointing: true
    cpu_offload: false
    moe_parallelizer:
      ignore_router_for_ac: true
      reshard_after_forward: true
    automodel_kwargs:
      backend:
        _target_: nemo_automodel.components.models.common.utils.BackendConfig
        attn: te
        linear: te
        rms_norm: torch_fp32
        experts: gmm
        dispatcher: hybridep
      fake_balanced_gate: false
      rope_fusion: false
      enable_hf_state_dict_adapter: true
```

Note: Set `loss_fn.use_importance_sampling_correction=True` to reduce variance between the vLLM rollout engine and the Megatron training engine. This is particularly important for mixture-of-experts (MoE) models. 

### Distillation pipeline

The following snippets score teacher rollouts by Tanimoto similarity and filter traces for the SFT corpus based on the user set threshold for the scores.

The generate script and launch command look like the below:

```
export POLICY_API_KEY=<your-inference-api-key>

# Step 1: Generate rollouts from Claude Sonnet 5.5
python3 tools/generate_sft.py \
    --model anthropic/claude-sonnet-5-5 \
    --split train \
    --num-repeats 8 \
    --min-reward 0.7 \
    --out-dir runs/metxbiodb_sft_distilled

# Step 2: Build OpenAI-format SFT dataset
python3 tools/build_metxbiodb_sft_messages.py \
    --in-file runs/metxbiodb_sft_distilled/generated.jsonl \
    --out-dir metxbiodb/distilled_datasets \
    --val-frac 0.05 \
    --seed 42
```

Where `generate_sft.py` is:

``` python
async def generate(row, semaphore):
    async with semaphore:
        resp = await client.responses.create(
            model=model,
            instructions=SYSTEM_PROMPT,
            input=[{"role": "user", "content": row["prompt"]}],
            tools=TOOLS,
            temperature=1.0,
        )
    predicted_smiles = None
    response_text = getattr(resp, "output_text", "") or ""
    for item in getattr(resp, "output", []):
        if (
            getattr(item, "type", None) == "function_call"
            and getattr(item, "name", None) == "submit_metabolite"
        ):
            predicted_smiles = (
                json.loads(item.arguments).get("smiles", "").strip()
            )
            break
    valid, reward = (
        tanimoto(predicted_smiles, ground_truth)
        if predicted_smiles
        else (False, 0.0)
    )
    return {
        "id": row["id"],
        "prompt": row["prompt"],
        "response": response_text,
        "predicted_smiles": predicted_smiles,
        "reward": reward,
    }
```

Note**:** Filtering at `score == 1.0` (exact Tanimoto match) keeps only traces where the teacher’s reasoning was chemically logical and resulted in the exact growth truth as the final answer. For softer ground truth to introduce some variations, set a threshold (e.g., score >= 0.7) instead.

#### SFT trainer launch

Use the following command to launch SFT training:

```
# Multi-node SFT run (16 GPUs)
python examples/run_sft.py \
    --config configs/sft_metxbiodb_super_vl_lora.yaml \
    cluster.num_nodes=4 cluster.gpus_per_node=4 \
    sft.max_num_steps=55
```

Note: `sft_metxbiodb_super_vl_lora.yaml` is the complete SFT trainer config for the environment in this blog.

#### SFT trainer configuration

The following SFT trainer config warm-starts the student model on distilled traces before the RL loop begins:

```
policy:
  optimizer:
    kwargs:
      lr: 1.0e-5
      weight_decay: 0.0
  automodel_cfg:
    model_save_format: safetensors
    checkpoint:
      save_consolidated: final
    consolidation_timeout_minutes: 120
    lora_cfg:
      enabled: false
```

Post-training gives research teams a way to adapt Nemotron models to the tasks and experimental evidence that matter in your domain. This recipe turns experimental records into prompts and reference answers, connects model submissions to an automated verifier, and uses reinforcement learning to train against that feedback.

The central design choice is the reward: It should measure the scientific outcome you want. Teacher-model distillation offers an optional starting point, while held-out evaluation helps determine whether either training path improves performance beyond the starting model.

Together, domain-specific data, task-appropriate verification, and reproducible evaluation provide a practical foundation for developing specialized scientific agents. Start with a narrowly defined task, validate the scoring method, and expand the workflow only as the evidence supports it.

Get started by exploring [NeMo Gym](https://github.com/NVIDIA-NeMo/Gym) and [NeMo RL](https://github.com/NVIDIA-NeMo/RL), then use the accompanying [recipe](https://gitlab-master.nvidia.com/gdharuman/nemo-science) as an example to build and evaluate an environment for your own scientific task.
