Castform now supports supervised finetuning

today we’re adding supervised finetuning to castform.

our goal’s to give developers everything necessary to post-train models for their tasks. reinforcement learning is all the rage these days (and castform makes it super easy to use), but supervised finetuning remains one of the most compute-efficient ways to teach a model exactly how you want it to behave.

while rl learns from a score/reward, sft learns from correct demonstration. specifically, we train the model via next-token prediction to replicate tokens in the demonstration exactly.

why sft #

sft is particularly useful in cases where you have clear expert outputs & ground truths. it’s also much cheaper to run than rl.

some use-cases:

classification and structured extraction. examples include routing, intent classification, field extraction. there is a clear correct answer & you likely have 1000s of labeled samples. finetuning a small model will help you match a much larger one, at a fraction of the cost & latency.distillation from a larger model. generate traces from a frontier model on your task, then train a smaller one on them. for simpler tasks, you can match quality and get a model that’s cheaper to serve and fast enough to sit in a hot path.warm-start for rl. reinforcement learning needs the model to get to a non-zero score to be able to reinforce useful behaviors. for more complex tasks, base models might not be able to get there. sft on expert traces/ground truths can help move it to the right part of the distribution first, so rl can work its magic.

how to sft with castform #

you can run an sft job in < 10 lines of code with castform. all you need to do is create an sft dataset. this dataset is basically a list of chat rows, and the model will be trained via next-token prediction to imitate all the messages with the role assistant

in a given row.

from benchmax.sft import SftDataset
from castform.platform import SftTrainingConfig, TrainerClient, upload_sft_assets

rows = [
    {
        "messages": [
            {"role": "system", "content": "Replace personal information with typed placeholders."},
            {"role": "user", "content": "Contact Jane Doe at jane@example.com."},
            {"role": "assistant", "content": "Contact [GIVENNAME] [SURNAME] at [EMAIL]."},
        ]
    },
]

dataset = SftDataset.from_rows(rows)
assets = upload_sft_assets(dataset=dataset, run_name="pii-masking")
run_id = TrainerClient().launch_sft_run(
    assets=assets,
    name="pii-masking",
    config=SftTrainingConfig(),
)

example use-case: pii masking #

a good use-case for sft is pii-masking, where the goal is to train a model to detect all personally identifiable information (emails, ssns, names, etc.) and replace it with a typed placeholder. pii masking has one right answer. every name, email, etc. should be replaced with the placeholder, and the rest of the text should be unchanged. it’s also very easy to collect/synthesize training data for this task.

to demonstrate this use-case, we trained qwen3.5-4b on the openpii 1m dataset (ai4privacy / ai suisse sa, cc by 4.0). you can look at the full castform code example here, and the training run with castform.

after only 3,000 training examples, our finetuned 4b model beats an untrained frontier model on a held out validation set. we cut leak rate from 40% to 3.4%.

leak rate is the share of documents where any pii got through, and f1 is a proxy for measuring catching pii vs over-masking clean text. our precision was 0.998, so we know it isn’t just redacting everything.

this is one task on one model, and pii masking is about as clean as supervised targets get. tasks with fuzzier notions of “correct” are where rl starts to earn its cost back.

your turn #

you can get started with sft today. to setup castform, run uv tool install -U castform

and castform setup

. you can prompt your way with the coding agent of your choice to get started.