#

When an LLM fine-tuning job crashes with a CUDA OOM or produces NaNs, the failure is obvious. The more insidious problems happen when a run completes cleanly with exit code 0, the loss curve descends predictably, but the loss tensor is not optimizing what you intended.

Between raw training records on disk and the computed loss tensor on GPU, a training framework executes a multi-stage transformation:

Raw records → tokenizer/template → collator → model forward
            → shifted labels → CrossEntropyLoss

Because PyTorch only requires compatible tensor shapes, data and masking defects can pass without exceptions. Here are three silent failure modes across that boundary—and how to audit them before spending GPU hours.

#

When an instruction sample exceeds max_seq_len, right-side truncation clips the tail. If a verbose prompt consumes the token budget, the assistant completion can be partially or completely removed.

When the completion is fully dropped, every label in the sequence becomes -100. The sample contributes zero supervised loss while still consuming sequence budget in the batch.

Depending on the chat template, empty completions can similarly result in delimiter-only supervision, effectively training the model to emit closing tokens immediately.

#

In Direct Preference Optimization (DPO), a row with identical chosen and rejected completions (chosen == rejected) has an implicit reward margin of zero: \Delta r = 0.

Under the deterministic DPO objective, this yields no learning signal. Forward and backward passes still execute, but the pair does not express a preference. Stochastic components such as dropout can introduce residual noise, but do not make such pairs useful preference data.

#

In causal SFT, the intended objective is to condition on the prompt and supervise completion tokens. If a custom collator or template integration fails to mask prompt positions with labels = -100, cross-entropy is calculated over both prompt and answer tokens.

The loss will still decrease, but the objective has changed: the model is also trained to reproduce prompt syntax rather than only generate the intended response.

#

I built TrainSight (trainsight-preflight), an Apache-2.0 diagnostic package for validating these training contracts. It works at two explicit checkpoints.

TrainSight audits JSONL datasets—and Parquet datasets when pyarrow is installed—against the selected tokenizer and chat template.

pip install trainsight-preflight

trainsight check data/train_sft.jsonl \
  --tokenizer Qwen/Qwen2.5-0.5B-Instruct \
  --max-seq-len 2048

trainsight check data/train_dpo.jsonl \
  --task dpo \
  --tokenizer Qwen/Qwen2.5-0.5B-Instruct

For custom collators or training pipelines, TrainSight audits compiled PyTorch batches. It checks that prompt and padding tokens are masked, and that each active label position (i \ge 1) can contribute to the shifted causal-loss objective.

from transformers import AutoTokenizer
from trainsight import SFTReconciler, LossContract

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")

contract = LossContract.from_tokenizer(
    tokenizer,
    max_seq_len=512,
    strict=False,
)

reconciler = SFTReconciler(
    tokenizer=tokenizer,
    contract=contract,
)

report = reconciler.audit_batch(
    batch=batch,
    prompt_lengths=prompt_lengths,
    row_ids=row_ids,
    raw_samples=raw_samples,
)

assert report.passed, report.diagnoses

TrainSight complements existing training frameworks—including TRL, Hugging Face transformers, and native PyTorch loops—by making the relevant loss-contract checks explicit before and during execution.

#

I would value input from practitioners here:

1. What silent data or loss-masking edge cases have you encountered in SFT or DPO runs that standard validation missed?
2. How does your team audit collators and template boundaries before launching expensive multi-GPU or multi-node training jobs?