# What can silently go wrong between an SFT JSONL file and the loss tensor?

> Source: <https://discuss.huggingface.co/t/what-can-silently-go-wrong-between-an-sft-jsonl-file-and-the-loss-tensor/191446#post_2>
> Published: 2026-11-05 05:20:21+00:00

# 

When an LLM fine-tuning job crashes with a CUDA OOM or produces NaNs, the failure is obvious. The more insidious problems happen when a run completes cleanly with exit code 0, the loss curve descends predictably, but the loss tensor is not optimizing what you intended.

Between raw training records on disk and the computed loss tensor on GPU, a training framework executes a multi-stage transformation:

```
Raw records → tokenizer/template → collator → model forward
            → shifted labels → CrossEntropyLoss
```

Because PyTorch only requires compatible tensor shapes, data and masking defects can pass without exceptions. Here are three silent failure modes across that boundary—and how to audit them before spending GPU hours.

## 

When an instruction sample exceeds `max_seq_len`, right-side truncation clips the tail. If a verbose prompt consumes the token budget, the assistant completion can be partially or completely removed.

When the completion is fully dropped, every label in the sequence becomes `-100`. The sample contributes zero supervised loss while still consuming sequence budget in the batch.

Depending on the chat template, empty completions can similarly result in delimiter-only supervision, effectively training the model to emit closing tokens immediately.

## 

In Direct Preference Optimization (DPO), a row with identical chosen and rejected completions (`chosen == rejected`) has an implicit reward margin of zero: \Delta r = 0.

Under the deterministic DPO objective, this yields no learning signal. Forward and backward passes still execute, but the pair does not express a preference. Stochastic components such as dropout can introduce residual noise, but do not make such pairs useful preference data.

## 

In causal SFT, the intended objective is to condition on the prompt and supervise completion tokens. If a custom collator or template integration fails to mask prompt positions with `labels = -100`, cross-entropy is calculated over both prompt and answer tokens.

The loss will still decrease, but the objective has changed: the model is also trained to reproduce prompt syntax rather than only generate the intended response.

## 

I built **TrainSight** (`trainsight-preflight`), an Apache-2.0 diagnostic package for validating these training contracts. It works at two explicit checkpoints.

### 

TrainSight audits JSONL datasets—and Parquet datasets when `pyarrow` is installed—against the selected tokenizer and chat template.

```
pip install trainsight-preflight

# Check an SFT dataset for truncation starvation and empty completions
trainsight check data/train_sft.jsonl \
  --tokenizer Qwen/Qwen2.5-0.5B-Instruct \
  --max-seq-len 2048

# Check a DPO dataset for identical pairs and length bias
trainsight check data/train_dpo.jsonl \
  --task dpo \
  --tokenizer Qwen/Qwen2.5-0.5B-Instruct
```

### 

For custom collators or training pipelines, TrainSight audits compiled PyTorch batches. It checks that prompt and padding tokens are masked, and that each active label position (i \ge 1) can contribute to the shifted causal-loss objective.

``` python
from transformers import AutoTokenizer
from trainsight import SFTReconciler, LossContract

tokenizer = AutoTokenizer.from_pretrained("Qwen/Qwen2.5-0.5B-Instruct")

contract = LossContract.from_tokenizer(
    tokenizer,
    max_seq_len=512,
    strict=False,
)

reconciler = SFTReconciler(
    tokenizer=tokenizer,
    contract=contract,
)

# `prompt_lengths` must be derived from the same template/collator path.
report = reconciler.audit_batch(
    batch=batch,
    prompt_lengths=prompt_lengths,
    row_ids=row_ids,
    raw_samples=raw_samples,
)

assert report.passed, report.diagnoses
```

TrainSight complements existing training frameworks—including TRL, Hugging Face `transformers`, and native PyTorch loops—by making the relevant loss-contract checks explicit before and during execution.

## 

I would value input from practitioners here:

1. What silent data or loss-masking edge cases have you encountered in SFT or DPO runs that standard validation missed?
2. How does your team audit collators and template boundaries before launching expensive multi-GPU or multi-node training jobs?