{"slug": "what-can-silently-go-wrong-between-an-sft-jsonl-file-and-the-loss-tensor", "title": "What can silently go wrong between an SFT JSONL file and the loss tensor?", "summary": "An Apache-2.0 diagnostic package called TrainSight (trainsight-preflight) was released to catch silent fine-tuning failures that occur between raw JSONL training records and the computed loss tensor, including truncation that drops assistant completions entirely, DPO rows with identical chosen and rejected completions (implicit reward margin Δr = 0), and collators that fail to mask prompt positions with labels = -100. TrainSight audits JSONL and Parquet datasets against a chosen tokenizer and chat template via `trainsight check`, and audits compiled PyTorch batches through an SFTReconciler and LossContract API that verifies prompt and padding tokens are masked and each active label position (i ≥ 1) can contribute to the shifted causal-loss objective. The package is positioned as a complement to TRL, Hugging Face transformers, and native PyTorch training loops, making loss-contract checks explicit before and during execution.", "body_md": "# \n\nWhen an LLM fine-tuning job crashes with a CUDA OOM or produces NaNs, the failure is obvious. The more insidious problems happen when a run completes cleanly with exit code 0, the loss curve descends predictably, but the loss tensor is not optimizing what you intended.\n\nBetween raw training records on disk and the computed loss tensor on GPU, a training framework executes a multi-stage transformation:\n\n```\nRaw records → tokenizer/template → collator → model forward\n            → shifted labels → CrossEntropyLoss\n```\n\nBecause PyTorch only requires compatible tensor shapes, data and masking defects can pass without exceptions. Here are three silent failure modes across that boundary—and how to audit them before spending GPU hours.\n\n## \n\nWhen an instruction sample exceeds `max_seq_len`, right-side truncation clips the tail. If a verbose prompt consumes the token budget, the assistant completion can be partially or completely removed.\n\nWhen the completion is fully dropped, every label in the sequence becomes `-100`. The sample contributes zero supervised loss while still consuming sequence budget in the batch.\n\nDepending on the chat template, empty completions can similarly result in delimiter-only supervision, effectively training the model to emit closing tokens immediately.\n\n## \n\nIn Direct Preference Optimization (DPO), a row with identical chosen and rejected completions (`chosen == rejected`) has an implicit reward margin of zero: \\Delta r = 0.\n\nUnder the deterministic DPO objective, this yields no learning signal. Forward and backward passes still execute, but the pair does not express a preference. Stochastic components such as dropout can introduce residual noise, but do not make such pairs useful preference data.\n\n## \n\nIn causal SFT, the intended objective is to condition on the prompt and supervise completion tokens. If a custom collator or template integration fails to mask prompt positions with `labels = -100`, cross-entropy is calculated over both prompt and answer tokens.\n\nThe loss will still decrease, but the objective has changed: the model is also trained to reproduce prompt syntax rather than only generate the intended response.\n\n## \n\nI built **TrainSight** (`trainsight-preflight`), an Apache-2.0 diagnostic package for validating these training contracts. It works at two explicit checkpoints.\n\n### \n\nTrainSight audits JSONL datasets—and Parquet datasets when `pyarrow` is installed—against the selected tokenizer and chat template.\n\n```\npip install trainsight-preflight\n\n# Check an SFT dataset for truncation starvation and empty completions\ntrainsight check data/train_sft.jsonl \\\n  --tokenizer Qwen/Qwen2.5-0.5B-Instruct \\\n  --max-seq-len 2048\n\n# Check a DPO dataset for identical pairs and length bias\ntrainsight check data/train_dpo.jsonl \\\n  --task dpo \\\n  --tokenizer Qwen/Qwen2.5-0.5B-Instruct\n```\n\n### \n\nFor custom collators or training pipelines, TrainSight audits compiled PyTorch batches. It checks that prompt and padding tokens are masked, and that each active label position (i \\ge 1) can contribute to the shifted causal-loss objective.\n\n``` python\nfrom transformers import AutoTokenizer\nfrom trainsight import SFTReconciler, LossContract\n\ntokenizer = AutoTokenizer.from_pretrained(\"Qwen/Qwen2.5-0.5B-Instruct\")\n\ncontract = LossContract.from_tokenizer(\n    tokenizer,\n    max_seq_len=512,\n    strict=False,\n)\n\nreconciler = SFTReconciler(\n    tokenizer=tokenizer,\n    contract=contract,\n)\n\n# `prompt_lengths` must be derived from the same template/collator path.\nreport = reconciler.audit_batch(\n    batch=batch,\n    prompt_lengths=prompt_lengths,\n    row_ids=row_ids,\n    raw_samples=raw_samples,\n)\n\nassert report.passed, report.diagnoses\n```\n\nTrainSight complements existing training frameworks—including TRL, Hugging Face `transformers`, and native PyTorch loops—by making the relevant loss-contract checks explicit before and during execution.\n\n## \n\nI would value input from practitioners here:\n\n1. What silent data or loss-masking edge cases have you encountered in SFT or DPO runs that standard validation missed?\n2. How does your team audit collators and template boundaries before launching expensive multi-GPU or multi-node training jobs?", "url": "https://wpnews.pro/news/what-can-silently-go-wrong-between-an-sft-jsonl-file-and-the-loss-tensor", "canonical_source": "https://discuss.huggingface.co/t/what-can-silently-go-wrong-between-an-sft-jsonl-file-and-the-loss-tensor/191446#post_2", "published_at": "2026-11-05 05:20:21+00:00", "updated_at": "2026-11-05 05:23:52.458321+00:00", "lang": "en", "topics": ["ai-tools", "machine-learning", "large-language-models", "developer-tools"], "entities": ["TrainSight", "trainsight-preflight", "PyTorch", "TRL", "Hugging Face", "Qwen2.5-0.5B-Instruct"], "also_reported_by": [], "alternates": {"html": "https://wpnews.pro/news/what-can-silently-go-wrong-between-an-sft-jsonl-file-and-the-loss-tensor", "markdown": "https://wpnews.pro/news/what-can-silently-go-wrong-between-an-sft-jsonl-file-and-the-loss-tensor.md", "text": "https://wpnews.pro/news/what-can-silently-go-wrong-between-an-sft-jsonl-file-and-the-loss-tensor.txt", "jsonld": "https://wpnews.pro/news/what-can-silently-go-wrong-between-an-sft-jsonl-file-and-the-loss-tensor.jsonld"}}