Turkey (Turkish)
Germany (German)
Worldwide (English)
With TRL `assistant_only_loss=True`, loss is computed only on assistant tokens and the chat template must produce a valid assistant mask.
Halkali merkez neighborhood fatih st ozgur apt no 46 , Kucukcekmece , Istanbul , 34303 , TR
Diagnose TRL SFTTrainer loss=0.0, NaN and eval_loss NaN using dataset format, -100 labels, truncation, assistant_only_loss, learning rate and precision.
With TRL `assistant_only_loss=True`, loss is computed only on assistant tokens and the chat template must produce a valid assistant mask.
If assistant tokens are truncated beyond max length, labels can become entirely -100 and training may show zero-loss behavior without real supervision.
Prompt-completion and conversational datasets have different masking semantics in TRL; the dataset format must match the training configuration.
A true language-model loss of exactly zero at the start of SFT is suspicious. If it is 0.0 from the first step, suspect masked labels rather than perfect learning.
In cross-entropy, `-100` is commonly the ignore index. If every label is -100, no supervised token remains. Inspect the collated batch and count `labels.ne(-100).sum()` before training.
python - <<'PY'
# batch = next(iter(trainer.get_train_dataloader()))
# print((batch['labels'] != -100).sum(dim=1))
PYWith long system/user prompts and a short max length, the assistant response may be entirely truncated. TRL issues show this can produce all-zero assistant masks and zero-loss behavior. Measure token-length percentiles across the dataset.
Current TRL requires the chat template to support assistant-token masks through generation markers for assistant-only loss. If the template cannot produce the mask, the flag may fail or mask incorrectly.
NaN is not only a dataset problem. Excessive learning rate, FP16 overflow, custom-loss bugs, invalid logits or model-specific attention bugs can produce NaNs. Run a direct training-mode forward pass and verify finite logits and loss.
The eval set may contain examples without assistant responses, different formatting, different truncation distribution or fully masked batches. Verify train and eval preprocessing use the same assumptions.
TRL supports language-modeling, prompt-completion and conversational formats. A plain `text` dataset and a `messages` conversational dataset do not share the same masking semantics. Identify the format before choosing loss masking.
To isolate Trainer complexity, feed one batch directly to the model. Inspect shapes, supervised-label counts, finite logits and raw loss. If NaN appears here, the problem is in model/data rather than Trainer logging.
For the first 20-50 steps, supervised-token count should be >0, loss and grad norm finite, and learning rate sane. Compare fixed validation prompts between the base model and checkpoint using identical decoding settings.
| Symptom | First suspect | Validation |
|---|---|---|
| Loss=0 at first step | All labels -100 | Count labels |
| Zero only on long examples | Truncation | Token-length distribution |
| Loss NaN | LR/precision/logits | Single-batch forward |
| Eval NaN | Eval masking/format | Inspect eval labels |
Do not blindly upgrade packages in a working training environment. Record GPU, driver, CUDA/PyTorch runtime, Transformers, Accelerate, PEFT, TRL, bitsandbytes/Diffusers, model revision and dataset fingerprint for every run. Reproduce minimally before changing production training.
Usually no. Zero from the beginning often indicates no supervised labels or a masking/logging issue.
No. The chat template must be able to produce assistant-token masks.
Evaluate model size, precision, context, batch, LoRA/QLoRA or full fine-tuning and multi-GPU needs together instead of choosing by GPU name alone.