# PyTorch Binary Classification: Fixing the Loss vs. Metric Gap

> Source: <https://promptcube3.com/en/threads/2763/>
> Published: 2026-07-24 14:06:18+00:00

# PyTorch Binary Classification: Fixing the Loss vs. Metric Gap

Accuracy is a lie when you're dealing with imbalanced datasets in binary classification. I hit a wall recently where my model showed 92% accuracy during training, but the precision-recall curve looked like a disaster. The core issue is the fundamental difference between the objective function (what the optimizer minimizes) and the evaluation metrics (what we actually care about).

For anyone building a custom training loop, don't rely on the raw loss. I use

In PyTorch, we typically use `BCEWithLogitsLoss`

for binary tasks. This is a smooth, differentiable function necessary for backpropagation. However, you can't use accuracy as a loss function because it's non-differentiable.

To properly diagnose model performance, I've moved to a setup that tracks these specific metrics per epoch:

**Log Loss (Cross-Entropy):** The actual value the optimizer is attacking. If this drops but accuracy doesn't move, you're likely dealing with a severe class imbalance.**Precision & Recall:** Essential for understanding the trade-off between false positives and false negatives.**F1-Score:** The harmonic mean of the two, which is the only number I trust for "overall" performance on skewed data.**AUC-ROC:** Measures the model's ability to distinguish between classes regardless of the decision threshold.

For anyone building a custom training loop, don't rely on the raw loss. I use

`torchmetrics`

to handle the accumulation across batches to avoid the "average of averages" mistake. Here is the basic logic for calculating the F1-score correctly within a validation loop:

``` python
import torch
from torchmetrics.classification import BinaryF1Score

# Initialize metric
f1_metric = BinaryF1Score().to(device)

model.eval()
with torch.no_grad():
    for inputs, targets in val_loader:
        inputs, targets = inputs.to(device), targets.to(device)
        logits = model(inputs)
        # Convert logits to binary predictions (0 or 1)
        preds = (torch.sigmoid(logits) > 0.5).float()
        f1_metric.update(preds, targets)

final_f1 = f1_metric.compute()
print(f"Validation F1 Score: {final_f1}")
```

If you see your loss decreasing while your F1-score plateaus or drops, check your class weights in the loss function. Adding `pos_weight`

to `BCEWithLogitsLoss`

usually fixes the bias toward the majority class.

[Next The Stack v3: Massive Scale vs. Data Noise →](/en/threads/2755/)

## All Replies （3）

R

Happened to me with fraud detection; high accuracy but it missed every single actual case.

0

S

Did you try adjusting the class weights in BCE loss, or just pray it worked?

0

P

Using an F1-score usually clears things up for me when the classes are skewed.

0