Catalog
Supplement
Loss Functions
A ground-up tour of all 20 PyTorch loss functions — regression, classification, distribution, ranking, embedding, and metric learning — with equation breakdowns, derivations from probability theory, and side-by-side PyTorch and TensorFlow implementations.
Readings
1
What Is a Loss Function?
Training objectives, reduction modes, and how gradients flow from loss to weights.
10 min
2
Regression Losses: L1, MSE, Huber, SmoothL1
Point-estimate regression losses — their formulas, gradient shapes, and robustness to outliers.
15 min
3
Probabilistic Regression: Poisson & Gaussian NLL
Deriving loss functions from negative log-likelihood when the target follows a Poisson or Gaussian distribution.
12 min
4
Classification Losses: BCE, CrossEntropy, NLL
Binary and multi-class classification losses derived from information theory and maximum likelihood estimation.
18 min
5
Multi-label & Margin Losses
Hinge-based and multi-label losses: SoftMargin, MultiLabelSoftMargin, MultiMarginLoss, and MultiLabelMarginLoss.
12 min
6
Distribution & Similarity Losses
KL divergence, margin ranking, hinge embedding, and cosine embedding losses.
12 min
7
Metric Learning: Triplet Losses
TripletMarginLoss and TripletMarginWithDistanceLoss — learning embedding spaces where similar inputs cluster together.
10 min
Quizzes
Regression & Probabilistic Losses
6 questions · 70% to pass
Classification & Metric Losses
6 questions · 70% to pass
Labs
Loss Functions in PyTorch
Implement, verify, and compare all major PyTorch loss functions from scratch.
45 min
Loss Functions in TensorFlow
Implement the same loss functions using TensorFlow/Keras, with custom training loops and GradientTape.
40 min
Practice