Catalog
Prerequisite Course
Probability Foundations
The probability and statistics prerequisites for modern ML — drawing on Downey's Think Bayes and Think Stats. Covers random variables, expectation, Bayes' theorem and the chain rule, key distributions (Gaussian, Bernoulli, Categorical, Dirichlet), importance sampling, and KL divergence. Directly addresses the Spinning Up background requirements for deep reinforcement learning.
Readings
1
Random Variables and Probability Distributions
Sample spaces, PMFs, PDFs, CDFs, joint distributions, marginalization, and the law of total probability.
16 min
2
Expected Value, Variance, and Standard Deviation
Expectation as a weighted average, linearity of expectation, variance, standard deviation, covariance, correlation, and their roles in weight initialization and SGD.
14 min
3
Bayes' Theorem and the Chain Rule of Probability
Conditional probability, the product rule, the chain rule of probability, Bayes' theorem with prior/likelihood/posterior, and sequential belief updating.
15 min
4
Key Distributions for Machine Learning
Bernoulli, Categorical, Gaussian, Multivariate Gaussian, Beta, and Dirichlet distributions — parameters, properties, and ML use cases.
18 min
5
Importance Sampling
The IS identity, importance weights, variance analysis, self-normalized IS, effective sample size, and applications in off-policy RL and PPO.
14 min
6
KL Divergence and Cross-Entropy
Entropy, cross-entropy, KL divergence, non-negativity, asymmetry, forward vs. reverse KL, connection to MLE, and appearances in VAE and policy optimization.
16 min
Quizzes
Random Variables, Expectation, and Bayes' Theorem
8 questions · 70% to pass
Distributions, Importance Sampling, and KL Divergence
9 questions · 70% to pass
Labs
Distributions and Bayes' Theorem
Plot and verify PMFs/PDFs, compute CDFs, sample from joint distributions, confirm the law of total probability via simulation, and sweep the Bayes disease-screening posterior over a range of priors.
45 min
Monte Carlo Estimation and Importance Sampling
Estimate expectations via Monte Carlo, verify linearity of expectation empirically, implement importance sampling, compute ESS for mismatched proposals, and demonstrate importance weight collapse.
50 min
Entropy, KL Divergence, and Fitting Distributions
Compute entropy and KL divergence from scratch, verify the cross-entropy decomposition, fit a Gaussian to a bimodal target via forward and reverse KL to observe mean-seeking vs mode-seeking, and implement Beta posterior updates.
45 min
Practice