Integration and Probability

Lab 3: Integration and Probability

Integration connects calculus to probability: probability densities are defined by integrals, expected values are integrals, entropy is an integral. This lab makes those connections concrete — you will compute integrals numerically and analytically, verify the Fundamental Theorem, and recover the key probabilistic quantities you will use throughout the rest of the platform.

What You'll Build

• A trapezoidal integrator from scratch and a convergence plot showing error vs number of trapezoids, confirming the $O(1/n^2)$ rate
• A Fundamental Theorem verifier: compare $\int_a^b f(x)\,dx$ computed numerically vs $F(b) - F(a)$ from the analytic antiderivative, for several $f$ and interval pairs
• A Gaussian normalization check: numerically integrate the standard normal PDF over $[-\infty, \infty]$ (using a wide finite range) and confirm the result is 1 — then verify the 68–95–99.7 rule by integrating over $[-1\sigma, 1\sigma]$, $[-2\sigma, 2\sigma]$, $[-3\sigma, 3\sigma]$
• A Monte Carlo integrator: estimate $\pi$ by sampling uniform points in the unit square and checking whether they fall in the unit circle (area $= \pi/4$), then plot how the estimate improves with sample count
• A KL divergence calculator: numerically integrate $D_{\text{KL}}(p \| q) = \int p(x) \ln(p(x)/q(x))\,dx$ for two Gaussians with different means, confirming non-negativity and asymmetry ($D_{\text{KL}}(p\|q) \neq D_{\text{KL}}(q\|p)$)

Key Concepts Practiced

By the end you will understand that the probabilistic quantities from the probability-foundations module — PDFs, expected values, KL divergence — are all just integrals, and that numerical integration via Monte Carlo sampling is why those quantities are tractable in high-dimensional ML systems.