Prerequisite · Linear Algebra

Geometry of Linear Transformations with Word Embeddings

Colab Notebook · ~45 min
Google Colab Notebook
Geometry of Linear Transformations with Word Embeddings
Python · ~45 min
Open in Colab
Lab Objectives
1
Load GloVe-50d embeddings and compute ℓ1\ell^1, ℓ2\ell^2, and ℓ∞\ell^\infty norms; interpret the distribution
2
Implement batch cosine similarity as a single matrix product X^X^T\hat{X}\hat{X}^T and verify against torch.nn.functional.cosine_similarity
3
Visualise what a 2×22\times 2 linear map does to the unit circle and identify the null space geometrically
4
Project a vector onto a subspace using P=UUTP = UU^T (PyTorch) and tf.linalg.matmul (TF); verify the residual is orthogonal to the subspace
5
Construct the General = Particular + Homogeneous decomposition for a rank-deficient system and verify numerically

Lab Overview

Word embeddings are high-dimensional vectors — but most of the interesting structure lives in the geometry: norms, dot products, angles, and subspace projections. In this lab you will load real GloVe vectors, build intuition for vector geometry, and connect it directly to the operations that power embedding-based retrieval and attention.

What You Will Build

A notebook that (1) computes norms and cosine similarities across a vocabulary slice, (2) visualises how matrix operations transform subspaces, and (3) implements orthogonal projection and verifies the General = Particular + Homogeneous decomposition on a toy linear system — all in both PyTorch and TensorFlow.