Dimension

All Bases Have the Same Size

The most surprising and important fact about bases is:

Theorem (Invariance of Dimension): any two bases for the same vector space $V$ have the same number of elements.

This number is the dimension of $V$, written $\dim(V)$. It is well-defined regardless of which basis you choose.

Proof sketch: if $B_1$ has $m$ vectors and $B_2$ has $n$ vectors and both are bases, then expressing each $B_1$ vector in terms of $B_2$ and using a linear-algebra counting argument shows $m \leq n$. Repeating with the roles swapped gives $n \leq m$, so $m = n$.

Dimensions of Common Spaces

Using Dimension to Identify Bases

In a space of dimension $n$, any set of exactly $n$ vectors that satisfies one of the following automatically satisfies both:

• $n$ independent vectors → automatically span $V$ → basis
• $n$ spanning vectors → automatically independent → basis

This is immensely practical: to show a set of $n$ vectors is a basis for $\mathbb{R}^n$, you only need to check independence (or spanning) — the other condition is free.

The Rank-Nullity Theorem

For any $m \times n$ matrix $A$, define:

• $\text{rank}(A)$ = dimension of the column space $\mathcal{C}(A)$ = number of pivots in echelon form
• $\text{nullity}(A)$ = dimension of the null space $\mathcal{N}(A)$ = number of free variables

Then:

$$\text{rank}(A) + \text{nullity}(A) = n$$

Every column of $A$ is either a pivot column (contributing to rank) or a free column (contributing to nullity). The two types partition all $n$ columns.

Equivalences for an $n \times n$ matrix $A$:

All of the following are equivalent:

• $\text{rank}(A) = n$
• $\text{nullity}(A) = 0$, i.e., $\mathcal{N}(A) = \{\vec{0}\}$
• The columns of $A$ are linearly independent
• The columns of $A$ span $\mathbb{R}^n$
• $A$ is invertible
• $\det(A) eq 0$
• $A\vec{x} = \vec{b}$ has a unique solution for every $\vec{b}$

This list of equivalences is sometimes called the Invertible Matrix Theorem. Knowing any one of them gives all the others for free.

Dimension of Subspaces

If $W$ is a subspace of $V$ with $\dim(V) = n$:

• $\dim(W) \leq n$
• $\dim(W) = n \iff W = V$ (the only $n$-dimensional subspace of $V$ is $V$ itself)
• $\dim(W) = 0 \iff W = \{\vec{0}\}$

Combining Subspaces

If $V = W_1 + W_2$ (every vector in $V$ is a sum of a vector from $W_1$ and one from $W_2$), the dimension formula is:

$$\dim(W_1 + W_2) = \dim(W_1) + \dim(W_2) - \dim(W_1 \cap W_2)$$

If the intersection is only $\{\vec{0}\}$, the sum is direct: $V = W_1 \oplus W_2$ and $\dim(V) = \dim(W_1) + \dim(W_2)$. The direct sum decomposes $V$ into orthogonal (or at least non-overlapping) pieces — the matrix analogue is block-diagonal decomposition.

Computing Rank and Nullity

import numpy as np

# 2 × 4 matrix — rank 2, nullity 2
A = np.array([[1., 2., 0., 1.],
              [2., 4., 1., 1.]])

# Rank and nullity via SVD
_, S, Vh = np.linalg.svd(A, full_matrices=True)
rank    = int(np.sum(S > 1e-9))
nullity = A.shape[1] - rank

print(f'rank    = {rank}')                                       # 2
print(f'nullity = {nullity}')                                    # 2
print(f'rank + nullity = {rank + nullity} = n = {A.shape[1]}')  # 2 + 2 = 4

# Null space basis: rows of Vh beyond rank index
null_basis = Vh[rank:]
print('null space basis (rows):')
print(np.round(null_basis, 4))

# Verify A @ null_vec = 0 for every null vector
for i, h in enumerate(null_basis):
    resid = np.einsum('ij,j->i', A, h)
    print(f'A @ null_basis[{i}] = {np.round(resid, 4)}')  # [0. 0.]

The row dimension of Vh[rank:] always equals the nullity — that is the Rank-Nullity Theorem made concrete. For a square invertible matrix, rank == n, so Vh[rank:] is empty and nullity == 0.