Principal Component Analysis

Key idea

Find directions of greatest variance using eigenvectors.

In depth

PCA finds directions of greatest variance in data: the eigenvectors of the covariance matrix (or the SVD of the centred data). Projecting onto the top few reduces dimension while keeping most information.

Worked example

Two-dimensional data stretched along y = x has first component (1, 1)/√2.

Real-world uses

• Reducing many survey questions to a few key factors
• Visualising high-dimensional data

Common mistake

Forgetting to centre the data before computing.

How to learn this

1. Read the key idea and say it in your own words.
2. Work through the worked example, then try a similar problem.
3. Use the interactive lesson’s visual and mini-quiz to check the pattern.

Frequently asked questions

Expand all Collapse all

Related expert topics in Linear Algebra

• Triple Product – Scalar triple product gives parallelepiped volume.
• Homogeneous Systems – Ax = 0 always has the trivial solution.
• Cofactor Expansion – Compute determinants recursively.
• Properties of Determinants – Multiplicative; row swaps change sign.
• Geometric Meaning of Determinants – Area or volume scaling factor.
• Cramer’s Rule for Systems – Solve systems using determinants.