Least Squares

Key idea

Best approximate solution minimising error.

In depth

When Ax = b has no solution, minimise |Ax − b|². The solution satisfies the normal equations AᵀA x̂ = Aᵀb. Geometrically Ax̂ is the projection of b onto the column space.

Worked example

Fit y = c + dx to (0, 1), (1, 2), (2, 2): solve AᵀA x̂ = Aᵀb to get c = 7/6, d = 1/2.

Formula

Real-world uses

• Fitting a best line to noisy sensor readings
• Estimating prices from house features

Common mistake

Using least squares when the data have outliers without checking.

How to learn this

1. Read the key idea and say it in your own words.
2. Work through the worked example, then try a similar problem.
3. Use the interactive lesson’s visual and mini-quiz to check the pattern.

Frequently asked questions

Expand all Collapse all

Related expert topics in Linear Algebra

• Triple Product – Scalar triple product gives parallelepiped volume.
• Homogeneous Systems – Ax = 0 always has the trivial solution.
• Cofactor Expansion – Compute determinants recursively.
• Properties of Determinants – Multiplicative; row swaps change sign.
• Geometric Meaning of Determinants – Area or volume scaling factor.
• Cramer’s Rule for Systems – Solve systems using determinants.