Linear Regression via Matrices

Key idea

Fit a line using normal equations.

In depth

Regression fits a line (or plane) by least squares: build the design matrix X with a column of ones, then β̂ = (XᵀX)⁻¹Xᵀy. The residuals are perpendicular to the columns of X.

Worked example

For data (1, 2), (2, 3), (3, 5): slope = 1.5, intercept ≈ 0.33.

Real-world uses

• Predicting sales or grades from several variables using matrix formulas in data science

Common mistake

Having correlated columns, making XᵀX nearly singular.

How to learn this

1. Read the key idea and say it in your own words.
2. Work through the worked example, then try a similar problem.
3. Use the interactive lesson’s visual and mini-quiz to check the pattern.

Frequently asked questions

Expand all Collapse all

Related expert topics in Linear Algebra

• Triple Product – Scalar triple product gives parallelepiped volume.
• Homogeneous Systems – Ax = 0 always has the trivial solution.
• Cofactor Expansion – Compute determinants recursively.
• Properties of Determinants – Multiplicative; row swaps change sign.
• Geometric Meaning of Determinants – Area or volume scaling factor.
• Cramer’s Rule for Systems – Solve systems using determinants.