In this article, I show that the normal equations define the orthogonal projection of a vector onto a linear subspace.

## Setup

Let be a vector space and a linear subspace. Let be a matrix whose column vectors are a basis for , i.e. . Each vector in can be expressed as a linear combination of the column vectors in :

## The orthogonal projection

Given a vector , the orthogonal projection of onto is the vector in such that the residual is orthogonal to every vector in :

- ,
- ,
- .

## Solving the equations

A vector is orthogonal to every vector in if and only if , hence we are looking to solve the following equation:

Since , we can find a vector such that:

So the equation to be solved is:

Which is the matrix form of the normal equations.