Given our usual dataset made of input vectors and output values, the design matrix is the matrix whose rows are the input vectors.
Definition
Let be a dataset made of paired input vectors and output values :
This is the usual training set for most of our supervised-learning articles, such as the OLS regression.
The design matrix is the matrix whose -th row is the transposed -th input vector :
If we note the -th component of vector :
Using the jargon of statistics: the design matrix is the matrix that contains data on the explanatory variables. Each row represents an individual object.
Design matrix for linear regressions
Design matrices for linear regressions are introduced in this article: Vector notation for linear regressions.
In linear regressions, we often consider that the input vectors have been modified during preprocessing to add a constant component . This component is called the bias-term and simplifies the notations for the linear regression. Using this convention, the design matrix has an additional column of ones:
Thanks to the bias-term, a linear function parametrized by some vector of parameters can be expressed in vector notation using a matrix product:
If we write the equation above for each input vectors:
We can use the design matrix as factor:
This convention greatly simplifies notations. For instance, we can use the design matrix to express the solution to an ordinary least squares regression easily.