Setup

We have a set of observations , and we want to find the parameters such that , where is Normally distributed.

Solution

We will apply Maximum Likelihood Estimation. Applied to this problem, we want to find the of the expression below:

In order to find the we equal all derivatives to zero:

Doing the same for :

Writing the 2 equations above in matrix form:

Applying the formula above, we get the solution for the Linear regression as a function of our observations. We can also calculate if we want to know the estimated error of our predictions, but it is not needed to build the predictor itself