(1) |
(2) |
(3) | |||
(4) |
We need to expand these out and solve for and when both of these equations are simultaneously set to zero.
Now we have derived from (3),
(5) |
(6) |
These can now be used to give equations for and ,
(7) | |||
(8) |
This is our optimal, least-squares, estimate of and . Note we should verify that this solution is the minimum solution and not the maximum (remember that the first derivative is zero at both places). This verification requires taking the second derivatives and establishing that they are positive. This is pretty easy to show if one looks at (5) and (6). The second derivative of with respect to is the derivative with respect to of the right hand side of (5), which is . Since this is the sum of squared quantities it is positive so the second derivative is positive. Doing the same for , we take the derivative with respect to of the right hand side of (6) and we get , which again is positive.
Listing 1, shows an example of a general purpose least squares fit routine. It takes data pairs and returns the optimal estimates of and . For the sample data file in listing 2, you should get a slope of 0.1781 and an intercept of 0.3687. With a sufficient amount of patience (or by putting Mathematica to work), we can work out equations like (7) and (8) for any polynomial form, not just a straight line.