The Method of Least Squares

Introduction

Least-Squares Solutions

Best-Fit Problems

Which equations determine the least-squares solutions $\hat x$ of $Ax = b$?

$Ax = b$ has a unique least-squares solution $\hat x$ exactly when which condition holds?

In two words: a vector $\hat x$ that makes $\|A\hat x - b\|$ as small as possible over all vectors $x$ is called a ______ solution.

For $A = \begin{pmatrix} 0&1 \\ 1&1 \\ 2&1 \end{pmatrix}$ and $b = (6,0,0)$, the normal equations $A^TA\hat x = A^Tb$ have the unique solution $\hat x = (x_1,x_2)$, which gives the best-fit line $y = x_1 t + x_2$. Enter $x_2$ as a whole number.

Fit the four points $(t,b) = (0,0),\,(1,8),\,(3,8),\,(4,20)$ with the least-squares line $b = C + Dt$: solve the normal equations $A^TA\hat x = A^Tb$ for the $4\times 2$ matrix $A$ with rows $(1,t)$. Enter the least-squares error $E = \|b - A\hat x\|^2$ as a whole number.

In your own words, state the single most important definition or theorem of this section, and explain what it says.