人工智能教程 - 数学基础课程1.3 - 高等代数(一)-13-16

最小二乘

y = C+ Dt

最佳总误差最小(optimal = overall error as small as I can)

make it

$\left\{\begin{matrix} C+ D = 1 \\ C+ 2D = 2 \\ C+ 3D = 2 \end{matrix}\right.$⎩⎨⎧​C+D=1C+2D=2C+3D=2​

联立无解，但可以有最优解(they don’t have a solution.but they 've got a best solution)

Ax = b

$\begin{bmatrix} 1&1 \\ 1&2 \\ 1&3 \\ \end{bmatrix}\begin{bmatrix} C \\ D \\ \end{bmatrix} \ =\begin{bmatrix} 1 \\ 2 \\ 3 \\ \end{bmatrix}$⎣⎡​111​123​⎦⎤​[CD​] =⎣⎡​123​⎦⎤​

Minimize: $||Ax-b||^2=||e||^2$∣∣Ax−b∣∣2=∣∣e∣∣2

$=e_1^2+e_2^2+e_3^2$=e12​+e22​+e32​

$=(C+D-1)^2+(C+2D-2)^2+e_3^2$=(C+D−1)2+(C+2D−2)2+e32​

实际上做的是线性回归分析
(It’s going to do regression here,linear regression)

最小二乘(Least squares)应用最为广泛，但是有点太容易受到离群量(outliners)的影响

(because it’s a little overcomplensates for outliners)

Find $\widehat x = \begin{bmatrix} \widehat C\\ \widehat D \end{bmatrix}, P$x=[CD​],P

“^” 为最优的估计(they 're the estimated the best line, not the perfect line)

$\left\{\begin{matrix} A^T.A.\widehat x=A^T.b \\ P=A\widehat x\end{matrix}\right.${AT.A.x=AT.bP=Ax​

$\begin{bmatrix} 1&1&1 \\ 1&2&3 \\ \end{bmatrix}\begin{bmatrix} 1&1&| \ 1 \\ 1&2&| \ 2 \\ 1&3&|\ 2 \\ \end{bmatrix} \ =\begin{bmatrix} 3&6&| \ 5 \\ 6&14& \ \ | \ 11 \\ \end{bmatrix}$[11​12​13​]⎣⎡​111​123​∣ 1∣ 2∣ 2​⎦⎤​ =[36​614​∣ 5  ∣ 11​]

正规方程组(normal equation): $\left\{\begin{matrix} 3C+ 6D = 5 \\ 6C+ 14D = 11 \end{matrix}\right.${3C+6D=56C+14D=11​

$\left\{\begin{matrix} C=\frac{2}{3} \\ D=\frac{1}{2} \end{matrix}\right.${C=32​D=21​​

$\therefore$∴ Best line: $y = \frac{2}{3}+\frac{1}{2}t$y=32​+21​t

$\left\{\begin{matrix} e_1=-1/6 \\ e_2 = +2/6 \\ e_3 = -1/6 \end{matrix}\right.$⎩⎨⎧​e1​=−1/6e2​=+2/6e3​=−1/6​

b = P+ e

$\begin{bmatrix} 1 \\ 2\\ 2\\ \end{bmatrix}=\begin{bmatrix} 7/6 \\ 5/3\\ 13/6\\ \end{bmatrix} \ +\begin{bmatrix} -1/6 \\ 2/6\\ -1/6\\ \end{bmatrix}$⎣⎡​122​⎦⎤​=⎣⎡​7/65/313/6​⎦⎤​ +⎣⎡​−1/62/6−1/6​⎦⎤​

最小二乘：最典型的应用就是拟合最优直线
(the special but most important example of fitting by straight line)