Learning to be Giant.

Math notes on Linear Algebra

|

为什么会有这样的一篇文章呢?因为时常我会发现,曾经学过的数学知识再用到的时候就已经忘了。好不容易研究明白,过两天又记不清了。所以干脆还是把它们记下来,以后翻一番什么的也是好事。若要使有别人也看到了这篇文章,并且能通过它复习起之前的很多知识,那更是功德一件。

##矩阵的伪逆 对于非方阵来讲,求逆也是可能的操作。在Matlab当中就有pinv()这个函数,称为pseudoinverse,即伪逆。这种方法常被用于求解最小二乘法。

先说结论,A的伪逆就是:\({({A^T}A)^{ - 1}}{A^T}\)

下面看一下推导:

\[\left\{ {\begin{array}{*{20}{c}} {{a_{11}}{x_1} + {a_{12}}{x_2} + ... + {a_{1q}}{x_q} = {b_1}}\\ {{a_{21}}{x_1} + {a_{22}}{x_2} + ... + {a_{2q}}{x_q} = {b_2}}\\ {...}\\ {{a_{p1}}{x_1} + {a_{p2}}{x_2} + ... + {a_{pq}}{x_q} = {b_p}} \end{array}} \right. \Leftrightarrow Ax = b\]

当我们考虑,当(p>q)并且(A)的秩为q,则此时,这个方程组是没有唯一解的。所以,我们需要一个近似解,即找到一个向量使得下式最小:

\[E(x) = \sum\limits_{i = 1}^p {{{({a_{i1}}{x_1} + {a_{i2}}{x_2} + ... + {a_{iq}}{x_q} - {b_i})}^2}} = {\left| {Ax - b} \right|^2}\]

进一步推导:

\[\begin{array}{l} \frac{{\partial E}}{{\partial {x_i}}} = 0(i = 1,...,q) \Leftrightarrow 2(Ax - b) \cdot \frac{{\partial (Ax - b)}}{{\partial {x_i}}} = 0\\ \Leftrightarrow {(Ax - b)_{p \times 1}} \cdot \left( {\begin{array}{*{20}{c}} {{a_{1i}}}\\ {{a_{2i}}}\\ {...}\\ {{a_{pi}}} \end{array}} \right) = 0\\ \Leftrightarrow {\left( {\begin{array}{*{20}{c}} {{a_{1i}}}&{{a_{2i}}}&{...}&{{a_{pi}}} \end{array}} \right)_{1 \times p}}{(Ax - b)_{p \times 1}} = 0\\ \Leftrightarrow \left( {\begin{array}{*{20}{c}} {{a_{11}}}&{{a_{21}}}&{...}&{{a_{p1}}}\\ {{a_{12}}}&{{a_{22}}}&{...}&{{a_{p2}}}\\ {...}&{}&{}&{}\\ {{a_{1q}}}&{{a_{2q}}}&{...}&{{a_{pq}}} \end{array}} \right){(Ax - b)_{p \times 1}} = {0_{q \times 1}}\\ \Leftrightarrow {A^T}(Ax - b) = 0 \Rightarrow {A^T}Ax = {A^T}b\\ \Rightarrow x = {({A^T}A)^{ - 1}}{A^T}b \end{array}\]

推导部分参考:http://sse.tongji.edu.cn/linzhang/CV14/slides/Lecture%2004.pdf

Disclaimer: This is a personal weblog. The opinions expressed here represent my own and not those of any entity with which I have been, am now, or will be affiliated.

Comments