|
|
|
@ -22,11 +22,13 @@ g(z_i)=g(\theta_i^T \mathbf{x})=\frac{e^{\theta_i^T\mathbf{x}}}{\sum\limits_{j=1
|
|
|
|
|
$$
|
|
|
|
|
构造似然函数,若有$m$个训练样本:
|
|
|
|
|
$$
|
|
|
|
|
\begin{align}
|
|
|
|
|
\begin{equation}
|
|
|
|
|
\begin{split}
|
|
|
|
|
L(\Theta)&=p(\mathbf{y}|\mathbf{X};\Theta) \\
|
|
|
|
|
& = \prod\limits_{i=1}^{m} p(y^{i}|\mathbf{x}^{i};\Theta) \\
|
|
|
|
|
& = \prod_{i=1}^m h_{\theta_i}(\mathbf{x})
|
|
|
|
|
\end{align}
|
|
|
|
|
\end{split}
|
|
|
|
|
\end{equation}
|
|
|
|
|
$$
|
|
|
|
|
对似然函数取对数,转换为:
|
|
|
|
|
$$
|
|
|
|
@ -41,10 +43,12 @@ $$
|
|
|
|
|
$$
|
|
|
|
|
转换后的似然函数对$\theta$求偏导,在这里我们以只有一个训练样本的情况为例:
|
|
|
|
|
$$
|
|
|
|
|
\begin{align}
|
|
|
|
|
\begin{equation}
|
|
|
|
|
\begin{split}
|
|
|
|
|
\frac{\partial}{\partial\theta_k}l(\Theta)&=\frac{\partial l(\Theta)}{\partial{z_k}}\cdot \frac{\partial z_k}{\partial \theta_k} \\
|
|
|
|
|
&=(y_k-h_{\theta_k}(\mathbf{x}))\mathbf{x}
|
|
|
|
|
\end{align}
|
|
|
|
|
\end{split}
|
|
|
|
|
\end{equation}
|
|
|
|
|
$$
|
|
|
|
|
上式中$y_k$的表达式如下:
|
|
|
|
|
$$
|
|
|
|
|