machine_learningヘッダファイルパッケージで用いている計算式
1. 数学関数の定義と微分形
関数softmax
(Formula used in machine_learning header file package;
1. Definitions and differential forms of mathematical functions
— function softmax)
◆定義(Definition)
\[\begin{equation}
f_i(y_0,\cdots,y_{J-1})=\frac{\exp(y_i)}{u},
\hspace{1em}
u=\sum_{i’=0}^{J-1}\exp(y_{i’})
\hspace{2em} (i=0,\cdots,J-1)
\end{equation}\]
◆微分形(Derivative)
\[\begin{eqnarray}
\PartialDiff{f_i}{y_{i’}}
&=& \frac{\exp(y_i)\delta_{ii’}}{u}
-\frac{\exp(y_i)}{u^2}\PartialDiff{u}{y_{i’}}
\nonumber \\
&=& \frac{\exp(y_i)}{u}\delta_{ii’}
-\frac{\exp(y_i)}{u^2}\exp(y_{i’})
\nonumber \\
&=& \frac{\exp(y_i)}{u}
\left[\delta_{ii’}-\frac{\exp(y_{i’})}{u}\right]
\nonumber \\
&=& f_i(y_0,\cdots,y_{J-1})
\left[\delta_{ii’}-f_{i’}(y_0,\cdots,y_{J-1})\right]
\nonumber \\
& & (i,i’=0,\cdots,J-1)
\end{eqnarray}\]