4.4. \(D_{i’,j}^{(m,n)}\)の計算

(Formula used in machine_learning header file package; 4. Derivative of the cross entropy error \(E\) with respect to model parameters \(W_{j,i}^{(m)}\); 4.4. Calculation of \(D_{i’,j}^{(m,n)}\)})

前節までで以下の式が得られた。
The formula below were obtained in the previous sections.

2節の(2)式
Eq. (2) of section 2:
\[\begin{eqnarray} y_j^{(m,n)} &=& \sum_{i=0}^{J^{(m)}-1}W_{j,i}^{(m)}x_i^{(m,n)}+W_{j,J^{(m)}}^{(m)} \nonumber \\ & & \left(m=0,\cdots,M; n=0,\cdots,N-1; j=0,\cdots,J^{(m+1)}-1\right) \label{eq.x2y} \end{eqnarray}\]
2節の(3)式
Eq. (3) of section 2:
\[\begin{eqnarray} x_j^{(m+1,n)} &=& f_j^{(m)}\left(y_0^{(m,n)},\cdots,y_{J^{(m+1)}-1}^{(m,n)}\right) \nonumber \\ & & \left(m=0,\cdots,M; n=0,\cdots,N-1; j=0,\cdots,J^{(m+1)}-1\right) \label{eq.y2x} \end{eqnarray}\]
4.1節の(4)式
Eq. (4) of section 4.1:
\[\begin{eqnarray} D_{i’,j}^{(m,n)} &=& \PartialDiff{x_{i’}^{(M+1,n)}}{y_{j}^{(m,n)}} \nonumber \\ & & \left(m=0,\cdots,M; n=0,\cdots,N-1; \right. \nonumber \\ & & \left. i’=0,\cdots,J^{(M+1)}-1; j=0,\cdots,J^{(m+1)}-1\right) \label{eq.D} \end{eqnarray}\]

\(D_{i’,j}^{(m,n)}\)は\(m=M\)の場合から始めて漸化式により\(m\)を小さくする方向に順次計算できる。まず\(m=M\)のとき、(\ref{eq.y2x})式を用いて \[\begin{eqnarray} D_{i’,j}^{(M,n)} &=& \PartialDiff{x_{i’}^{(M+1,n)}}{y_{j}^{(M,n)}} \nonumber \\ &=& \PartialDiff{f_{i’}^{(M)} \left(y_0^{(M,n)},\cdots,y_{J^{(M+1)}-1}^{(M,n)}\right)} {y_{j}^{(M,n)}} \nonumber \\ & & \left(n=0,\cdots,N-1; i’=0,\cdots,J^{(M+1)}-1; j=0,\cdots,J^{(M+1)}-1\right) \label{eq.D.M1} \end{eqnarray}\] と計算できる。次に\(m<M\)の場合であるが、(\ref{eq.D})式を \[\begin{eqnarray} D_{i’,j}^{(m,n)} &=& \sum_{j’=0}^{J^{(m+2)}-1} \PartialDiff{x_{i’}^{(M+1,n)}}{y_{j’}^{(m+1,n)}} \PartialDiff{y_{j’}^{(m+1,n)}}{y_{j}^{(m,n)}} \nonumber \\ &=& \sum_{j’=0}^{J^{(m+2)}-1}D_{i’,j’}^{(m+1,n)} \PartialDiff{y_{j’}^{(m+1,n)}}{y_{j}^{(m,n)}} \nonumber \\ & & \left(m=0,\cdots,M-1; n=0,\cdots,N-1; \right. \nonumber \\ & & \left. i’=0,\cdots,J^{(M+1)}-1; j=0,\cdots,J^{(m+1)}-1\right) \label{eq.D.recursive.derive1} \end{eqnarray}\] と変形する。これによって\(D_{i’,j’}^{(m+1,n)}\)から \(D_{i’,j}^{(m,n)}\)を計算できる漸化式の形となる。 (\ref{eq.D.recursive.derive1})式の最後の偏微分については (\ref{eq.x2y})(\ref{eq.y2x})式を用いて \[\begin{eqnarray} \PartialDiff{y_{j’}^{(m+1,n)}}{y_{j}^{(m,n)}} &=& \PartialDiff{}{y_{j}^{(m,n)}}\left[ \sum_{i{’}{’}=0}^{J^{(m+1)-1}} W_{j’,i{’}{’}}^{(m+1)}x_{i{’}{’}}^{(m+1,n)} +W_{j’,J^{(m+1)}}^{(m+1)} \right] \nonumber \\ &=& \sum_{i{’}{’}=0}^{J^{(m+1)-1}} W_{j’,i{’}{’}}^{(m+1)} \PartialDiff{x_{i{’}{’}}^{(m+1,n)}}{y_{j}^{(m,n)}} \nonumber \\ &=& \sum_{i{’}{’}=0}^{J^{(m+1)-1}} W_{j’,i{’}{’}}^{(m+1)} \PartialDiff{f_{i{’}{’}}^{(m)} \left(y_0^{(m,n)},\cdots,y_{J^{(m+1)}-1}^{(m,n)}\right)} {y_{j}^{(m,n)}} \nonumber \\ & & \left(m=0,\cdots,M-1; n=0,\cdots,N-1; \right. \nonumber \\ & & \left. j=0,\cdots,J^{(m+1)}-1; j’=0,\cdots,J^{(m+2)}-1\right) \label{eq.D.recursive.derive2} \end{eqnarray}\] と計算できる。 (\ref{eq.D.recursive.derive2})を(\ref{eq.D.recursive.derive1})に代入して \[\begin{eqnarray} D_{i’,j}^{(m,n)} &=& \sum_{j’=0}^{J^{(m+2)}-1}D_{i’,j’}^{(m+1,n)} \sum_{i{’}{’}=0}^{J^{(m+1)-1}} W_{j’,i{’}{’}}^{(m+1)} \PartialDiff{f_{i{’}{’}}^{(m)} \left(y_0^{(m,n)},\cdots,y_{J^{(m+1)}-1}^{(m,n)}\right)} {y_{j}^{(m,n)}} \nonumber \\ & & \left(m=0,\cdots,M-1; n=0,\cdots,N-1; \right. \nonumber \\ & & \left. i’=0,\cdots,J^{(M+1)}-1; j=0,\cdots,J^{(m+1)}-1\right) \label{eq.D.recursive} \end{eqnarray}\] を得る。最初に\(m=M\)での値を(\ref{eq.D.M1})によって計算し、あとは\(m\)を1つずつ小さくしながら(\ref{eq.D.recursive})式を順次適用すれば全ての\(m\)に対する\(D_{i’,j}^{(m,n)}\)が求まることになる。
The values of \(D_{i’,j}^{(m,n)}\) can be computed recursively, starting from \(m=M\) and decreasing \(m\). At \(m=M\), \(D_{i’,j}^{(m,n)}\) is calculated as (\ref{eq.D.M1}), where Eq. (\ref{eq.y2x}) is used. For \(m<M\), Eq. (\ref{eq.D}) is arranged as (\ref{eq.D.recursive.derive1}), which gives a recursive form to compute \(D_{i’,j}^{(m,n)}\) using \(D_{i’,j’}^{(m+1,n)}\). The last partial derivative in Eq. (\ref{eq.D.recursive.derive1}) can be computed using Eqs. (\ref{eq.x2y}) and (\ref{eq.y2x}) as (\ref{eq.D.recursive.derive2}). Inserting Eq. (\ref{eq.D.recursive.derive2}) into (\ref{eq.D.recursive.derive1}), we finally obtain (\ref{eq.D.recursive}). The \(D_{i’,j}^{(m,n)}\) values for all \(m\) are calculated sequentially by first computing the values at \(m=M\) by Eq. (\ref{eq.D.M1}), then computing the values for decreasing \(m\) by (\ref{eq.D.recursive}).