Derivative value outputs for neural networks

Yoshikawa, Shinji ; Okusa, Ryoichi ; Ozawa, Kenji

This report discusses a method to equip a multi layer neural network(NN) with a calculational function to derive differential values of the output parameters against the input parameters. Multi layer NNs have been applied in various domains of engineering, because of easy construction, flexible interpolation of nonlinear multi-input functions, and some other preferable features. However, derivatives of those output parameters have been approximately calculated by interpolating between two different output values. And new methods to guarantee the accuracy of the derivatives have been desired. We payed their attention at sigmoid functions, which are commonly used to realize the nonlinear characteristics of nodes in NNs, and at one of important features of this function type that the derivative is represented by a polinomial of itself. And, we developed a method to add a calculational function to derive differentiated values of the output parameters to multi layer NNs, whose CPU cost is smaller than the original NNs.