Second Order Backpropagation Efficient Computation Of The Hessian Matrix For Neural Networks

Download or read book Second Order Backpropagation: Efficient Computation of the Hessian Matrix for Neural Networks written by International Computer Science Institute and published by . This book was released on 1993 with total page 11 pages. Available in PDF, EPUB and Kindle. Book excerpt: Abstract: "Traditional learning methods for neural networks use some kind of gradient descent in order to determine the network's weights for a given task. Some second order learning algorithms deal with a quadratic approximation of the error function determined from the calculation of the Hessian matrix, and achieve improved convergence rates in many cases. We introduce in this paper second order backpropagation, a method to calculate efficiently the Hessian of a linear network of one- dimensional functions. This technique can be used to get explicit symbolic expressions or numerical approximations of the Hessian and could be used in parallel computers to improve second order learning algorithms for neural networks. It can be of interest also for computer algebra systems."