A three-term backpropagation algorithm

Zweiri, Y. H., Whidborne, J. F. and Seneviratne, L. D. (2003) A three-term backpropagation algorithm. Neurocomputing, 50, 305 - 318. ISSN (print) 0925-2312

Full text not available from this archive.

Abstract

Efficient learning by the backpropagation (BP) algorithm is required for many practical applications. The BP algorithm calculates the weight changes of artificial neural networks, and a common approach is to use a two-term algorithm consisting of a learning rate (LR) and a momentum factor (MF). The major drawbacks of the two-term BP learning algorithm are the problems of local minima and slow convergence speeds, which limit the scope for real-time applications. Recently the addition of an extra term, called a proportional factor (PF), to the two-term BP algorithm was proposed. The third increases the speed of the BP algorithm. However, the PF term also reduces the convergence of the BP algorithm, and criteria for evaluating convergence are required to facilitate the application of the three terms BP algorithm. This paper analyzes the convergence of the new three-term backpropagation algorithm. If the learning parameters of the three-term BP algorithm satisfy the conditions given in this paper, then it is guaranteed that the system is stable and will converge to a local minimum. It is proved that if at least one of the eigenvalues of matrix F (compose of the Hessian of the cost function and the system Jacobian of the error vector at each iteration) is negative, then the system becomes unstable. Also the paper shows that all the local minima of the three-term BP algorithm cost function are stable. The relationship between the learning parameters are established in this paper such that the stability conditions are met.

Item Type: Article
Uncontrolled Keywords: backpropagation, three-term algorithm, learning algorithms, multi-layer neural networks
Research Area: Computer science and informatics
Mechanical, aeronautical and manufacturing engineering
Faculty, School or Research Centre: Faculty of Science, Engineering and Computing (until 2017)
Faculty of Science, Engineering and Computing (until 2017) > School of Mechanical and Automotive Engineering
Related URLs:
Depositing User: Davina Omar
Date Deposited: 25 Nov 2014 14:03
Last Modified: 25 Nov 2014 14:03
DOI: https://doi.org/10.1016/S0925-2312(02)00569-6
URI: http://eprints.kingston.ac.uk/id/eprint/29452

Actions (Repository Editors)

Item Control Page Item Control Page