Theoretical Derivation of Momentum Term in Back-propagation

Masafumi Hagiwara

doi:10.1109/IJCNN.1992.287108

Theoretical Derivation of Momentum Term in Back-propagation

Masafumi Hagiwara

Department of Information and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

19 Citations (Scopus)

Abstract

This paper explains clearly the theoretical origin of a momentum term in Back-propagation algorithm. We will prove that the Back-propagation algorithm, which has a momentum term, can be derived through the following assumptions: 1) The cost function is {equation presented}, where Ep is the sum of squared error at the output layer. 2) The most recent weights are assumed in calculating E n.

Original language	English
Title of host publication	Proceedings - 1992 International Joint Conference on Neural Networks, IJCNN 1992
Publisher	Institute of Electrical and Electronics Engineers Inc.
Pages	682-686
Number of pages	5
ISBN (Electronic)	0780305590
DOIs	https://doi.org/10.1109/IJCNN.1992.287108
Publication status	Published - 1992
Event	1992 International Joint Conference on Neural Networks, IJCNN 1992 - Baltimore, United States Duration: 1992 Jun 7 → 1992 Jun 11

Publication series

Name	Proceedings of the International Joint Conference on Neural Networks
Volume	1

Conference

Conference	1992 International Joint Conference on Neural Networks, IJCNN 1992
Country/Territory	United States
City	Baltimore
Period	92/6/7 → 92/6/11

ASJC Scopus subject areas

Software
Artificial Intelligence

Access to Document

10.1109/IJCNN.1992.287108

Cite this

Theoretical Derivation of Momentum Term in Back-propagation. / Hagiwara, Masafumi.
Proceedings - 1992 International Joint Conference on Neural Networks, IJCNN 1992. Institute of Electrical and Electronics Engineers Inc., 1992. p. 682-686 (Proceedings of the International Joint Conference on Neural Networks; Vol. 1).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Hagiwara, M 1992, Theoretical Derivation of Momentum Term in Back-propagation. in Proceedings - 1992 International Joint Conference on Neural Networks, IJCNN 1992. Proceedings of the International Joint Conference on Neural Networks, vol. 1, Institute of Electrical and Electronics Engineers Inc., pp. 682-686, 1992 International Joint Conference on Neural Networks, IJCNN 1992, Baltimore, United States, 92/6/7. https://doi.org/10.1109/IJCNN.1992.287108

@inproceedings{e3c0ffa4ccee434ea04d1769f94334e6,

title = "Theoretical Derivation of Momentum Term in Back-propagation",

abstract = "This paper explains clearly the theoretical origin of a momentum term in Back-propagation algorithm. We will prove that the Back-propagation algorithm, which has a momentum term, can be derived through the following assumptions: 1) The cost function is {equation presented}, where Ep is the sum of squared error at the output layer. 2) The most recent weights are assumed in calculating E n.",

author = "Masafumi Hagiwara",

note = "Publisher Copyright: {\textcopyright} 1992 IEEE.; 1992 International Joint Conference on Neural Networks, IJCNN 1992 ; Conference date: 07-06-1992 Through 11-06-1992",

year = "1992",

doi = "10.1109/IJCNN.1992.287108",

language = "English",

series = "Proceedings of the International Joint Conference on Neural Networks",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

pages = "682--686",

booktitle = "Proceedings - 1992 International Joint Conference on Neural Networks, IJCNN 1992",

}

TY - GEN

T1 - Theoretical Derivation of Momentum Term in Back-propagation

AU - Hagiwara, Masafumi

PY - 1992

Y1 - 1992

N2 - This paper explains clearly the theoretical origin of a momentum term in Back-propagation algorithm. We will prove that the Back-propagation algorithm, which has a momentum term, can be derived through the following assumptions: 1) The cost function is {equation presented}, where Ep is the sum of squared error at the output layer. 2) The most recent weights are assumed in calculating E n.

AB - This paper explains clearly the theoretical origin of a momentum term in Back-propagation algorithm. We will prove that the Back-propagation algorithm, which has a momentum term, can be derived through the following assumptions: 1) The cost function is {equation presented}, where Ep is the sum of squared error at the output layer. 2) The most recent weights are assumed in calculating E n.

UR - http://www.scopus.com/inward/record.url?scp=0001611924&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=0001611924&partnerID=8YFLogxK

U2 - 10.1109/IJCNN.1992.287108

DO - 10.1109/IJCNN.1992.287108

M3 - Conference contribution

AN - SCOPUS:0001611924

T3 - Proceedings of the International Joint Conference on Neural Networks

SP - 682

EP - 686

BT - Proceedings - 1992 International Joint Conference on Neural Networks, IJCNN 1992

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 1992 International Joint Conference on Neural Networks, IJCNN 1992

Y2 - 7 June 1992 through 11 June 1992

ER -

Theoretical Derivation of Momentum Term in Back-propagation

Abstract

Publication series

Conference

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this