Acceleration for both Boltzmann Machine Learning and Mean Field Theory Learning

This paper proposes new learning algorithms for both the Boltzmann Machine (BM) learning and the Mean Field Theory (MFT) learning to accelerate their learning speeds. The derivation of the new algorithms are based on the following assumptions: 1) The alternative cost function is {equation presented} where G τis the information-theoretical measure at the learning epoch τ, not G which is the commonly used information-theoretical measure in the derivation of BM learning. 2) The most recent weights are assumed in calculating Gn, which technique is used in the derivation of Recursive Least-Squares (RLS) algorithm. As a result, momentum terms which accelerate learning can be derived in the BM and the MFT learning algorithms. Comparing the proposed MFT learning algorithm with the conventional MFT algorithm by computer simulation, we show the effectiveness of the proposed method.