A pulse neural network reinforcement learning algorithm for partially observable Markov decision processes

Koichiro Takita, Masafumi Hagiwara

Research output: Contribution to journalArticlepeer-review

4 Citations (Scopus)


This paper considers learning by a pulse neural network and proposes a new reinforcement learning algorithm focusing on the ability of pulse neuron elements to process time series. The conventional integrator neuron element is modeled in terms of the average firing rate of the biological neuron. But the pulse neuron is a modeling of the input-output relation of the time-series pulse (spike) and the decay of the internal state (internal potential). The application of such neural networks has been considered in recent engineering studies. It is known in particular that a pulse neuron with a high decay rate acts as a coincidence detector. The proposed model combines pulse neuron elements with different decay rates, which facilitates the processing of the time-series input information and the discrimination of fuzzy states in a partially observable Markov decision process. The proposed network is a four-layered feedforward network in which the pulse neuron elements forming the two hidden layers provide a pseudo-representation of the state in the environment. The elements generate a secondary reinforcement signal which results in learning similar to the conventional reinforcement scheme based on the state evaluation function. A computer experiment verifies that the proposed model works effectively in an environment which is strongly partially observable.

Original languageEnglish
Pages (from-to)42-52
Number of pages11
JournalSystems and Computers in Japan
Issue number3
Publication statusPublished - 2005 Mar


  • Partially observable Markov decision process
  • Pulse neural network
  • Reinforcement learning

ASJC Scopus subject areas

  • Theoretical Computer Science
  • Information Systems
  • Hardware and Architecture
  • Computational Theory and Mathematics


Dive into the research topics of 'A pulse neural network reinforcement learning algorithm for partially observable Markov decision processes'. Together they form a unique fingerprint.

Cite this