Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices

Song Ju Kim; Kaori Ohkoda; Masashi Aono; Hisashi Shima; Makoto Takahashi; Yasuhisa Naitoh; Hiroyuki Akinaga

doi:10.1109/IRPS.2019.8720428

Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices

Song Ju Kim, Kaori Ohkoda, Masashi Aono, Hisashi Shima, Makoto Takahashi, Yasuhisa Naitoh, Hiroyuki Akinaga

Faculty of Environment and Information Studies

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

2 Citations (Scopus)

Abstract

Reinforcement learning algorithms are widely used in many practical artificial-intelligence (AI) applications. Herein, we propose a compact hardware-based learning system comprising analogue resistive memory devices called 'resistive analogue neuromorphic devices (RAND).' We begin by showing that the resistance of a RAND linearly varies with the voltage pulses, meaning that a single RAND is sufficient to solve a reinforcement learning problem (i.e. the '2-armed bandit problem') using 'tug-of-war (TOW) dynamics.' Next, we show that 2^k-armed bandit problems are also solved by hierarchically combining 2^k-1 RANDs. Finally, we numerically demonstrate that the proposed methods are promising as compared with conventional methods.

Original language	English
Title of host publication	2019 IEEE International Reliability Physics Symposium, IRPS 2019
Publisher	Institute of Electrical and Electronics Engineers Inc.
ISBN (Electronic)	9781538695043
DOIs	https://doi.org/10.1109/IRPS.2019.8720428
Publication status	Published - 2019 May 22
Event	2019 IEEE International Reliability Physics Symposium, IRPS 2019 - Monterey, United States Duration: 2019 Mar 31 → 2019 Apr 4

Publication series

Name	IEEE International Reliability Physics Symposium Proceedings
Volume	2019-March
ISSN (Print)	1541-7026

Conference

Conference	2019 IEEE International Reliability Physics Symposium, IRPS 2019
Country/Territory	United States
City	Monterey
Period	19/3/31 → 19/4/4

Keywords

Decision making
Natural computing
RAND
Reinforcement learning
TOW dynamics

ASJC Scopus subject areas

Engineering(all)

Access to Document

10.1109/IRPS.2019.8720428

Cite this

Kim, S. J., Ohkoda, K., Aono, M., Shima, H., Takahashi, M., Naitoh, Y., & Akinaga, H. (2019). Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices. In 2019 IEEE International Reliability Physics Symposium, IRPS 2019 Article 8720428 (IEEE International Reliability Physics Symposium Proceedings; Vol. 2019-March). Institute of Electrical and Electronics Engineers Inc.. https://doi.org/10.1109/IRPS.2019.8720428

Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices. / Kim, Song Ju; Ohkoda, Kaori; Aono, Masashi et al.
2019 IEEE International Reliability Physics Symposium, IRPS 2019. Institute of Electrical and Electronics Engineers Inc., 2019. 8720428 (IEEE International Reliability Physics Symposium Proceedings; Vol. 2019-March).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Kim, SJ, Ohkoda, K, Aono, M, Shima, H, Takahashi, M, Naitoh, Y & Akinaga, H 2019, Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices. in 2019 IEEE International Reliability Physics Symposium, IRPS 2019., 8720428, IEEE International Reliability Physics Symposium Proceedings, vol. 2019-March, Institute of Electrical and Electronics Engineers Inc., 2019 IEEE International Reliability Physics Symposium, IRPS 2019, Monterey, United States, 19/3/31. https://doi.org/10.1109/IRPS.2019.8720428

Kim SJ, Ohkoda K, Aono M, Shima H, Takahashi M, Naitoh Y et al. Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices. In 2019 IEEE International Reliability Physics Symposium, IRPS 2019. Institute of Electrical and Electronics Engineers Inc. 2019. 8720428. (IEEE International Reliability Physics Symposium Proceedings). doi: 10.1109/IRPS.2019.8720428

@inproceedings{8fb4d0e9630d489a9dc244cf397fff87,

title = "Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices",

abstract = "Reinforcement learning algorithms are widely used in many practical artificial-intelligence (AI) applications. Herein, we propose a compact hardware-based learning system comprising analogue resistive memory devices called 'resistive analogue neuromorphic devices (RAND).' We begin by showing that the resistance of a RAND linearly varies with the voltage pulses, meaning that a single RAND is sufficient to solve a reinforcement learning problem (i.e. the '2-armed bandit problem') using 'tug-of-war (TOW) dynamics.' Next, we show that 2k-armed bandit problems are also solved by hierarchically combining 2k-1 RANDs. Finally, we numerically demonstrate that the proposed methods are promising as compared with conventional methods.",

keywords = "Decision making, Natural computing, RAND, Reinforcement learning, TOW dynamics",

author = "Kim, {Song Ju} and Kaori Ohkoda and Masashi Aono and Hisashi Shima and Makoto Takahashi and Yasuhisa Naitoh and Hiroyuki Akinaga",

note = "Funding Information: This work is based on results obtained from a project commissioned by NEDO (Grant Number: 16101009-0). Publisher Copyright: {\textcopyright} 2019 IEEE.; 2019 IEEE International Reliability Physics Symposium, IRPS 2019 ; Conference date: 31-03-2019 Through 04-04-2019",

year = "2019",

month = may,

day = "22",

doi = "10.1109/IRPS.2019.8720428",

language = "English",

series = "IEEE International Reliability Physics Symposium Proceedings",

publisher = "Institute of Electrical and Electronics Engineers Inc.",

booktitle = "2019 IEEE International Reliability Physics Symposium, IRPS 2019",

}

TY - GEN

T1 - Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices

AU - Kim, Song Ju

AU - Ohkoda, Kaori

AU - Aono, Masashi

AU - Shima, Hisashi

AU - Takahashi, Makoto

AU - Naitoh, Yasuhisa

AU - Akinaga, Hiroyuki

PY - 2019/5/22

Y1 - 2019/5/22

N2 - Reinforcement learning algorithms are widely used in many practical artificial-intelligence (AI) applications. Herein, we propose a compact hardware-based learning system comprising analogue resistive memory devices called 'resistive analogue neuromorphic devices (RAND).' We begin by showing that the resistance of a RAND linearly varies with the voltage pulses, meaning that a single RAND is sufficient to solve a reinforcement learning problem (i.e. the '2-armed bandit problem') using 'tug-of-war (TOW) dynamics.' Next, we show that 2k-armed bandit problems are also solved by hierarchically combining 2k-1 RANDs. Finally, we numerically demonstrate that the proposed methods are promising as compared with conventional methods.

AB - Reinforcement learning algorithms are widely used in many practical artificial-intelligence (AI) applications. Herein, we propose a compact hardware-based learning system comprising analogue resistive memory devices called 'resistive analogue neuromorphic devices (RAND).' We begin by showing that the resistance of a RAND linearly varies with the voltage pulses, meaning that a single RAND is sufficient to solve a reinforcement learning problem (i.e. the '2-armed bandit problem') using 'tug-of-war (TOW) dynamics.' Next, we show that 2k-armed bandit problems are also solved by hierarchically combining 2k-1 RANDs. Finally, we numerically demonstrate that the proposed methods are promising as compared with conventional methods.

KW - Decision making

KW - Natural computing

KW - RAND

KW - Reinforcement learning

KW - TOW dynamics

UR - http://www.scopus.com/inward/record.url?scp=85066745105&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85066745105&partnerID=8YFLogxK

U2 - 10.1109/IRPS.2019.8720428

DO - 10.1109/IRPS.2019.8720428

M3 - Conference contribution

AN - SCOPUS:85066745105

T3 - IEEE International Reliability Physics Symposium Proceedings

BT - 2019 IEEE International Reliability Physics Symposium, IRPS 2019

PB - Institute of Electrical and Electronics Engineers Inc.

T2 - 2019 IEEE International Reliability Physics Symposium, IRPS 2019

Y2 - 31 March 2019 through 4 April 2019

ER -

Reinforcement Learning System Comprising Resistive Analog Neuromorphic Devices

Abstract

Publication series

Conference

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this