Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy

Yosuke Fukuchi; Masahiko Osawa; Hiroshi Yamakawa; Michita Imai

doi:10.1007/978-3-319-70087-8_11

Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy

Yosuke Fukuchi, Masahiko Osawa, Hiroshi Yamakawa, Michita Imai

Department of Information and Computer Science

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

3 Citations (Scopus)

Abstract

Agents that acquire their own policies autonomously have the risk of accidents caused by the agents’ unexpected behavior. Therefore, it is necessary to improve the predictability of the agents’ behavior in order to ensure the safety. Instruction-based Behavior Explanation (IBE) is a method for a reinforcement learning agent to announce the agent’s future behavior. However, it was not verified that the IBE was applicable to an agent that changes the policy dynamically. In this paper, we consider agents under training and improve the IBE for the application to agents with changing policy. We conducted an experiment to verify if the behavior explanation model of an immature agent worked even after the agent’s further training. The results indicated the applicability of the improved IBE to agents under training.

Original language	English
Title of host publication	Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings
Publisher	Springer Verlag
Pages	100-108
Number of pages	9
Volume	10634 LNCS
ISBN (Print)	9783319700861
DOIs	https://doi.org/10.1007/978-3-319-70087-8_11
Publication status	Published - 2017 Jan 1
Event	24th International Conference on Neural Information Processing, ICONIP 2017 - Guangzhou, China Duration: 2017 Nov 14 → 2017 Nov 18

Publication series

Name	Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)
Volume	10634 LNCS
ISSN (Print)	0302-9743
ISSN (Electronic)	1611-3349

Other

Other	24th International Conference on Neural Information Processing, ICONIP 2017
Country/Territory	China
City	Guangzhou
Period	17/11/14 → 17/11/18

Keywords

Instruction-based behavior explanation (IBE)
Reinforcement learning

ASJC Scopus subject areas

Theoretical Computer Science
Computer Science(all)

Access to Document

10.1007/978-3-319-70087-8_11

Cite this

Fukuchi, Y., Osawa, M., Yamakawa, H., & Imai, M. (2017). Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy. In Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings (Vol. 10634 LNCS, pp. 100-108). (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10634 LNCS). Springer Verlag. https://doi.org/10.1007/978-3-319-70087-8_11

Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy. / Fukuchi, Yosuke; Osawa, Masahiko; Yamakawa, Hiroshi et al.
Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. Vol. 10634 LNCS Springer Verlag, 2017. p. 100-108 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics); Vol. 10634 LNCS).

Research output: Chapter in Book/Report/Conference proceeding › Conference contribution

Fukuchi, Y, Osawa, M, Yamakawa, H & Imai, M 2017, Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy. in Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. vol. 10634 LNCS, Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics), vol. 10634 LNCS, Springer Verlag, pp. 100-108, 24th International Conference on Neural Information Processing, ICONIP 2017, Guangzhou, China, 17/11/14. https://doi.org/10.1007/978-3-319-70087-8_11

Fukuchi Y, Osawa M, Yamakawa H, Imai M. Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy. In Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. Vol. 10634 LNCS. Springer Verlag. 2017. p. 100-108. (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)). doi: 10.1007/978-3-319-70087-8_11

Fukuchi, Yosuke ; Osawa, Masahiko ; Yamakawa, Hiroshi et al. / Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy. Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings. Vol. 10634 LNCS Springer Verlag, 2017. pp. 100-108 (Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)).

@inproceedings{7aba8d8f337e448fa98b63e09feef22f,

title = "Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy",

abstract = "Agents that acquire their own policies autonomously have the risk of accidents caused by the agents{\textquoteright} unexpected behavior. Therefore, it is necessary to improve the predictability of the agents{\textquoteright} behavior in order to ensure the safety. Instruction-based Behavior Explanation (IBE) is a method for a reinforcement learning agent to announce the agent{\textquoteright}s future behavior. However, it was not verified that the IBE was applicable to an agent that changes the policy dynamically. In this paper, we consider agents under training and improve the IBE for the application to agents with changing policy. We conducted an experiment to verify if the behavior explanation model of an immature agent worked even after the agent{\textquoteright}s further training. The results indicated the applicability of the improved IBE to agents under training.",

keywords = "Instruction-based behavior explanation (IBE), Reinforcement learning",

author = "Yosuke Fukuchi and Masahiko Osawa and Hiroshi Yamakawa and Michita Imai",

year = "2017",

month = jan,

day = "1",

doi = "10.1007/978-3-319-70087-8_11",

language = "English",

isbn = "9783319700861",

volume = "10634 LNCS",

series = "Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)",

publisher = "Springer Verlag",

pages = "100--108",

booktitle = "Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings",

address = "Germany",

note = "24th International Conference on Neural Information Processing, ICONIP 2017 ; Conference date: 14-11-2017 Through 18-11-2017",

}

TY - GEN

T1 - Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy

AU - Fukuchi, Yosuke

AU - Osawa, Masahiko

AU - Yamakawa, Hiroshi

AU - Imai, Michita

PY - 2017/1/1

Y1 - 2017/1/1

N2 - Agents that acquire their own policies autonomously have the risk of accidents caused by the agents’ unexpected behavior. Therefore, it is necessary to improve the predictability of the agents’ behavior in order to ensure the safety. Instruction-based Behavior Explanation (IBE) is a method for a reinforcement learning agent to announce the agent’s future behavior. However, it was not verified that the IBE was applicable to an agent that changes the policy dynamically. In this paper, we consider agents under training and improve the IBE for the application to agents with changing policy. We conducted an experiment to verify if the behavior explanation model of an immature agent worked even after the agent’s further training. The results indicated the applicability of the improved IBE to agents under training.

AB - Agents that acquire their own policies autonomously have the risk of accidents caused by the agents’ unexpected behavior. Therefore, it is necessary to improve the predictability of the agents’ behavior in order to ensure the safety. Instruction-based Behavior Explanation (IBE) is a method for a reinforcement learning agent to announce the agent’s future behavior. However, it was not verified that the IBE was applicable to an agent that changes the policy dynamically. In this paper, we consider agents under training and improve the IBE for the application to agents with changing policy. We conducted an experiment to verify if the behavior explanation model of an immature agent worked even after the agent’s further training. The results indicated the applicability of the improved IBE to agents under training.

KW - Instruction-based behavior explanation (IBE)

KW - Reinforcement learning

UR - http://www.scopus.com/inward/record.url?scp=85035146881&partnerID=8YFLogxK

UR - http://www.scopus.com/inward/citedby.url?scp=85035146881&partnerID=8YFLogxK

U2 - 10.1007/978-3-319-70087-8_11

DO - 10.1007/978-3-319-70087-8_11

M3 - Conference contribution

AN - SCOPUS:85035146881

SN - 9783319700861

VL - 10634 LNCS

T3 - Lecture Notes in Computer Science (including subseries Lecture Notes in Artificial Intelligence and Lecture Notes in Bioinformatics)

SP - 100

EP - 108

BT - Neural Information Processing - 24th International Conference, ICONIP 2017, Proceedings

PB - Springer Verlag

T2 - 24th International Conference on Neural Information Processing, ICONIP 2017

Y2 - 14 November 2017 through 18 November 2017

ER -

Application of instruction-based behavior explanation to a reinforcement learning agent with changing policy

Abstract

Publication series

Other

Keywords

ASJC Scopus subject areas

Access to Document

Other files and links

Fingerprint

Cite this