Deep Reinforcement Learning with Dual Targeting Algorithm

Naoki Kodama, Taku Harada, Kazuteru Miyazaki

研究成果: Conference contribution査読

1 被引用数 (Scopus)

抄録

Recently, deep reinforcement learning using the Deep Q-networks (DQN) algorithm has attracted attention, and extended methods continue to improve its learning performance. A multi-step DQN using an n-step TD method in the extended method contributes to faster learning. However, in the n-step TD methods, improvement in learning speed is better when using the intermediate prediction over the long-term prediction. Therefore, to further accelerate learning, methods that can use a long-term prediction effectively are required. A learning-accelerated DQN learns faster than DQN through a training neural network with bootstrap targets up to the next positive reward and 1-step bootstrap targets. It is, however, not possible for that method to use long-term prediction for tasks in which rewards are continuously observed. Furthermore, the use of two independent updates leads to instability with respect to the convergence of the neural network. We therefore propose a dual targeting algorithm that uses a single update with bootstrap targets up to the last reward in the next consecutive positive reward and 1-step bootstrap targets. The aim of the proposed method is to reduce instability in the convergence of the neural network by calculating the dual target from the same sampled experience. We apply the proposed method to a few classic control problems involving OpenAI Gym, compare it with DQN and multi-step DQN, and verify its effectiveness.

本文言語English
ホスト出版物のタイトル2019 International Joint Conference on Neural Networks, IJCNN 2019
出版社Institute of Electrical and Electronics Engineers Inc.
ISBN(電子版)9781728119854
DOI
出版ステータスPublished - 7月 2019
イベント2019 International Joint Conference on Neural Networks, IJCNN 2019 - Budapest, Hungary
継続期間: 14 7月 201919 7月 2019

出版物シリーズ

名前Proceedings of the International Joint Conference on Neural Networks
2019-July

Conference

Conference2019 International Joint Conference on Neural Networks, IJCNN 2019
国/地域Hungary
CityBudapest
Period14/07/1919/07/19

フィンガープリント

「Deep Reinforcement Learning with Dual Targeting Algorithm」の研究トピックを掘り下げます。これらがまとまってユニークなフィンガープリントを構成します。

引用スタイル