Model-Free Temporal Difference Learning for Non-Zero-Sum Games
Abstract
In this paper, we consider the two-player nonzero-sum games problem for continuous-time linear dynamic systems. It is shown that the non-zero-sum games problem results in solving the coupled algebraic Riccati equations, which are nonlinear algebraic matrix equations. Compared with the algebraic Riccati equation of the linear dynamic systems with only one player, the coupled algebraic Riccati equations of nonzero-sum games with multi-player are more difficult to be solved directly. First, the policy iteration algorithm is introduced to find the Nash equilibrium of the non-zero-sum games, which is the sufficient and necessary condition to solve the coupled algebraic Riccati equations. However, the policy iteration algorithm is offline and requires complete knowledge of the system dynamics. To overcome the above issues, a novel online iterative algorithm, named integral temporal difference learning algorithm, is developed. Moreover, an equivalent compact form of the integral temporal difference learning algorithm is also presented. It is shown that the integral temporal difference learning algorithm can be implemented in an online fashion and requires only partial knowledge of the system dynamics. In addition, in each iteration step, the closed-loop stability using the integral temporal difference learning algorithm is analyzed. Finally, the simulation study shows the effectiveness of the presented algorithm.
Recommended Citation
L. Wang et al., "Model-Free Temporal Difference Learning for Non-Zero-Sum Games," Proceedings of the International Joint Conference on Neural Networks (2019, Budapest, Hungary), Institute of Electrical and Electronics Engineers (IEEE), Jul 2019.
The definitive version is available at https://doi.org/10.1109/IJCNN.2019.8851866
Meeting Name
2019 International Joint Conference on Neural Networks, IJCNN 2019 (2019: Jul. 14-19, Budapest, Hungary)
Department(s)
Electrical and Computer Engineering
Research Center/Lab(s)
Center for High Performance Computing Research
Keywords and Phrases
Integral Temporal Difference Learning; Nash Equilibrium; Non-Zero-Sum Games; Value Iteration
International Standard Book Number (ISBN)
978-172811985-4
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2019 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
01 Jul 2019
