Optimal and Autonomous Control using Reinforcement Learning: A Survey

Author

Bahare Kiumarsi
Kyriakos G. Vamvoudakis
Hamidreza Modares, Missouri University of Science and TechnologyFollow
Frank L. Lewis

Abstract

This paper reviews the current state of the art on reinforcement learning (RL)-based feedback control solutions to optimal regulation and tracking of single and multiagent systems. Existing RL solutions to both optimal H₂ and H_∞ control problems, as well as graphical games, will be reviewed. RL methods learn the solution to optimal control and game problems online and using measured data along the system trajectories. We discuss Q-learning and the integral RL algorithm as core algorithms for discrete-time (DT) and continuous-time (CT) systems, respectively. Moreover, we discuss a new direction of off-policy RL for both CT and DT systems. Finally, we review several applications.

Recommended Citation

B. Kiumarsi et al., "Optimal and Autonomous Control using Reinforcement Learning: A Survey," IEEE Transactions on Neural Networks and Learning Systems, pp. 1 - 21, Institute of Electrical and Electronics Engineers (IEEE), Dec 2017.

The definitive version is available at https://doi.org/10.1109/TNNLS.2017.2773458

Department(s)

Electrical and Computer Engineering

Comments

Early Access

Keywords and Phrases

Approximation Algorithms; Continuous Time Systems; Discrete Time Control Systems; Heuristic Algorithms; Learning Algorithms; Multi Agent Systems; Optimal Control Systems; Optimization; Algorithm Design and Analysis; Autonomy; Data-Based Optimization; Games; Learning (artificial Intelligence); Optimal Controls; System Dynamics; Reinforcement Learning; Optimal Control; Reinforcement Learning (RL)

International Standard Serial Number (ISSN)

2162-237X

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2017 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Dec 2017