H∞ Optimal Control of Unknown Linear Discrete-Time Systems: An Off-Policy Reinforcement Learning Approach
Abstract
This paper proposes a model-free H∞ control design for linear discrete-time systems using reinforcement learning (RL). A novel off-policy RL algorithm is used to solve the game algebraic Riccati equation (GARE) online using the measured data along the system trajectories. The proposed RL algorithm has the following advantages compared to existing model-free RL methods for solving H∞ control problem: 1) It is data efficient and fast since a stream of experiences which is obtained from executing a fixed behavioral policy is reused to update many value functions correspond to different leaning policies sequentially. 2) The disturbance input does not need to be adjusted in a specific manner. 3) There is no bias as a result of adding a probing noise to the control input to maintain persistence of excitation conditions. A simulation example is used to verify the effectiveness of the proposed control scheme.
Recommended Citation
B. Kiumarsi et al., "H∞ Optimal Control of Unknown Linear Discrete-Time Systems: An Off-Policy Reinforcement Learning Approach," Proceedings of the 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM) (2015, Siem Reap, Cambodia), pp. 41 - 46, Institute of Electrical and Electronics Engineers (IEEE), Jul 2015.
The definitive version is available at https://doi.org/10.1109/ICCIS.2015.7274545
Meeting Name
2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM) (2015: Jul. 15-17, Siem Reap, Cambodia)
Department(s)
Electrical and Computer Engineering
Keywords and Phrases
Algebra; Cybernetics; Digital Control Systems; Intelligent Systems; Reinforcement Learning; Riccati Equations; Robotics; Algebraic Riccati Equations; Control Problems; Linear Discrete-Time Systems; Optimal Controls; Persistence of Excitation; Reinforcement Learning Approach; Simulation Example; System Trajectory; Discrete Time Control Systems; Game Algebraic Riccati Equation; Off-Policy
International Standard Book Number (ISBN)
978-1467373364
International Standard Serial Number (ISSN)
2326-8239; 2326-8123
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2015 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
01 Jul 2015
