H∞ Optimal Control of Unknown Linear Discrete-Time Systems: An Off-Policy Reinforcement Learning Approach
This paper proposes a model-free H∞ control design for linear discrete-time systems using reinforcement learning (RL). A novel off-policy RL algorithm is used to solve the game algebraic Riccati equation (GARE) online using the measured data along the system trajectories. The proposed RL algorithm has the following advantages compared to existing model-free RL methods for solving H∞ control problem: 1) It is data efficient and fast since a stream of experiences which is obtained from executing a fixed behavioral policy is reused to update many value functions correspond to different leaning policies sequentially. 2) The disturbance input does not need to be adjusted in a specific manner. 3) There is no bias as a result of adding a probing noise to the control input to maintain persistence of excitation conditions. A simulation example is used to verify the effectiveness of the proposed control scheme.
B. Kiumarsi et al., "H∞ Optimal Control of Unknown Linear Discrete-Time Systems: An Off-Policy Reinforcement Learning Approach," Proceedings of the 2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM) (2015, Siem Reap, Cambodia), pp. 41-46, Institute of Electrical and Electronics Engineers (IEEE), Jul 2015.
The definitive version is available at http://dx.doi.org/10.1109/ICCIS.2015.7274545
2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM) (2015: Jul. 15-17, Siem Reap, Cambodia)
Electrical and Computer Engineering
Keywords and Phrases
Algebra; Cybernetics; Digital Control Systems; Intelligent Systems; Reinforcement Learning; Riccati Equations; Robotics; Algebraic Riccati Equations; Control Problems; Linear Discrete-Time Systems; Optimal Controls; Persistence of Excitation; Reinforcement Learning Approach; Simulation Example; System Trajectory; Discrete Time Control Systems; Game Algebraic Riccati Equation; Off-Policy
International Standard Book Number (ISBN)
International Standard Serial Number (ISSN)
Article - Conference proceedings
© 2015 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.