H Optimal Control of Unknown Linear Discrete-Time Systems: An Off-Policy Reinforcement Learning Approach


This paper proposes a model-free H control design for linear discrete-time systems using reinforcement learning (RL). A novel off-policy RL algorithm is used to solve the game algebraic Riccati equation (GARE) online using the measured data along the system trajectories. The proposed RL algorithm has the following advantages compared to existing model-free RL methods for solving H control problem: 1) It is data efficient and fast since a stream of experiences which is obtained from executing a fixed behavioral policy is reused to update many value functions correspond to different leaning policies sequentially. 2) The disturbance input does not need to be adjusted in a specific manner. 3) There is no bias as a result of adding a probing noise to the control input to maintain persistence of excitation conditions. A simulation example is used to verify the effectiveness of the proposed control scheme.

Meeting Name

2015 IEEE 7th International Conference on Cybernetics and Intelligent Systems (CIS) and IEEE Conference on Robotics, Automation and Mechatronics (RAM) (2015: Jul. 15-17, Siem Reap, Cambodia)


Electrical and Computer Engineering

Keywords and Phrases

Algebra; Cybernetics; Digital Control Systems; Intelligent Systems; Reinforcement Learning; Riccati Equations; Robotics; Algebraic Riccati Equations; Control Problems; Linear Discrete-Time Systems; Optimal Controls; Persistence of Excitation; Reinforcement Learning Approach; Simulation Example; System Trajectory; Discrete Time Control Systems; Game Algebraic Riccati Equation; Off-Policy

International Standard Book Number (ISBN)


International Standard Serial Number (ISSN)

2326-8239; 2326-8123

Document Type

Article - Conference proceedings

Document Version


File Type





© 2015 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Jul 2015