H∞ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning
This paper deals with the design of an H∞ tracking controller for nonlinear continuous-time systems with completely unknown dynamics. A general bounded L2-gain tracking problem with a discounted performance function is introduced for the H∞ tracking. A tracking Hamilton-Jacobi-Isaac (HJI) equation is then developed that gives a Nash equilibrium solution to the associated min-max optimization problem. A rigorous analysis of bounded L2-gain and stability of the control solution obtained by solving the tracking HJI equation is provided. An upper-bound is found for the discount factor to assure local asymptotic stability of the tracking error dynamics. An off-policy reinforcement learning algorithm is used to learn the solution to the tracking HJI equation online without requiring any knowledge of the system dynamics. Convergence of the proposed algorithm to the solution to the tracking HJI equation is shown. Simulation examples are provided to verify the effectiveness of the proposed method.
H. Modares et al., "H∞ Tracking Control of Completely Unknown Continuous-Time Systems via Off-Policy Reinforcement Learning," IEEE Transactions on Neural Networks and Learning Systems, vol. 26, no. 10, pp. 2550-2562, Institute of Electrical and Electronics Engineers (IEEE), Oct 2015.
The definitive version is available at http://dx.doi.org/10.1109/TNNLS.2015.2441749
Electrical and Computer Engineering
Keywords and Phrases
Asymptotic Stability; Controllers; Learning Algorithms; Nonlinear Control Systems; Optimization; Reinforcement Learning; Telecommunication Networks; Bounded L2-Gain; Hamilton-Jacobi; Local Asymptotic Stability; Min-Max Optimization; Nonlinear Continuous-Time Systems; Performance Functions; Tracking Controller; Tracking Error Dynamics; Continuous Time Systems; Bounded L2-Gain; HÃ¢Ë†Å¾ Tracking Controller; Reinforcement Learning (RL); Tracking Hamilton-Jacobi-Isaac (HJI) Equation
International Standard Serial Number (ISSN)
Article - Journal
© 2015 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.