Online Solution of Nonquadratic Two-Player Zero-Sum Games Arising in the H∞ Control of Constrained Input Systems
In this paper, we present an online learning algorithm to find the solution to the H∞ control problem of continuous-time systems with input constraints. A suitable nonquadratic functional is utilized to encode the input constraints into the H∞ control problem, and the related H∞ control problem is formulated as a two-player zero-sum game with a nonquadratic performance. Then, a policy iteration algorithm on an actor-critic-disturbance structure is developed to solve the Hamilton-Jacobi-Isaacs (HJI) equation associated with this nonquadratic zero-sum game. That is, three NN approximators, namely, actor, critic, and disturbance, are tuned online and simultaneously for approximating the HJI solution. The value of the actor and disturbance policies is approximated continuously by the critic NN, and then on the basis of this value estimate, the actor and disturbance NNs are updated in real time to improve their policies. The disturbance tries to make the worst possible disturbance, whereas the actor tries to make the best control input. A persistence of excitation condition is shown to guarantee convergence to the optimal saddle point solution. Stability of the closed-loop system is also guaranteed. A simulation on a nonlinear benchmark problem is performed to validate the effectiveness of the proposed approach.
H. Modares et al., "Online Solution of Nonquadratic Two-Player Zero-Sum Games Arising in the H∞ Control of Constrained Input Systems," International Journal of Adaptive Control and Signal Processing, vol. 28, no. 3-5, pp. 232-254, John Wiley & Sons, Mar 2014.
The definitive version is available at http://dx.doi.org/10.1002/acs.2348
Electrical and Computer Engineering
Keywords and Phrases
Algorithms; Control; Game Theory; Iterative Methods; Neural Networks; Nonlinear Control Systems; Constrained Input Systems; Input Constraints; Non-Linear Benchmark Problems; Online Learning Algorithms; Persistence of Excitation; Policy Iteration; Policy Iteration Algorithms; Zero-Sum Game; Online Systems; Two-Player Zero-Sum Games
International Standard Serial Number (ISSN)
Article - Journal
© 2014 John Wiley & Sons, All rights reserved.