Adaptive Optimal Control of Unknown Constrained-Input Systems using Policy Iteration and Neural Networks

Abstract

This paper presents an online policy iteration (PI) algorithm to learn the continuous-time optimal control solution for unknown constrained-input systems. The proposed PI algorithm is implemented on an actor-critic structure where two neural networks (NNs) are tuned online and simultaneously to generate the optimal bounded control policy. The requirement of complete knowledge of the system dynamics is obviated by employing a novel NN identifier in conjunction with the actor and critic NNs. It is shown how the identifier weights estimation error affects the convergence of the critic NN. A novel learning rule is developed to guarantee that the identifier weights converge to small neighborhoods of their ideal values exponentially fast. To provide an easy-to-check persistence of excitation condition, the experience replay technique is used. That is, recorded past experiences are used simultaneously with current data for the adaptation of the identifier weights. Stability of the whole system consisting of the actor, critic, system state, and system identifier is guaranteed while all three networks undergo adaptation. Convergence to a near-optimal control law is also shown. The effectiveness of the proposed method is illustrated with a simulation example.

Department(s)

Electrical and Computer Engineering

Keywords and Phrases

Adaptive Optimal Control; Input Constraints; Near-Optimal Control; Neural Networks (NNS); Optimal Control Solution; Optimal Controls; Persistence of Excitation; Simulation Example; Algorithms; Control; Neural Networks; Online Systems; Optimal Control Systems; Reinforcement Learning; Iterative Methods; Algorithm; Artificial Intelligence; Artificial Neural Network; Computer Simulation; Feedback System; Learning; Nonlinear System; Signal Processing; Theoretical Model; Feedback; Models; Theoretical; Neural Networks (Computer); Nonlinear Dynamics; Computer-Assisted; Optimal Control; Unknown Dynamics

International Standard Serial Number (ISSN)

2162-237X

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2013 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Oct 2013

Link to Full Text

Share

 
COinS