Adaptive Optimal Control of Partially-Unknown Constrained-Input Systems using Policy Iteration with Experience Replay

Abstract

This paper develops an online learning algorithm to find optimal control solutions for partially-unknown continuous-time systems subject to input constraints. The input constraints are encoded into the optimal control problem through a nonquadratic performance functional. An online policy iteration algorithm that uses integral reinforcement knowledge is developed to learn the solution to the optimal control problem online without knowing the full dynamics model. The policy iteration algorithm is implemented on an actor-critic structure, where two neural network approximators are tuned online and simultaneously to generate the optimal control law. A novel technique based on experience replay is introduced to retain past data in updating the neural network weights. This uses the recorded data concurrently with current data for adaptation of the critic neural network weights. Concurrent learning provides an easy-to-check real-time condition for persistence of excitation that is sufficient to guarantee convergence to a near optimal control law. Stability of the proposed feedback control law is shown and its performance is evaluated through simulations.

Meeting Name

AIAA Guidance, Navigation, and Control (GNC) Conference (2013: Aug 19-22, Boston, MA)

Department(s)

Electrical and Computer Engineering

Second Department

Mechanical and Aerospace Engineering

Keywords and Phrases

Adaptive Optimal Control; Feedback Control Law; Near-Optimal Control; Online Learning Algorithms; Optimal Control Problem; Optimal Control Solution; Persistence of Excitation; Policy Iteration Algorithms; Algorithms; Control; Control Theory; Iterative Methods; Neural Networks; Optimal Control Systems

International Standard Book Number (ISBN)

978-1624102240

Document Type

Article - Conference proceedings

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2013 American Institute of Aeronautics and Astronautics (AIAA), All rights reserved.

Publication Date

01 Aug 2013

This document is currently not available here.

Share

 
COinS