Adaptive Optimal Control of Partially-Unknown Constrained-Input Systems using Policy Iteration with Experience Replay
This paper develops an online learning algorithm to find optimal control solutions for partially-unknown continuous-time systems subject to input constraints. The input constraints are encoded into the optimal control problem through a nonquadratic performance functional. An online policy iteration algorithm that uses integral reinforcement knowledge is developed to learn the solution to the optimal control problem online without knowing the full dynamics model. The policy iteration algorithm is implemented on an actor-critic structure, where two neural network approximators are tuned online and simultaneously to generate the optimal control law. A novel technique based on experience replay is introduced to retain past data in updating the neural network weights. This uses the recorded data concurrently with current data for adaptation of the critic neural network weights. Concurrent learning provides an easy-to-check real-time condition for persistence of excitation that is sufficient to guarantee convergence to a near optimal control law. Stability of the proposed feedback control law is shown and its performance is evaluated through simulations.
H. Modares et al., "Adaptive Optimal Control of Partially-Unknown Constrained-Input Systems using Policy Iteration with Experience Replay," Proceedings of the AIAA Guidance, Navigation, and Control Conference (2013, Boston, MA), pp. 1-11, American Institute of Aeronautics and Astronautics (AIAA), Aug 2013.
AIAA Guidance, Navigation, and Control (GNC) Conference (2013: Aug 19-22, Boston, MA)
Electrical and Computer Engineering
Mechanical and Aerospace Engineering
Keywords and Phrases
Adaptive Optimal Control; Feedback Control Law; Near-Optimal Control; Online Learning Algorithms; Optimal Control Problem; Optimal Control Solution; Persistence of Excitation; Policy Iteration Algorithms; Algorithms; Control; Control Theory; Iterative Methods; Neural Networks; Optimal Control Systems
International Standard Book Number (ISBN)
Article - Conference proceedings
© 2013 American Institute of Aeronautics and Astronautics (AIAA), All rights reserved.