A nonaffine discrete-time system represented by the nonlinear autoregressive moving average with eXogenous input (NARMAX) representation with unknown nonlinear system dynamics is considered. An equivalent affinelike representation in terms of the tracking error dynamics is first obtained from the original nonaffine nonlinear discrete-time system so that reinforcement-learning-based near-optimal neural network (NN) controller can be developed. The control scheme consists of two linearly parameterized NNs. One NN is designated as the critic NN, which approximates a predefined long-term cost function, and an action NN is employed to derive a near-optimal control signal for the system to track a desired trajectory while minimizing the cost function simultaneously. The NN weights are tuned online. by using the standard Lyapunov approach, the stability of the closed-loop system is shown. The net result is a supervised actor-critic NN controller scheme which can be applied to a general nonaffine nonlinear discrete-time system without needing the affinelike representation. Simulation results demonstrate satisfactory performance of the controller
Q. Yang et al., "Control of Nonaffine Nonlinear Discrete Time Systems using Reinforcement-learning-Based Linearly Parameterized Neural Networks," IEEE Transactions on Systems, Man, and Cybernetics, Part B, Institute of Electrical and Electronics Engineers (IEEE), Aug 2008.
The definitive version is available at http://dx.doi.org/10.1109/TSMCB.2008.926607
Electrical and Computer Engineering
National Science Foundation (U.S.)
United States. Department of Education
Keywords and Phrases
Lyapunov Stability; Adaptive Critic; Adaptive Dynamic Programming; Neural Network Control; Reinforcement Learning Control
Article - Journal
© 2008 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.