In this paper, a novel neural network (NN) based online reinforcement learning controller is designed for nonaffine nonlinear discrete-time systems with bounded disturbances. The nonaffine systems are represented by nonlinear auto regressive moving average with exogenous input (NARMAX) model with unknown nonlinear functions. An equivalent affine-like representation for the tracking error dynamics is developed first from the original nonaffine system. Subsequently, a reinforcement learning-based neural network (NN) controller is proposed for the affine-like nonlinear error dynamic system. The control scheme consists of two NNs. One NN is designated as the critic, which approximates a predefined long-term cost function, whereas an action NN is employed to derive a control signal for the system to track a desired trajectory while minimizing the cost function simultaneously. Offline NN training is not required and online NN weight tuning rules are derived. By using the standard Lyapunov approach, the uniformly ultimate boundedness (UUB) of the tracking error and weight estimates is demonstrated.
Q. Yang and J. Sarangapani, "Online Reinforcement Learning Control of Unknown Nonaffine Nonlinear Discrete Time Systems," Proceedings of the 46th IEEE Conference on Decision and Control, Institute of Electrical and Electronics Engineers (IEEE), Jan 2007.
The definitive version is available at https://doi.org/10.1109/CDC.2007.4434959
46th IEEE Conference on Decision and Control
Electrical and Computer Engineering
National Science Foundation (U.S.)
University of Missouri--Rolla. Intelligent Systems Center
Keywords and Phrases
Lyapunov Methods; Autoregressive Moving Average Processes; Control System Synthesis; Learning Systems; Neurocontrollers; Nonlinear Control Systems; Performance Index; Discrete-time systems
Article - Conference proceedings
© 2007 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
01 Jan 2007