A novel reinforcement-learning based output-adaptive neural network (NN) controller, also referred as the adaptive-critic NN controller, is developed to track a desired trajectory for a class of complex nonlinear discrete-time systems in the presence of bounded and unknown disturbances. The controller includes an observer for estimating states and the outputs, critic, and two action NNs for generating virtual, and actual control inputs. The critic approximates certain strategic utility function and the action NNs are used to minimize both the strategic utility function and their outputs. All NN weights adapt online towards minimization of a performance index, utilizing gradient-descent based rule. A Lyapunov function proves the uniformly ultimate boundedness (UUB) of the closed-loop tracking error, weight, and observer estimation. Separation principle and certainty equivalence principles are relaxed; persistency of excitation condition and linear in the unknown parameter assumption is not needed. The performance of this adaptive critic NN controller is evaluated through simulation with the Daw engine model in lean mode. The objective is to reduce the cyclic dispersion in heat release by using the controller.

Meeting Name

American Control Conference, 2007. ACC'07


Electrical and Computer Engineering

Second Department

Computer Science

Third Department

Mechanical and Aerospace Engineering


National Science Foundation (U.S.)

Keywords and Phrases

Lyapunov Methods; Adaptive Control; Closed Loop Systems; Feedback Function Approximation; Gradient Methods; Internal Combustion Engines; Learning (Artificial Intelligence); Learning Systems; Neurocontrollers; Nonlinear Control Systems; Observers; Discrete-time systems; Large scale systems

Document Type

Article - Conference proceedings

Document Version

Final Version

File Type





© 2007 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Jul 2007