Abstract
A novel reinforcement-learning-based output adaptive neural network (NN) controller, which is also referred to as the adaptive-critic NN controller, is developed to deliver the desired tracking performance for a class of nonlinear discrete-time systems expressed in nonstrict feedback form in the presence of bounded and unknown disturbances. The adaptive-critic NN controller consists of an observer, a critic, and two action NNs. The observer estimates the states and output, and the two action NNs provide virtual and actual control inputs to the nonlinear discrete-time system. The critic approximates a certain strategic utility function, and the action NNs minimize the strategic utility function and control inputs. All NN weights adapt online toward minimization of a performance index, utilizing the gradient-descent-based rule, in contrast with iteration-based adaptive-critic schemes. Lyapunov functions are used to show the stability of the closed-loop tracking error, weights, and observer estimates. Separation and certainty equivalence principles, persistency of excitation condition, and linearity in the unknown parameter assumption are not needed. Experimental results on a spark ignition (SI) engine operating lean at an equivalence ratio of 0.75 show a significant (25%) reduction in cyclic dispersion in heat release with control, while the average fuel input changes by less than 1% compared with the uncontrolled case. Consequently, oxides of nitrogen (NOx) drop by 30%, and unburned hydrocarbons drop by 16% with control. Overall, NOx's are reduced by over 80% compared with stoichiometric levels.
Recommended Citation
P. Shih et al., "Reinforcement-Learning-Based Output-Feedback Control of Nonstrict Nonlinear Discrete-Time Systems with Application to Engine Emission Control," IEEE Transactions on Systems, Man, and Cybernetics: Part B, Institute of Electrical and Electronics Engineers (IEEE), Oct 2009.
The definitive version is available at https://doi.org/10.1109/TSMCB.2009.2013272
Department(s)
Electrical and Computer Engineering
Second Department
Computer Science
Third Department
Mechanical and Aerospace Engineering
Sponsor(s)
National Science Foundation (U.S.)
Keywords and Phrases
Adaptive Critic; Discrete-Time System; Engine Emission Control; Nonstrict Nonlinear Output Feedback; Reinforcement Learning Control
Document Type
Article - Conference proceedings
Document Version
Final Version
File Type
text
Language(s)
English
Rights
© 2009 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
01 Oct 2009
Included in
Aerospace Engineering Commons, Computer Sciences Commons, Electrical and Computer Engineering Commons, Mechanical Engineering Commons, Operations Research, Systems Engineering and Industrial Engineering Commons
