Approximate Optimal Control of Affine Nonlinear Continuous-Time Systems using Event-Sampled Neurodynamic Programming


This paper presents an approximate optimal control of nonlinear continuous-time systems in affine form by using the adaptive dynamic programming (ADP) with event-sampled state and input vectors. The knowledge of the system dynamics is relaxed by using a neural network (NN) identifier with event-sampled inputs. The value function, which becomes an approximate solution to the Hamilton-Jacobi-Bellman equation, is generated by using event-sampled NN approximator. Subsequently, the NN identifier and the approximated value function are utilized to obtain the optimal control policy. Both the identifier and value function approximator weights are tuned only at the event-sampled instants leading to an aperiodic update scheme. A novel adaptive event sampling condition is designed to determine the sampling instants, such that the approximation accuracy and the stability are maintained. A positive lower bound on the minimum inter-sample time is guaranteed to avoid accumulation point, and the dependence of inter-sample time upon the NN weight estimates is analyzed. A local ultimate boundedness of the resulting nonlinear impulsive dynamical closed-loop system is shown. Finally, a numerical example is utilized to evaluate the performance of the near-optimal design. The net result is the design of an event-sampled ADP-based controller for nonlinear continuous-time systems.


Electrical and Computer Engineering

Research Center/Lab(s)

Intelligent Systems Center


This work was supported in part by the National Science Foundation under Grant ECCS 1406533 and in part by the Intelligent Systems Center, Missouri Science and Technology.

Keywords and Phrases

Adaptive control systems; Closed loop systems; Dynamic programming; Nonlinear systems; Adaptive dynamic programming; Approximate solution; Approximation accuracy; Hamilton Jacobi Bellman equation; Neuro dynamic programming; Nonlinear continuous-time systems; Optimal control policy; Ultimate boundedness; Continuous time systems; Adaptive dynamic programming (ADP); Event-sampled control; Hamilton-Jacobi-Bellman (HJB) equation; Neural networks (NNs); Optimal control

International Standard Serial Number (ISSN)

2162-237X; 2162-2388

Document Type

Article - Journal

Document Version


File Type





© 2017 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Mar 2017