Stochastic Optimal Regulation of Nonlinear Networked Control Systems by using Event-Driven Adaptive Dynamic Programming


In this paper, an event-driven stochastic adaptive dynamic programming (ADP)-based technique is introduced for nonlinear systems with a communication network within its feedback loop. A near optimal control policy is designed using an actor-critic framework and ADP with event sampled state vector. First, the system dynamics are approximated by using a novel neural network (NN) identifier with event sampled state vector. The optimal control policy is generated via an actor NN by using the NN identifier and value function approximated by a critic NN through ADP. The stochastic NN identifier, actor, and critic NN weights are tuned at the event sampled instants leading to aperiodic weight tuning laws. Above all, an adaptive event sampling condition based on estimated NN weights is designed by using the Lyapunov technique to ensure ultimate boundedness of all the closed-loop signals along with the approximation accuracy. The net result is event-driven stochastic ADP technique that can significantly reduce the computation and network transmissions. Finally, the analytical design is substantiated with simulation results.


Electrical and Computer Engineering

Research Center/Lab(s)

Intelligent Systems Center


This work was supported in part by the National Science Foundation under Grant ECCS 1406533, and in part by the Intelligent Systems Center.

Keywords and Phrases

Adaptive control systems; Networked control systems; Neural networks; Nonlinear feedback; Stochastic systems; Telecommunication networks; Adaptive dynamic programming (ADP); Approximation accuracy; Near-optimal control; Network transmission; Nonlinear networked control systems; Novel neural network; Optimal control policy; Ultimate boundedness; Dynamic programming; Event sampled control; Neural networks (NNs); Optimal control

International Standard Serial Number (ISSN)

2168-2267; 2168-2275

Document Type

Article - Journal

Document Version


File Type





© 2017 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Feb 2017