Neural Network-Based Finite Horizon Stochastic Optimal Control Design for Nonlinear Networked Control Systems


The stochastic optimal control of nonlinear networked control systems (NNCSs) using neuro-dynamic programming (NDP) over a finite time horizon is a challenging problem due to terminal constraints, system uncertainties, and unknown network imperfections, such as network-induced delays and packet losses. Since the traditional iteration or time-based infinite horizon NDP schemes are unsuitable for NNCS with terminal constraints, a novel time-based NDP scheme is developed to solve finite horizon optimal control of NNCS by mitigating the above-mentioned challenges. First, an online neural network (NN) identifier is introduced to approximate the control coefficient matrix that is subsequently utilized in conjunction with the critic and actor NNs to determine a time-based stochastic optimal control input over finite horizon in a forward-in-time and online manner. Eventually, Lyapunov theory is used to show that all closed-loop signals and NN weights are uniformly ultimately bounded with ultimate bounds being a function of initial conditions and final time. Moreover, the approximated control input converges close to optimal value within finite time. The simulation results are included to show the effectiveness of the proposed scheme.


Electrical and Computer Engineering


This work was supported by the National Science Foundation under Grant ECCS 1128281.

Keywords and Phrases

Control systems; Control theory; Dynamic programming; Optimal control systems; Social networking (online); Stochastic control systems; Stochastic systems; Finite horizon optimal control; Network-induced delays; Neuro-Dynamic Programming; Nonlinear networked control systems; Nonlinear networked control systems (NNCS); On-line neural networks; Stochastic optimal control; Uniformly ultimately bounded; Networked control systems; Artificial neural network; Nonlinear system; Statistics; Time; Neural Networks (Computer); Nonlinear Dynamics; Stochastic Processes; Time Factors; Neuro-dynamic programming (NDP); Nonlinear networked control system (NNCS); Stochastic optimal control

International Standard Serial Number (ISSN)


Document Type

Article - Journal

Document Version


File Type





© 2015 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Mar 2015