Distributed Adaptive Optimal Regulation of Uncertain Large-Scale Linear Networked Control Systems using Q-Learning


A novel Q-learning approach is presented for the design of an adaptive optimal regulator for linear large-scale interconnected system. The subsystems communicate among each other through a communication network while another communication network is inserted within the feedback loop of each subsystem. The network induced random delays and data dropouts of the network in the feedback are modelled along with the system dynamics. Stochastic Q-learning is used to adaptively learn the Q-function parameters with periodic and intermittent feedback. For efficient parameter learning with event-sampled feedback, a novel hybrid learning algorithm is proposed. Boundedness of the estimated parameters and asymptotic convergence of state vector in the mean square is achieved and it is demonstrated using Lyapunov stability analysis. Moreover, if the regression function of the QFE is persistently exciting (PE), the estimated parameters converge to their expected target values. The proposed analytical design is validated using a numerical example via simulation.

Meeting Name

2015 IEEE Symposium Series on Computational Intelligence, SSCI 2015 (2015: Dec. 7-10, Cape Town, South Africa)


Electrical and Computer Engineering


This research supported in part by NSF ECCS 1128281, 1406533 and Intelligent Systems Center, MST, Rolla.

Keywords and Phrases

Artificial intelligence; Networked control systems; Parameter estimation; Stochastic systems; Adaptive regulators; Analytical design; Asymptotic convergence; Estimated parameter; Large-scale interconnected systems; Persistency of excitation; Q-learning approach; Regression vectors; Adaptive control systems

International Standard Book Number (ISBN)


Document Type

Article - Conference proceedings

Document Version


File Type





© 2015 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Dec 2015