Distributed Adaptive Optimal Regulation of Uncertain Large-Scale Linear Networked Control Systems using Q-Learning
Abstract
A novel Q-learning approach is presented for the design of an adaptive optimal regulator for linear large-scale interconnected system. The subsystems communicate among each other through a communication network while another communication network is inserted within the feedback loop of each subsystem. The network induced random delays and data dropouts of the network in the feedback are modelled along with the system dynamics. Stochastic Q-learning is used to adaptively learn the Q-function parameters with periodic and intermittent feedback. For efficient parameter learning with event-sampled feedback, a novel hybrid learning algorithm is proposed. Boundedness of the estimated parameters and asymptotic convergence of state vector in the mean square is achieved and it is demonstrated using Lyapunov stability analysis. Moreover, if the regression function of the QFE is persistently exciting (PE), the estimated parameters converge to their expected target values. The proposed analytical design is validated using a numerical example via simulation.
Recommended Citation
V. Narayanan and J. Sarangapani, "Distributed Adaptive Optimal Regulation of Uncertain Large-Scale Linear Networked Control Systems using Q-Learning," Proceedings of the 2015 IEEE Symposium Series on Computational Intelligence (2015, Cape Town, South Africa), pp. 587 - 592, Institute of Electrical and Electronics Engineers (IEEE), Dec 2015.
The definitive version is available at https://doi.org/10.1109/SSCI.2015.92
Meeting Name
2015 IEEE Symposium Series on Computational Intelligence, SSCI 2015 (2015: Dec. 7-10, Cape Town, South Africa)
Department(s)
Electrical and Computer Engineering
Keywords and Phrases
Artificial intelligence; Networked control systems; Parameter estimation; Stochastic systems; Adaptive regulators; Analytical design; Asymptotic convergence; Estimated parameter; Large-scale interconnected systems; Persistency of excitation; Q-learning approach; Regression vectors; Adaptive control systems
International Standard Book Number (ISBN)
978-1-4799-7560-0
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2015 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
01 Dec 2015
Comments
This research supported in part by NSF ECCS 1128281, 1406533 and Intelligent Systems Center, MST, Rolla.