Event-Triggered Distributed Control of Nonlinear Interconnected Systems using Online Reinforcement Learning with Exploration
Alternative Title
A Reinforcement Learning with Exploration-Based Event-Triggered Distributed Control of Nonlinear Interconnected Systems
Abstract
In this paper, a distributed control scheme for an interconnected system composed of uncertain input affine nonlinear subsystems with event triggered state feedback is presented by using a novel hybrid learning scheme-based approximate dynamic programming with online exploration. First, an approximate solution to the Hamilton-Jacobi-Bellman equation is generated with event sampled neural network (NN) approximation and subsequently, a near optimal control policy for each subsystem is derived. Artificial NNs are utilized as function approximators to develop a suite of identifiers and learn the dynamics of each subsystem. The NN weight tuning rules for the identifier and event-triggering condition are derived using Lyapunov stability theory. Taking into account, the effects of NN approximation of system dynamics and boot-strapping, a novel NN weight update is presented to approximate the optimal value function. Finally, a novel strategy to incorporate exploration in online control framework, using identifiers, is introduced to reduce the overall cost at the expense of additional computations during the initial online learning phase. System states and the NN weight estimation errors are regulated and local uniformly ultimately bounded results are achieved. The analytical results are substantiated using simulation studies.
Recommended Citation
V. Narayanan and J. Sarangapani, "Event-Triggered Distributed Control of Nonlinear Interconnected Systems using Online Reinforcement Learning with Exploration," IEEE Transactions on Cybernetics, vol. 48, no. 9, pp. 2510 - 2519, Institute of Electrical and Electronics Engineers (IEEE), Sep 2018.
The definitive version is available at https://doi.org/10.1109/TCYB.2017.2741342
Department(s)
Electrical and Computer Engineering
Research Center/Lab(s)
Intelligent Systems Center
Keywords and Phrases
Cost functions; Decentralized control; Distributed computer systems; Distributed parameter control systems; E-learning; Large scale systems; Natural resources exploration; Neural networks; Online systems; Reinforcement learning; State feedback; System theory; Uncertainty analysis; Approximate dynamic programming (ADP); Event-triggered controls; Learning (artificial intelligence); Neural network control; Optimal controls; System Dynamics; Dynamic programming; Exploration
International Standard Serial Number (ISSN)
2168-2267; 2168-2275
Document Type
Article - Journal
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2018 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
01 Sep 2018
Comments
This work was supported in part by the Intelligent Systems Center at Rolla, in part by NSF under Grant ECCS 1406533, and in part by CMMI under Grant 1547042.