Event-Triggered Distributed Control of Nonlinear Interconnected Systems using Online Reinforcement Learning with Exploration
A Reinforcement Learning with Exploration-Based Event-Triggered Distributed Control of Nonlinear Interconnected Systems
In this paper, a distributed control scheme for an interconnected system composed of uncertain input affine nonlinear subsystems with event triggered state feedback is presented by using a novel hybrid learning scheme-based approximate dynamic programming with online exploration. First, an approximate solution to the Hamilton-Jacobi-Bellman equation is generated with event sampled neural network (NN) approximation and subsequently, a near optimal control policy for each subsystem is derived. Artificial NNs are utilized as function approximators to develop a suite of identifiers and learn the dynamics of each subsystem. The NN weight tuning rules for the identifier and event-triggering condition are derived using Lyapunov stability theory. Taking into account, the effects of NN approximation of system dynamics and boot-strapping, a novel NN weight update is presented to approximate the optimal value function. Finally, a novel strategy to incorporate exploration in online control framework, using identifiers, is introduced to reduce the overall cost at the expense of additional computations during the initial online learning phase. System states and the NN weight estimation errors are regulated and local uniformly ultimately bounded results are achieved. The analytical results are substantiated using simulation studies.
V. Narayanan and J. Sarangapani, "Event-Triggered Distributed Control of Nonlinear Interconnected Systems using Online Reinforcement Learning with Exploration," IEEE Transactions on Cybernetics, vol. 48, no. 9, pp. 2510-2519, Institute of Electrical and Electronics Engineers (IEEE), Sep 2018.
The definitive version is available at https://doi.org/10.1109/TCYB.2017.2741342
Electrical and Computer Engineering
Intelligent Systems Center
Keywords and Phrases
Cost functions; Decentralized control; Distributed computer systems; Distributed parameter control systems; E-learning; Large scale systems; Natural resources exploration; Neural networks; Online systems; Reinforcement learning; State feedback; System theory; Uncertainty analysis; Approximate dynamic programming (ADP); Event-triggered controls; Learning (artificial intelligence); Neural network control; Optimal controls; System Dynamics; Dynamic programming; Exploration
International Standard Serial Number (ISSN)
Article - Journal
© 2018 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
01 Sep 2018