Online Reinforcement with Exploration for Distributed Control
Abstract
This paper introduces an online reinforcement learning scheme with exploration for distributed approximate optimal control of uncertain nonlinear interconnected system. The subsystem, interconnection dynamics and input gain matrix are approximated using neural network (NN) identifiers with event-based state feedback. A second NN is designed at each subsystem to construct the mapping of states to future reward prediction via reinforcement signals with which a sequence of approximately optimal distributed control actions are generated. Since the identifiers and the controllers at each subsystem requires the local and other subsystem state vector with non-zero interconnections, a decentralized event-triggering mechanism using Lyapunov theory is developed to dynamically determine the feedback instants so as to reduce the communication overhead. Further, a novel strategy to incorporate exploration in the online control framework using identifiers is proposed to minimize the overall cost during the learning phase. The effects of network delay are discussed and finally, simulation results are presented to verify the effectiveness of the proposed controller.
Recommended Citation
V. Narayanan and J. Sarangapani, "Online Reinforcement with Exploration for Distributed Control," Proceedings of the 2017 International Joint Conference on Neural Networks (2017, Anchorage, AK), pp. 4022 - 4027, Institute of Electrical and Electronics Engineers (IEEE), May 2017.
The definitive version is available at https://doi.org/10.1109/IJCNN.2017.7966363
Meeting Name
2017 International Joint Conference on Neural Networks, IJCNN (2017: May 14-19, Anchorage, AK)
Department(s)
Electrical and Computer Engineering
Research Center/Lab(s)
Intelligent Systems Center
Keywords and Phrases
Controllers; Distributed parameter control systems; Natural resources exploration; Optimal control systems; Reinforcement learning; State feedback; Communication overheads; Decentralized event-triggering; Distributed control; Lyapunov theories; Neural network (NN); Nonlinear interconnected systems; On-line controls; Reinforcement signal; Feedback; Event sampled control; Exploration
International Standard Book Number (ISBN)
978-1-5090-6182-2
International Standard Serial Number (ISSN)
2161-4407
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2017 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
01 May 2017
Comments
Research supported in parts by NSF grants ECCS #1406533 and CMMI #1547042 and Intelligent Systems Center, Rolla.