Event-Driven Off-Policy Reinforcement Learning for Control of Interconnected Systems

Abstract

In this article, we introduce a novel approximate optimal decentralized control scheme for uncertain input-affine nonlinear-interconnected systems. In the proposed scheme, we design a controller and an event-triggering mechanism (ETM) at each subsystem to optimize a local performance index and reduce redundant control updates, respectively. To this end, we formulate a noncooperative dynamic game at every subsystem in which we collectively model the interconnection inputs and the event-triggering error as adversarial players that deteriorate the subsystem performance and model the control policy as the performance optimizer, competing against these adversarial players. To obtain a solution to this game, one has to solve the associated Hamilton-Jacobi-Isaac (HJI) equation, which does not have a closed-form solution even when the subsystem dynamics are accurately known. In this context, we introduce an event-driven off-policy integral reinforcement learning (OIRL) approach to learn an approximate solution to this HJI equation using artificial neural networks (NNs). We then use this NN approximated solution to design the control policy and event-triggering threshold at each subsystem. In the learning framework, we guarantee the Zeno-free behavior of the ETMs at each subsystem using the exploration policies. Finally, we derive sufficient conditions to guarantee uniform ultimate bounded regulation of the controlled system states and demonstrate the efficacy of the proposed framework with numerical examples.

Department(s)

Electrical and Computer Engineering

Comments

This work was supported in part by NSF under Grant ECCS 1406533 and Grant CMMI 1547042.

Keywords and Phrases

Decentralized Control; Differential Game; Event-Triggered Learning; Off-Policy Learning

International Standard Serial Number (ISSN)

2168-2275; 2168-2267

Document Type

Article - Journal

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2022 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Mar 2022

PubMed ID

32639933

Share

 
COinS