Containment Control of Heterogeneous Systems with Active Leaders of Bounded Unknown Control using Reinforcement Learning


This paper solves the containment problem of multi-agent systems on undirected graph with multiple active leaders using off-policy reinforcement learning (RL). The leaders are active in the sense that there exists bounded control input in the dynamics which is unknown to all followers and the followers are heterogeneous with different dynamics. Not only the steady states of agent i but also the transient trajectories are taken into account to impose optimality to the proposed containment control. Inhomogeneous algebraic Riccati equations (ARE) are derived to solve the optimal containment control protocol. To avoid the requirement of agents' dynamics to obtain containment control, an off-policy RL algorithm is developed to solve the inhomogeneous AREs online in real time and without requiring any knowledge of the agents' dynamics. Finally, a simulation example is presented to illustrate the effectiveness of the proposed algorithm.

Meeting Name

2017 IEEE Symposium Series on Computational Intelligence, SSCI 2017 (2017: Nov. 27-Dec. 1, Honolulu, HI)


Electrical and Computer Engineering

Research Center/Lab(s)

Intelligent Systems Center


This work was supported in part by the National Natural Science Foundation of China (NSFC Grant No. 61333002 and No. 61673054), the Mary K. Finley Missouri Endowment, the Missouri S&T Intelligent Systems Center, the National Science Foundation and in part by the Open Research Project from SKLMCCS under Grant 20150104.

Keywords and Phrases

Dynamics; Intelligent agents; Multi agent systems; Riccati equations; Active leaders; Containment control; Distributed observer; Heterogeneous systems; Model free; Reinforcement learning; Model-free

International Standard Book Number (ISBN)


Document Type

Article - Conference proceedings

Document Version


File Type





© 2017 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Nov 2017