Optimal Containment Control of Unknown Heterogeneous Systems with Active Leaders


This brief presents a partially model-free solution to the distributed containment control of multiagent systems using off-policy reinforcement learning (RL). The followers are assumed to be heterogeneous with different dynamics, and the leaders are assumed to be active in the sense that their control inputs can be nonzero. Optimality is explicitly imposed in solving the containment problem to not only drive the agents' states into a convex hull of the leaders' states but also minimize their transient responses. Inhomogeneous algebraic Riccati equations (AREs) are derived to solve the optimal containment control with active leaders. The resulting control protocol for each agent depends on its own state and an estimation of an interior point inside the convex hull spanned by the leaders. This estimation is provided by designing a distributed observer for a trajectory inside the convex hull of active leaders. Only the knowledge of the leaders' dynamics is required by the observer. An off-policy RL algorithm is developed to solve the inhomogeneous AREs online in real time without requiring any knowledge of the followers' dynamics. Finally, a simulation example is presented to show the effectiveness of the presented algorithm.


Electrical and Computer Engineering

Second Department

Computer Science

Research Center/Lab(s)

Center for High Performance Computing Research


Early Access

Keywords and Phrases

Computational Geometry; Dynamics; Multi Agent Systems; Riccati Equations; Transient Analysis; Active Leaders; Containment Control; Distributed Observer; Heterogeneous Systems; Model Free; Reinforcement Learning; Active Leader; Reinforcement Learning (RL)

International Standard Serial Number (ISSN)


Document Type

Article - Journal

Document Version


File Type





© 2018 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.

Publication Date

01 Jan 2018