Abstract

Existing coflow scheduling frameworks effectively shorten communication time and completion time of cluster applications. However, existing frameworks only consider available bandwidth on hosts and overlook congestion in the network when making scheduling decisions. Through extensive simulations using the realistic workload probability distribution from Facebook, we observe the performance degradation of the state-of-the-art coflow scheduling framework, Varys, in the cloud environment on a shared data center network (DCN) because of the lack of network congestion information. We propose Coflourish, the first coflow scheduling framework that exploits the congestion feedback assistances from the software-defined-networking (SDN)-enabled switches in the networks for available bandwidth estimation. Our simulation results demonstrate that Coflourish outperforms Varys by up to 75.5% in terms of average coflow completion time under various workload conditions. The proposed work also reveals the potentials of integration with traffic engineering mechanisms in lower levels for further performance optimization.

Department(s)

Computer Science

Comments

National Science Foundation, Grant 1341008

Keywords and Phrases

Application-aware Networks; Cloud Computing; Coflow Scheduling; Data Center Networks; Software-defined Networking

International Standard Book Number (ISBN)

978-153861993-3

International Standard Serial Number (ISSN)

2159-6190; 2159-6182

Document Type

Article - Conference proceedings

Document Version

Citation

File Type

text

Language(s)

English

Rights

© 2024 Institute of Electrical and Electronics Engineers, All rights reserved.

Publication Date

08 Sep 2017

Download

Full Text Link

Share

 
COinS