BeeFlow: A Workflow Management System for in Situ Processing Across HPC and Cloud Systems
Abstract
In this paper, we propose BeeFlow - an in situ analysis enabled workflow management system across multiple platforms using Docker containers. BeeFlow can support both traditional workflows as well as workflows with in situ analysis. BeeFlow leverages Docker containers to provide a portable, flexible, and reproducible workflow management system across HPC and cloud platforms. We showcase how current in situ visualization workflows can apply BeeFlow with DOE production codes VPIC and Flecsale.
Recommended Citation
J. Chen et al., "BeeFlow: A Workflow Management System for in Situ Processing Across HPC and Cloud Systems," Proceedings of the 38th International Conference on Distributed Computing Systems (2018, Vienna, Austria), pp. 1029 - 1038, Institute of Electrical and Electronics Engineers (IEEE), Jul 2018.
The definitive version is available at https://doi.org/10.1109/ICDCS.2018.00103
Meeting Name
38th International Conference on Distributed Computing Systems, ICDCS (2018: Jul. 2-6, Vienna, Austria)
Department(s)
Computer Science
Keywords and Phrases
Cloud Computing; Container; High Performance Computing; Scientific Workflow
International Standard Book Number (ISBN)
978-153866871-9
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2018 Institute of Electrical and Electronics Engineers (IEEE), All rights reserved.
Publication Date
19 Jul 2018
