Abstract
High-performance analysis of big data demands more computing resources, forcing similar growth in computation cost. So, the challenge to the HPC system designers is providing not only high performance but also high performance at lower cost. For high performance yet cost-effective cyberinfrastructure, we propose a new system model augmenting Amdahl's second law for balanced system to optimize price-performance-ratio. We express the optimal balance among CPU-speed, I/O-bandwidth and DRAM-size (i.e., Amdahl's I/O-and memory-number) in terms of application characteristics and hardware cost. Considering Xeon processor and recent hardware prices, we showed that a system needs almost 0.17GBPS I/O-bandwidth and 3GB DRAM per GHz CPU-speed to minimize the price-performance-ratio for data-and compute-intensive applications. To substantiate our claim, we evaluate three different cluster architectures: 1) SupermikeII, a traditional HPC cluster, 2) SwatIII, a regular datacenter, and 3) CeresII, a MicroBrick-based novel hyperscale system. CeresII with 6-Xeon-D1541 cores (2GHz/core), 1-NVMe SSD (2GBPS I/O-bandwidth) and 64GB DRAM per node, closely resembles the optimum produced by our model. Consequently, in terms of price-performance-ratio CeresII outperformed both SupermikeII (by 65-85%) and SwatIII (by 40-50%) for data-and compute-intensive Hadoop benchmarks (TeraSort and WordCount) and our own benchmark genome assembler based on Hadoop and Giraph.
Recommended Citation
A. K. Das et al., "Augmenting Amdahl's Second Law: A Theoretical Model To Build Cost-Effective Balanced HPC Infrastructure For Data-Driven Science," IEEE International Conference on Cloud Computing, CLOUD, pp. 147 - 154, article no. 8030583, Institute of Electrical and Electronics Engineers, Sep 2017.
The definitive version is available at https://doi.org/10.1109/CLOUD.2017.27
Department(s)
Computer Science
Keywords and Phrases
Amdahl's Second Law; Balanced HPC; Big data; Giraph; Hadoop; Price-Performance-Ratio
International Standard Book Number (ISBN)
978-153861993-3
International Standard Serial Number (ISSN)
2159-6190; 2159-6182
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2024 Institute of Electrical and Electronics Engineers, All rights reserved.
Publication Date
08 Sep 2017
Comments
National Science Foundation, Grant 1338051