Abstract
File transfers in high-speed networks require network and I/O parallelism to reach high speeds, however, creating arbitrarily large numbers of I/O and network threads overwhelms system resources and causes fairness issues. In this paper, we introduce Falcon that combines a novel utility function with state-of-the-art online optimization algorithms to discover the degree of I/O and network parallelism for file transfer that can maximize the throughput while keeping system overhead low and ensuring fairness among competing transfers. Our extensive evaluations in several dedicated and production high-speed networks show that Falcon can find near optimal solution in as little as 20 seconds and outperforms existing transfer application by 2-6×. Moreover, unlike other file transfer optimization solutions that fail to ensure fair resource allocation between competing transfers, Falcon is guaranteed to converge to Nash Equilibrium when multiple Falcon agents compete for the network resources with the help of its game theory-inspired utility function.
Recommended Citation
M. Arifuzzaman and E. Arslan, "Online Optimization of File Transfers in High-Speed Networks," Proceedings of the International Conference for High Performance Computing, Networking, Storage and Analysis, Association for Computing Machinery, Nov 2021.
The definitive version is available at https://doi.org/10.1145/3458817.3476208
Department(s)
Computer Science
Publication Status
Public Access
Document Type
Article - Conference proceedings
Document Version
Citation
File Type
text
Language(s)
English
Rights
© 2024 Association for Computing Machinery, All rights reserved.
Publication Date
14 Nov 2021
