Third-generation DNA sequencing technologies such as single-molecule real-time sequencing (SMRT) and nanopore sequencing have the potential to fill the gaps in the existing genome databases since the raw sequences produced by these machines are much longer than those of previous generations and therefore result in more contiguous assemblies. However, these long reads have a high error rate, which makes the assembly process computationally challenging. Moreover, since existing long-read assemblers are designed to run on a single machine, they either take days to complete or run out of memory on even moderate-sized datasets. In this paper, we present a distributed long-read assembler that can assemble large-scale noisy sequence datasets on thousands of cores, resulting in orders of magnitude faster assembly times. By effectively using the map-reduce computation model with a distributed hash-map, both built using a high-performance active messaging middleware, we can assemble a PacBio human genome dataset with 139 billion base-pairs (about 130 GB) in about 33 minutes (using 2,560 cores) compared to more than 38 hours (using 28 cores) with the current state-of-the-art assembler.


Computer Science


National Science Foundation, Grant IBSS-L-1620451

Keywords and Phrases

big data; genome assembly; high-performance computing; long reads; map-reduce; third-generation sequences

International Standard Book Number (ISBN)


Document Type

Article - Conference proceedings

Document Version


File Type





© 2024 Institute of Electrical and Electronics Engineers, All rights reserved.

Publication Date

10 Dec 2020