Scaling Read Aligners to Hundreds of Threads on General-purpose Processors

Overview

Journal Bioinformatics

Publisher Oxford University Press

Specialty Biology

Date 2018 Jul 19

PMID 30020410

Citations 341

Authors

Ben Langmead

Christopher Wilks

Valentin Antonescu

Rone Charles

Affiliations

Soon will be listed here.

Abstract

Motivation: General-purpose processors can now contain many dozens of processor cores and support hundreds of simultaneous threads of execution. To make best use of these threads, genomics software must contend with new and subtle computer architecture issues. We discuss some of these and propose methods for improving thread scaling in tools that analyze each read independently, such as read aligners.

Results: We implement these methods in new versions of Bowtie, Bowtie 2 and HISAT. We greatly improve thread scaling in many scenarios, including on the recent Intel Xeon Phi architecture. We also highlight how bottlenecks are exacerbated by variable-record-length file formats like FASTQ and suggest changes that enable superior scaling.

Availability And Implementation: Experiments for this study: https://github.com/BenLangmead/bowtie-scaling.

Bowtie: http://bowtie-bio.sourceforge.net.

Bowtie 2: http://bowtie-bio.sourceforge.net/bowtie2.

Hisat: http://www.ccb.jhu.edu/software/hisat.

Supplementary Information: Supplementary data are available at Bioinformatics online.

Citing Articles

Heart rate variability, daily cortisol indices and their association with psychometric characteristics and gut microbiota composition in an Italian community sample.

Ravenda S, Mancabelli L, Gambetta S, Barbetti M, Turroni F, Carnevali L Sci Rep. 2025; 15(1):8584.

PMID: 40074815 PMC: 11903775. DOI: 10.1038/s41598-025-93137-8.

EBAX-1/ZSWIM8 destabilizes miRNAs, resulting in transgenerational inheritance of a predatory trait.

Quiobe S, Kalirad A, Roseler W, Witte H, Wang Y, Rodelsperger C Sci Adv. 2025; 11(11):eadu0875.

PMID: 40073139 PMC: 11900880. DOI: 10.1126/sciadv.adu0875.

Estropausal gut microbiota transplant improves measures of ovarian function in adult mice.

Kim M, Wang J, Pilley S, Lu R, Xu A, Kim Y bioRxiv. 2025; .

PMID: 40060387 PMC: 11888174. DOI: 10.1101/2024.05.03.592475.

Effect of fieldwork-friendly coffee blender-based extraction methods and leaf tissue storage on the transcriptome of non-model plants.

Dagva S, Galipon J J Plant Res. 2025; .

PMID: 40053276 DOI: 10.1007/s10265-025-01624-w.

Genome-wide CRISPR guide RNA design and specificity analysis with GuideScan2.

Schmidt H, Zhang M, Chakarov D, Bansal V, Mourelatos H, Sanchez-Rivera F Genome Biol. 2025; 26(1):41.

PMID: 40011959 PMC: 11863968. DOI: 10.1186/s13059-025-03488-8.

References

Langmead B, Trapnell C, Pop M, Salzberg S . Ultrafast and memory-efficient alignment of short DNA sequences to the human genome. Genome Biol. 2009; 10(3):R25. PMC: 2690996. DOI: 10.1186/gb-2009-10-3-r25. View

Rustagi N, Zhou A, Watkins W, Gedvilaite E, Wang S, Ramesh N . Extremely low-coverage whole genome sequencing in South Asians captures population genomics information. BMC Genomics. 2017; 18(1):396. PMC: 5440948. DOI: 10.1186/s12864-017-3767-6. View

Cock P, Fields C, Goto N, Heuer M, Rice P . The Sanger FASTQ file format for sequences with quality scores, and the Solexa/Illumina FASTQ variants. Nucleic Acids Res. 2009; 38(6):1767-71. PMC: 2847217. DOI: 10.1093/nar/gkp1137. View

Eberle M, Fritzilas E, Krusche P, Kallberg M, Moore B, Bekritsky M . A reference data set of 5.4 million phased human variants validated by genetic inheritance from sequencing a three-generation 17-member pedigree. Genome Res. 2016; 27(1):157-164. PMC: 5204340. DOI: 10.1101/gr.210500.116. View

Srivastava A, Sarkar H, Gupta N, Patro R . RapMap: a rapid, sensitive and accurate tool for mapping RNA-seq reads to transcriptomes. Bioinformatics. 2016; 32(12):i192-i200. PMC: 4908361. DOI: 10.1093/bioinformatics/btw277. View

Wood D, Salzberg S . Kraken: ultrafast metagenomic sequence classification using exact alignments. Genome Biol. 2014; 15(3):R46. PMC: 4053813. DOI: 10.1186/gb-2014-15-3-r46. View

Church D, Schneider V, Steinberg K, Schatz M, Quinlan A, Chin C . Extending reference assembly models. Genome Biol. 2015; 16:13. PMC: 4305238. DOI: 10.1186/s13059-015-0587-3. View

Li H . Tabix: fast retrieval of sequence features from generic TAB-delimited files. Bioinformatics. 2011; 27(5):718-9. PMC: 3042176. DOI: 10.1093/bioinformatics/btq671. View

Langmead B, Salzberg S . Fast gapped-read alignment with Bowtie 2. Nat Methods. 2012; 9(4):357-9. PMC: 3322381. DOI: 10.1038/nmeth.1923. View

10.

Marco-Sola S, Sammeth M, Guigo R, Ribeca P . The GEM mapper: fast, accurate and versatile alignment by filtration. Nat Methods. 2012; 9(12):1185-8. DOI: 10.1038/nmeth.2221. View

11.

Auton A, Brooks L, Durbin R, Garrison E, Kang H, Korbel J . A global reference for human genetic variation. Nature. 2015; 526(7571):68-74. PMC: 4750478. DOI: 10.1038/nature15393. View

12.

Kim D, Langmead B, Salzberg S . HISAT: a fast spliced aligner with low memory requirements. Nat Methods. 2015; 12(4):357-60. PMC: 4655817. DOI: 10.1038/nmeth.3317. View

13.

Li H, Handsaker B, Wysoker A, Fennell T, Ruan J, Homer N . The Sequence Alignment/Map format and SAMtools. Bioinformatics. 2009; 25(16):2078-9. PMC: 2723002. DOI: 10.1093/bioinformatics/btp352. View

14.

Kelley D, Schatz M, Salzberg S . Quake: quality-aware detection and correction of sequencing errors. Genome Biol. 2010; 11(11):R116. PMC: 3156955. DOI: 10.1186/gb-2010-11-11-r116. View

15.

Segata N, Waldron L, Ballarini A, Narasimhan V, Jousson O, Huttenhower C . Metagenomic microbial community profiling using unique clade-specific marker genes. Nat Methods. 2012; 9(8):811-4. PMC: 3443552. DOI: 10.1038/nmeth.2066. View

16.

Waldrop M . The chips are down for Moore's law. Nature. 2016; 530(7589):144-7. DOI: 10.1038/530144a. View

17.

Misale C, Ferrero G, Torquati M, Aldinucci M . Sequence alignment tools: one parallel pattern to rule them all?. Biomed Res Int. 2014; 2014:539410. PMC: 4131566. DOI: 10.1155/2014/539410. View

18.

Bolger A, Lohse M, Usadel B . Trimmomatic: a flexible trimmer for Illumina sequence data. Bioinformatics. 2014; 30(15):2114-20. PMC: 4103590. DOI: 10.1093/bioinformatics/btu170. View

19.

Bray N, Pimentel H, Melsted P, Pachter L . Near-optimal probabilistic RNA-seq quantification. Nat Biotechnol. 2016; 34(5):525-7. DOI: 10.1038/nbt.3519. View

20.

Song L, Florea L, Langmead B . Lighter: fast and memory-efficient sequencing error correction without counting. Genome Biol. 2014; 15(11):509. PMC: 4248469. DOI: 10.1186/s13059-014-0509-9. View