» Articles » PMID: 37471333

Ten Quick Tips for Bioinformatics Analyses Using an Apache Spark Distributed Computing Environment

Overview
Specialty Biology
Date 2023 Jul 20
PMID 37471333
Authors
Affiliations
Soon will be listed here.
Abstract

Some scientific studies involve huge amounts of bioinformatics data that cannot be analyzed on personal computers usually employed by researchers for day-to-day activities but rather necessitate effective computational infrastructures that can work in a distributed way. For this purpose, distributed computing systems have become useful tools to analyze large amounts of bioinformatics data and to generate relevant results on virtual environments, where software can be executed for hours or even days without affecting the personal computer or laptop of a researcher. Even if distributed computing resources have become pivotal in multiple bioinformatics laboratories, often researchers and students use them in the wrong ways, making mistakes that can cause the distributed computers to underperform or that can even generate wrong outcomes. In this context, we present here ten quick tips for the usage of Apache Spark distributed computing systems for bioinformatics analyses: ten simple guidelines that, if taken into account, can help users avoid common mistakes and can help them run their bioinformatics analyses smoothly. Even if we designed our recommendations for beginners and students, they should be followed by experts too. We think our quick tips can help anyone make use of Apache Spark distributed computing systems more efficiently and ultimately help generate better, more reliable scientific results.

References
1.
Ferraro Petrillo U, Sorella M, Cattaneo G, Giancarlo R, Rombo S . Analyzing big datasets of genomic sequences: fast and scalable collection of k-mer statistics. BMC Bioinformatics. 2019; 20(Suppl 4):138. PMC: 6471689. DOI: 10.1186/s12859-019-2694-8. View

2.
Roguski L, Deorowicz S . DSRC 2--Industry-oriented compression of FASTQ files. Bioinformatics. 2014; 30(15):2213-5. DOI: 10.1093/bioinformatics/btu208. View

3.
Ferraro Petrillo U, Roscigno G, Cattaneo G, Giancarlo R . Informational and linguistic analysis of large genomic sequence collections via efficient Hadoop cluster algorithms. Bioinformatics. 2018; 34(11):1826-1833. DOI: 10.1093/bioinformatics/bty018. View

4.
Chicco D, Agapito G . Nine quick tips for pathway enrichment analysis. PLoS Comput Biol. 2022; 18(8):e1010348. PMC: 9371296. DOI: 10.1371/journal.pcbi.1010348. View

5.
Davis S, Meltzer P . GEOquery: a bridge between the Gene Expression Omnibus (GEO) and BioConductor. Bioinformatics. 2007; 23(14):1846-7. DOI: 10.1093/bioinformatics/btm254. View