Metadata Retrieval from Sequence Databases with Ffq

Overview

Journal Bioinformatics

Publisher Oxford University Press

Specialty Biology

Date 2023 Jan 7

PMID 36610997

Authors

Angel Galvez-Merchan

Kyung Hoi Joseph Min

Lior Pachter

A Sina Booeshaghi

Affiliations

Soon will be listed here.

Abstract

Motivation: Several genomic databases host data and metadata for an ever-growing collection of sequence datasets. While these databases have a shared hierarchical structure, there are no tools specifically designed to leverage it for metadata extraction.

Results: We present a command-line tool, called ffq, for querying user-generated data and metadata from sequence databases. Given an accession or a paper's DOI, ffq efficiently fetches metadata and links to raw data in JSON format. ffq's modularity and simplicity make it extensible to any genomic database exposing its data for programmatic access.

Availability And Implementation: ffq is free and open source, and the code can be found here: https://github.com/pachterlab/ffq.

Citing Articles

iSeq: an integrated tool to fetch public sequencing data.

Chao H, Li Z, Chen D, Chen M Bioinformatics. 2024; 40(11).

PMID: 39447029 PMC: 11561040. DOI: 10.1093/bioinformatics/btae641.

Meta-analysis of the Microbial Diversity Cultured in Bioreactors Simulating the Gut Microbiome.

Mendez D, Egan S, Wist J, Holmes E, Sanabria J Microb Ecol. 2024; 87(1):57.

PMID: 38587527 PMC: 11001690. DOI: 10.1007/s00248-024-02369-0.

GINSA: an accumulator for paired locality and next-generation small ribosomal subunit sequence data.

Odle E, Kahng S, Riewluang S, Kurihara K, Wakeman K Bioinformatics. 2024; 40(4).

PMID: 38502961 PMC: 10987208. DOI: 10.1093/bioinformatics/btae152.

Quantifying orthogonal barcodes for sequence census assays.

Booeshaghi A, Min K, Gehring J, Pachter L Bioinform Adv. 2024; 4(1):vbad181.

PMID: 38213823 PMC: 10783946. DOI: 10.1093/bioadv/vbad181.

Efficient and accurate detection of viral sequences at single-cell resolution reveals putative novel viruses perturbing host gene expression.

Luebbert L, Sullivan D, Carilli M, Hjorleifsson K, Viloria Winnett A, Chari T bioRxiv. 2024; .

PMID: 38168363 PMC: 10760059. DOI: 10.1101/2023.12.11.571168.

References

. Database Resources of the National Genomics Data Center, China National Center for Bioinformation in 2022. Nucleic Acids Res. 2021; 50(D1):D27-D38. PMC: 8728233. DOI: 10.1093/nar/gkab951. View

Wartmann H, Heins S, Kloiber K, Bonn S . Bias-invariant RNA-sequencing metadata annotation. Gigascience. 2021; 10(9). PMC: 8559615. DOI: 10.1093/gigascience/giab064. View

Luebbert L, Pachter L . Efficient querying of genomic reference databases with gget. Bioinformatics. 2023; 39(1). PMC: 9835474. DOI: 10.1093/bioinformatics/btac836. View

Klie A, Tsui B, Mollah S, Skola D, Dow M, Hsu C . Increasing metadata coverage of SRA BioSample entries using deep learning-based named entity recognition. Database (Oxford). 2021; 2021. PMC: 8083811. DOI: 10.1093/database/baab021. View

Davis C, Hitz B, Sloan C, Chan E, Davidson J, Gabdank I . The Encyclopedia of DNA elements (ENCODE): data portal update. Nucleic Acids Res. 2017; 46(D1):D794-D801. PMC: 5753278. DOI: 10.1093/nar/gkx1081. View

Hippen A, Greene C . Expanding and Remixing the Metadata Landscape. Trends Cancer. 2020; 7(4):276-278. PMC: 8324015. DOI: 10.1016/j.trecan.2020.10.011. View

Al Mahi N, Najafabadi M, Pilarczyk M, Kouril M, Medvedovic M . GREIN: An Interactive Web Platform for Re-analyzing GEO RNA-seq Data. Sci Rep. 2019; 9(1):7580. PMC: 6527554. DOI: 10.1038/s41598-019-43935-8. View

Bernstein M, Doan A, Dewey C . MetaSRA: normalized human sample-specific metadata for the Sequence Read Archive. Bioinformatics. 2017; 33(18):2914-2923. PMC: 5870770. DOI: 10.1093/bioinformatics/btx334. View

Rajesh A, Chang Y, Abedalthagafi M, Wong-Beringer A, Love M, Mangul S . Improving the completeness of public metadata accompanying omics studies. Genome Biol. 2021; 22(1):106. PMC: 8048353. DOI: 10.1186/s13059-021-02332-z. View

10.

Zhu Y, Stephens R, Meltzer P, Davis S . SRAdb: query and use public next-generation sequencing data from within R. BMC Bioinformatics. 2013; 14:19. PMC: 3560148. DOI: 10.1186/1471-2105-14-19. View

11.

Lung P, Zhong D, Pang X, Li Y, Zhang J . Maximizing the reusability of gene expression data by predicting missing metadata. PLoS Comput Biol. 2020; 16(11):e1007450. PMC: 7673503. DOI: 10.1371/journal.pcbi.1007450. View

12.

Bernstein M, Gladstein A, Latt K, Clough E, Busby B, Dillman A . Jupyter notebook-based tools for building structured datasets from the Sequence Read Archive. F1000Res. 2020; 9:376. PMC: 7445559. DOI: 10.12688/f1000research.23180.2. View

13.

Simon L, Karg S, Westermann A, Engel M, Elbehery A, Hense B . MetaMap: an atlas of metatranscriptomic reads in human disease-related RNA-seq data. Gigascience. 2018; 7(6). PMC: 6025204. DOI: 10.1093/gigascience/giy070. View

14.

Wang Z, Lachmann A, Maayan A . Mining data and metadata from the gene expression omnibus. Biophys Rev. 2018; 11(1):103-110. PMC: 6381352. DOI: 10.1007/s12551-018-0490-8. View

15.

. An integrated encyclopedia of DNA elements in the human genome. Nature. 2012; 489(7414):57-74. PMC: 3439153. DOI: 10.1038/nature11247. View

16.

Kasmanas J, Bartholomaus A, Correa F, Tal T, Jehmlich N, Herberth G . HumanMetagenomeDB: a public repository of curated and standardized metadata for human metagenomes. Nucleic Acids Res. 2020; 49(D1):D743-D750. PMC: 7778935. DOI: 10.1093/nar/gkaa1031. View

17.

Li Z, Li J, Yu P . GEOMetaCuration: a web-based application for accurate manual curation of Gene Expression Omnibus metadata. Database (Oxford). 2018; 2018. PMC: 5868185. DOI: 10.1093/database/bay019. View

18.

Chen T, Chen X, Zhang S, Zhu J, Tang B, Wang A . The Genome Sequence Archive Family: Toward Explosive Data Growth and Diverse Data Types. Genomics Proteomics Bioinformatics. 2021; 19(4):578-583. PMC: 9039563. DOI: 10.1016/j.gpb.2021.08.001. View

19.

Choudhary S . pysradb: A Python package to query next-generation sequencing metadata and data from NCBI Sequence Read Archive. F1000Res. 2019; 8:532. PMC: 6505635. DOI: 10.12688/f1000research.18676.1. View

20.

Melsted P, Booeshaghi A, Liu L, Gao F, Lu L, Min K . Modular, efficient and constant-memory single-cell RNA-seq preprocessing. Nat Biotechnol. 2021; 39(7):813-818. DOI: 10.1038/s41587-021-00870-2. View