Best practices for benchmarking germline small-variant calls in human genomes.

Abstract:

:Standardized benchmarking approaches are required to assess the accuracy of variants called from sequence data. Although variant-calling tools and the metrics used to assess their performance continue to improve, important challenges remain. Here, as part of the Global Alliance for Genomics and Health (GA4GH), we present a benchmarking framework for variant calling. We provide guidance on how to match variant calls with different representations, define standard performance metrics, and stratify performance by variant type and genome context. We describe limitations of high-confidence calls and regions that can be used as truth sets (for example, single-nucleotide variant concordance of two methods is 99.7% inside versus 76.5% outside high-confidence regions). Our web-based app enables comparison of variant calls against truth sets to obtain a standardized performance report. Our approach has been piloted in the PrecisionFDA variant-calling challenges to identify the best-in-class variant-calling methods within high-confidence regions. Finally, we recommend a set of best practices for using our tools and evaluating the results.

journal_name

Nat Biotechnol

journal_title

Nature biotechnology

authors

Krusche P,Trigg L,Boutros PC,Mason CE,De La Vega FM,Moore BL,Gonzalez-Porta M,Eberle MA,Tezak Z,Lababidi S,Truty R,Asimenos G,Funke B,Fleharty M,Chapman BA,Salit M,Zook JM,Global Alliance for Genomics and Health Benchma

doi

10.1038/s41587-019-0054-x

subject

Has Abstract

pub_date

2019-05-01 00:00:00

pages

555-560

issue

5

eissn

1087-0156

issn

1546-1696

pii

10.1038/s41587-019-0054-x

journal_volume

37

pub_type

杂志文章
  • Directed evolution of proteins by exon shuffling.

    abstract::Evolution of eukaryotes is mediated by sexual recombination of parental genomes. Crossovers occur in random, but homologous, positions at a frequency that depends on DNA length. As exons occupy only 1% of the human genome and introns about 24%, by far most of the crossovers occur between exons, rather than inside. The...

    journal_title:Nature biotechnology

    pub_type: 杂志文章,评审

    doi:10.1038/88084

    authors: Kolkman JA,Stemmer WP

    更新日期:2001-05-01 00:00:00

  • Efficient hammerhead ribozyme and antisense RNA targeting in a slow ribosome Escherichia coli mutant.

    abstract::We have evaluated inhibition of the plasmid-born chloramphenicol acetyl transferase gene (CAT) by the hammerhead ribozyme and antisense RNA in Escherichia coli where the translation and transcription rates have been modified. Whereas neither antisense nor the hammerhead had an inhibitory effect on CAT activity in wild...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt0597-432

    authors: Chen H,Ferbeyre G,Cedergren R

    更新日期:1997-05-01 00:00:00

  • Artificial cell-cell communication in yeast Saccharomyces cerevisiae using signaling elements from Arabidopsis thaliana.

    abstract::The construction of synthetic cell-cell communication networks can improve our quantitative understanding of naturally occurring signaling pathways and enhance our capabilities to engineer coordinated cellular behavior in cell populations. Towards accomplishing these goals in eukaryotes, we developed and analyzed two ...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt1162

    authors: Chen MT,Weiss R

    更新日期:2005-12-01 00:00:00

  • Insecticidal proteins from Bacillus thuringiensis protect corn from corn rootworms.

    abstract::Field tests of corn co-expressing two new delta-endotoxins from Bacillus thuringiensis (Bt) have demonstrated protection from root damage by western corn rootworm (Diabrotica virgifera virgifera LeConte). The level of protection exceeds that provided by chemical insecticides. In the bacterium, these proteins form crys...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/90282

    authors: Moellenbeck DJ,Peters ML,Bing JW,Rouse JR,Higgins LS,Sims L,Nevshemal T,Marshall L,Ellis RT,Bystrak PG,Lang BA,Stewart JL,Kouba K,Sondag V,Gustafson V,Nour K,Xu D,Swenson J,Zhang J,Czapla T,Schwab G,Jayne S,St

    更新日期:2001-07-01 00:00:00

  • Resolution of cutaneous inflammation after local elimination of macrophages.

    abstract::We constructed an immunotoxin, composed of an antibody directed against the high-affinity IgG receptor CD64 and Ricin-A, with the aim of resolving chronic inflammation through elimination of activated macrophages. In vitro, this immunotoxin proved very efficient in inducing apoptosis in activated macrophages, leaving ...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/71908

    authors: Thepen T,van Vuuren AJ,Kiekens RC,Damen CA,Vooijs WC,van De Winkel JG

    更新日期:2000-01-01 00:00:00

  • Data storage in DNA with fewer synthesis cycles using composite DNA letters.

    abstract::The density and long-term stability of DNA make it an appealing storage medium, particularly for long-term data archiving. Existing DNA storage technologies involve the synthesis and sequencing of multiple nominally identical molecules in parallel, resulting in information redundancy. We report the development of enco...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/s41587-019-0240-x

    authors: Anavy L,Vaknin I,Atar O,Amit R,Yakhini Z

    更新日期:2019-10-01 00:00:00

  • Genome architectures revealed by tethered chromosome conformation capture and population-based modeling.

    abstract::We describe tethered conformation capture (TCC), a method for genome-wide mapping of chromatin interactions. By performing ligations on solid substrates rather than in solution, TCC substantially enhances the signal-to-noise ratio, thereby facilitating a detailed analysis of interactions within and between chromosomes...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt.2057

    authors: Kalhor R,Tjong H,Jayathilaka N,Alber F,Chen L

    更新日期:2011-12-25 00:00:00

  • Wavelength-shifting molecular beacons.

    abstract::We describe wavelength-shifting molecular beacons, which are nucleic acid hybridization probes that fluoresce in a variety of different colors, yet are excited by a common monochromatic light source. The twin functions of absorption of energy from the excitation light and emission of that energy in the form of fluores...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/81192

    authors: Tyagi S,Marras SA,Kramer FR

    更新日期:2000-11-01 00:00:00

  • Problems in monitoring horizontal gene transfer in field trials of transgenic plants.

    abstract::Transgenic crops are approved for release in some countries, while many more countries are wrestling with the issue of how to conduct risk assessments. Controls on field trials often include monitoring of horizontal gene transfer (HGT) from crops to surrounding soil microorganisms. Our analysis of antibiotic-resistant...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt1009

    authors: Heinemann JA,Traavik T

    更新日期:2004-09-01 00:00:00

  • StringTie enables improved reconstruction of a transcriptome from RNA-seq reads.

    abstract::Methods used to sequence the transcriptome often produce more than 200 million short sequences. We introduce StringTie, a computational method that applies a network flow algorithm originally developed in optimization theory, together with optional de novo assembly, to assemble these complex data sets into transcripts...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt.3122

    authors: Pertea M,Pertea GM,Antonescu CM,Chang TC,Mendell JT,Salzberg SL

    更新日期:2015-03-01 00:00:00

  • A catalog of the mouse gut metagenome.

    abstract::We established a catalog of the mouse gut metagenome comprising ∼2.6 million nonredundant genes by sequencing DNA from fecal samples of 184 mice. To secure high microbiome diversity, we used mouse strains of diverse genetic backgrounds, from different providers, kept in different housing laboratories and fed either a ...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt.3353

    authors: Xiao L,Feng Q,Liang S,Sonne SB,Xia Z,Qiu X,Li X,Long H,Zhang J,Zhang D,Liu C,Fang Z,Chou J,Glanville J,Hao Q,Kotowska D,Colding C,Licht TR,Wu D,Yu J,Sung JJ,Liang Q,Li J,Jia H,Lan Z,Tremaroli V,Dworzyn

    更新日期:2015-10-01 00:00:00

  • Recent Supreme Court decisions and licensing power.

    abstract::The recently decided eBay and MedImmune cases interject both reassurance and uncertainty into the present patent licensing landscape, affecting the strategic decisions to be considered during negotiations. ...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt0208-183

    authors: Giordano-Coltart J,Calkins CW

    更新日期:2008-02-01 00:00:00

  • Systematic comparison of single-cell and single-nucleus RNA-sequencing methods.

    abstract::The scale and capabilities of single-cell RNA-sequencing methods have expanded rapidly in recent years, enabling major discoveries and large-scale cell mapping efforts. However, these methods have not been systematically and comprehensively benchmarked. Here, we directly compare seven methods for single-cell and/or si...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/s41587-020-0465-8

    authors: Ding J,Adiconis X,Simmons SK,Kowalczyk MS,Hession CC,Marjanovic ND,Hughes TK,Wadsworth MH,Burks T,Nguyen LT,Kwon JYH,Barak B,Ge W,Kedaigle AJ,Carroll S,Li S,Hacohen N,Rozenblatt-Rosen O,Shalek AK,Villani AC,Regev

    更新日期:2020-06-01 00:00:00

  • Microarray reality checks in the context of a complex disease.

    abstract::A problem in analyzing microarray-based gene expression data is the separation of genes causally involved in a disease from innocent bystander genes, whose expression levels have been secondarily altered by primary changes elsewhere. To investigate this issue systematically in the context of a class of complex human d...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt965

    authors: Miklos GL,Maleszka R

    更新日期:2004-05-01 00:00:00

  • Lipid production in Nannochloropsis gaditana is doubled by decreasing expression of a single transcriptional regulator.

    abstract::Lipid production in the industrial microalga Nannochloropsis gaditana exceeds that of model algal species and can be maximized by nutrient starvation in batch culture. However, starvation halts growth, thereby decreasing productivity. Efforts to engineer N. gaditana strains that can accumulate biomass and overproduce ...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt.3865

    authors: Ajjawi I,Verruto J,Aqui M,Soriaga LB,Coppersmith J,Kwok K,Peach L,Orchard E,Kalb R,Xu W,Carlson TJ,Francis K,Konigsfeld K,Bartalis J,Schultz A,Lambert W,Schwartz AS,Brown R,Moellering ER

    更新日期:2017-07-01 00:00:00

  • Prevention of systemic lupus erythematosus in MRL/lpr mice by administration of an immunoglobulin-binding peptide.

    abstract::Systemic lupus erythematosus (SLE) is a multisystem chronic inflammatory disease of unknown etiology that affects many organs, including the kidney. The presence of multiple autoantibodies and other immunological abnormalities point to basic defects in immunoregulatory controls that normally maintain self-tolerance. T...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/77296

    authors: Marino M,Ruvo M,De Falco S,Fassina G

    更新日期:2000-07-01 00:00:00

  • Excision of selectable marker genes from transgenic plants.

    abstract::Selectable marker genes are required to ensure the efficient genetic modification of crops. Economic incentives and safety concerns have prompted the development of several strategies (site-specific recombination, homologous recombination, transposition, and co-transformation) to eliminate these genes from the genome ...

    journal_title:Nature biotechnology

    pub_type: 杂志文章,评审

    doi:10.1038/nbt0602-575

    authors: Hare PD,Chua NH

    更新日期:2002-06-01 00:00:00

  • Mapping recently identified nucleotide variants in the genome and transcriptome.

    abstract::Nucleotide variants, especially those related to epigenetic functions, provide critical regulatory information beyond simple genomic sequence, and they define cell status in higher organisms. 5-Methylcytosine, which is found in DNA, was until recently the only nucleotide variant studied in terms of epigenetics in euka...

    journal_title:Nature biotechnology

    pub_type: 杂志文章,评审

    doi:10.1038/nbt.2398

    authors: Song CX,Yi C,He C

    更新日期:2012-11-01 00:00:00

  • Long-term two-photon fluorescence imaging of mammalian embryos without compromising viability.

    abstract::A major challenge for fluorescence imaging of living mammalian cells is maintaining viability following prolonged exposure to excitation illumination. We have monitored the dynamics of mitochondrial distribution in hamster embryos at frequent intervals over 24 h using two-photon microscopy (1,047 nm) while maintaining...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/11698

    authors: Squirrell JM,Wokosin DL,White JG,Bavister BD

    更新日期:1999-08-01 00:00:00

  • Production of human monoclonal antibody in eggs of chimeric chickens.

    abstract::The tubular gland of the chicken oviduct is an attractive system for protein expression as large quantities of proteins are deposited in the egg, the production of eggs is easily scalable and good manufacturing practices for therapeutics from eggs have been established. Here we examined the ability of upstream and dow...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt1132

    authors: Zhu L,van de Lavoir MC,Albanese J,Beenhouwer DO,Cardarelli PM,Cuison S,Deng DF,Deshpande S,Diamond JH,Green L,Halk EL,Heyer BS,Kay RM,Kerchner A,Leighton PA,Mather CM,Morrison SL,Nikolov ZL,Passmore DB,Pradas-Monne

    更新日期:2005-09-01 00:00:00

  • The search for the ideal biocatalyst.

    abstract::While the use of enzymes as biocatalysts to assist in the industrial manufacture of fine chemicals and pharmaceuticals has enormous potential, application is frequently limited by evolution-led catalyst traits. The advent of designer biocatalysts, produced by informed selection and mutation through recombinant DNA tec...

    journal_title:Nature biotechnology

    pub_type: 杂志文章,评审

    doi:10.1038/nbt0102-37

    authors: Burton SG,Cowan DA,Woodley JM

    更新日期:2002-01-01 00:00:00

  • Fingerprinting polysaccharides with single-molecule atomic force microscopy.

    abstract::We report the use of an atomic force microscopy (AFM)-based force spectroscopy technique to identify, at the single-molecule level, the components of mixtures of polysaccharides. Previously, we showed that the elasticity of certain types of polysaccharides is governed by force-induced conformational transitions of the...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/85712

    authors: Marszalek PE,Li H,Fernandez JM

    更新日期:2001-03-01 00:00:00

  • DNA patenting: the end of an era?

    abstract::Debates on patenting DNA must evolve to reflect the global decline in filings and regional disparities in patenting activity. ...

    journal_title:Nature biotechnology

    pub_type: 杂志文章,评审

    doi:10.1038/nbt0207-185

    authors: Hopkins MM,Mahdi S,Patel P,Thomas SM

    更新日期:2007-02-01 00:00:00

  • Targeted RNA sequencing reveals the deep complexity of the human transcriptome.

    abstract::Transcriptomic analyses have revealed an unexpected complexity to the human transcriptome, whose breadth and depth exceeds current RNA sequencing capability. Using tiling arrays to target and sequence select portions of the transcriptome, we identify and characterize unannotated transcripts whose rare or transient exp...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt.2024

    authors: Mercer TR,Gerhardt DJ,Dinger ME,Crawford J,Trapnell C,Jeddeloh JA,Mattick JS,Rinn JL

    更新日期:2011-11-13 00:00:00

  • Dynamic single-cell imaging of direct reprogramming reveals an early specifying event.

    abstract::The study of induced pluripotency often relies on experimental approaches that average measurements across a large population of cells, the majority of which do not become pluripotent. Here we used high-resolution, time-lapse imaging to trace the reprogramming process over 2 weeks from single mouse embryonic fibroblas...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt.1632

    authors: Smith ZD,Nachman I,Regev A,Meissner A

    更新日期:2010-05-01 00:00:00

  • Targeted isolation and cultivation of uncultivated bacteria by reverse genomics.

    abstract::Most microorganisms from all taxonomic levels are uncultured. Single-cell genomes and metagenomes continue to increase the known diversity of Bacteria and Archaea; however, while 'omics can be used to infer physiological or ecological roles for species in a community, most of these hypothetical roles remain unvalidate...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/s41587-019-0260-6

    authors: Cross KL,Campbell JH,Balachandran M,Campbell AG,Cooper SJ,Griffen A,Heaton M,Joshi S,Klingeman D,Leys E,Yang Z,Parks JM,Podar M

    更新日期:2019-11-01 00:00:00

  • Discovery of directional and nondirectional pioneer transcription factors by modeling DNase profile magnitude and shape.

    abstract::We describe protein interaction quantitation (PIQ), a computational method for modeling the magnitude and shape of genome-wide DNase I hypersensitivity profiles to identify transcription factor (TF) binding sites. Through the use of machine-learning techniques, PIQ identified binding sites for >700 TFs from one DNase ...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt.2798

    authors: Sherwood RI,Hashimoto T,O'Donnell CW,Lewis S,Barkal AA,van Hoff JP,Karun V,Jaakkola T,Gifford DK

    更新日期:2014-02-01 00:00:00

  • Resequencing 50 accessions of cultivated and wild rice yields markers for identifying agronomically important genes.

    abstract::Rice is a staple crop that has undergone substantial phenotypic and physiological changes during domestication. Here we resequenced the genomes of 40 cultivated accessions selected from the major groups of rice and 10 accessions of their wild progenitors (Oryza rufipogon and Oryza nivara) to >15 × raw data coverage. W...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/nbt.2050

    authors: Xu X,Liu X,Ge S,Jensen JD,Hu F,Li X,Dong Y,Gutenkunst RN,Fang L,Huang L,Li J,He W,Zhang G,Zheng X,Zhang F,Li Y,Yu C,Kristiansen K,Zhang X,Wang J,Wright M,McCouch S,Nielsen R,Wang J,Wang W

    更新日期:2011-12-11 00:00:00

  • Author Correction: Brain leaks and consumer neurotechnology.

    abstract::An amendment to this paper has been published and can be accessed via a link at the top of the paper. ...

    journal_title:Nature biotechnology

    pub_type: 已发布勘误

    doi:10.1038/s41587-019-0167-2

    authors: Ienca M,Haselager P,Emanuel EJ

    更新日期:2019-07-01 00:00:00

  • A humanized monoclonal antibody produced in transgenic plants for immunoprotection of the vagina against genital herpes.

    abstract::The ability to produce monoclonal antibodies (Mabs) in plants offers the opportunity for the development of an inexpensive method of mucosal immunoprotection against sexually transmitted diseases. To investigate the suitability of plant-expressed Mabs for vaginal preventive applications, we compared a humanized anti-h...

    journal_title:Nature biotechnology

    pub_type: 杂志文章

    doi:10.1038/4344

    authors: Zeitlin L,Olmsted SS,Moench TR,Co MS,Martinell BJ,Paradkar VM,Russell DR,Queen C,Cone RA,Whaley KJ

    更新日期:1998-12-01 00:00:00