Measuring the effectiveness of scientific gatekeeping.

Abstract:

:Peer review is the main institution responsible for the evaluation and gestation of scientific research. Although peer review is widely seen as vital to scientific evaluation, anecdotal evidence abounds of gatekeeping mistakes in leading journals, such as rejecting seminal contributions or accepting mediocre submissions. Systematic evidence regarding the effectiveness--or lack thereof--of scientific gatekeeping is scant, largely because access to rejected manuscripts from journals is rarely available. Using a dataset of 1,008 manuscripts submitted to three elite medical journals, we show differences in citation outcomes for articles that received different appraisals from editors and peer reviewers. Among rejected articles, desk-rejected manuscripts, deemed as unworthy of peer review by editors, received fewer citations than those sent for peer review. Among both rejected and accepted articles, manuscripts with lower scores from peer reviewers received relatively fewer citations when they were eventually published. However, hindsight reveals numerous questionable gatekeeping decisions. Of the 808 eventually published articles in our dataset, our three focal journals rejected many highly cited manuscripts, including the 14 most popular; roughly the top 2 percent. Of those 14 articles, 12 were desk-rejected. This finding raises concerns regarding whether peer review is ill--suited to recognize and gestate the most impactful ideas and research. Despite this finding, results show that in our case studies, on the whole, there was value added in peer review. Editors and peer reviewers generally--but not always-made good decisions regarding the identification and promotion of quality in scientific manuscripts.

authors

Siler K,Lee K,Bero L

doi

10.1073/pnas.1418218112

subject

Has Abstract

pub_date

2015-01-13 00:00:00

pages

360-5

issue

2

eissn

0027-8424

issn

1091-6490

pii

1418218112

journal_volume

112

pub_type

杂志文章
  • K63 ubiquitylation triggers proteasomal degradation by seeding branched ubiquitin chains.

    abstract::Different polyubiquitin chain linkages direct substrates toward distinct cellular pathways. K63-linked ubiquitylation is known to regulate proteasome-independent events such as signal transduction, but its function in the context of heterogeneous ubiquitin chains remains unclear. Here, we report that K63 ubiquitylatio...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.1716673115

    authors: Ohtake F,Tsuchiya H,Saeki Y,Tanaka K

    更新日期:2018-02-13 00:00:00

  • Computing trisections of 4-manifolds.

    abstract::We describe an algorithm to compute trisections of orientable four-manifolds using arbitrary triangulations as input. This results in explicit complexity bounds for the trisection genus of a 4-manifold in terms of the number of pentachora (4-simplices) in a triangulation. ...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.1717173115

    authors: Bell M,Hass J,Rubinstein JH,Tillmann S

    更新日期:2018-10-23 00:00:00

  • MicroRNA-binding viral protein interferes with Arabidopsis development.

    abstract::MicroRNAs (miRNAs) are small (approximately 21 nt), noncoding RNAs that negatively regulate target mRNAs at the posttranscriptional level that are involved in development. In plants, virus-induced disease symptoms often result in developmental abnormalities resembling perturbation of miRNA-mediated function. Here, we ...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.0504439102

    authors: Chellappan P,Vanitharani R,Fauquet CM

    更新日期:2005-07-19 00:00:00

  • Cytoplasmic isoforms of Kaposi sarcoma herpesvirus LANA recruit and antagonize the innate immune DNA sensor cGAS.

    abstract::The latency-associated nuclear antigen (LANA) of Kaposi sarcoma herpesvirus (KSHV) is mainly localized and functions in the nucleus of latently infected cells, playing a pivotal role in the replication and maintenance of latent viral episomal DNA. In addition, N-terminally truncated cytoplasmic isoforms of LANA, resul...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.1516812113

    authors: Zhang G,Chan B,Samarina N,Abere B,Weidner-Glunde M,Buch A,Pich A,Brinkmann MM,Schulz TF

    更新日期:2016-02-23 00:00:00

  • Uncoupling of growth plate maturation and bone formation in mice lacking both Schnurri-2 and Schnurri-3.

    abstract::Formation and remodeling of the skeleton relies on precise temporal and spatial regulation of genes expressed in cartilage and bone cells. Debilitating diseases of the skeletal system occur when mutations arise that disrupt these intricate genetic regulatory programs. Here, we report that mice bearing parallel null mu...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.1003727107

    authors: Jones DC,Schweitzer MN,Wein M,Sigrist K,Takagi T,Ishii S,Glimcher LH

    更新日期:2010-05-04 00:00:00

  • Lymphospecific toxicity in adenosine deaminase deficiency and purine nucleoside phosphorylase deficiency: possible role of nucleoside kinase(s).

    abstract::Inherited deficiencies of the enzymes adenosine deaminase (adenosine aminohydrolase; EC 3.5.4.4) and purine nucleoside phosphorylase (purine-nucleoside:orthophosphate ribosyltransferase; EC 2.4.2.1) preferentially interfere with lymphocyte development while sparing most other organ systems. Previous experiments have s...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.74.12.5677

    authors: Carson DA,Kaye J,Seegmiller JE

    更新日期:1977-12-01 00:00:00

  • Src kinase associates with a member of a distinct subfamily of protein-tyrosine phosphatases containing an ezrin-like domain.

    abstract::A 6.2-kb full-length clone encoding a distinct protein-tyrosine phosphatase (PTP; EC 3.1.3.48), PTPD1, was isolated from a human skeletal muscle cDNA library. The cDNA encodes a protein of 1174 amino acids with N-terminal sequence homology to the ezrin-band 4.1-merlin-radixin protein family, which also includes the tw...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.91.16.7477

    authors: Møller NP,Møller KB,Lammers R,Kharitonenkov A,Sures I,Ullrich A

    更新日期:1994-08-02 00:00:00

  • Higher-Order Curvature and Local Solvability of D(theta).

    abstract::Let E and F be vector bundles and D:E --> F an operator of order k. We associate a sequence of invariants ((l))(D), l >/= 0 with D which generalize the concept of curvature in a natural way. In the case where D = D(theta) is the differential operator of a connection theta on a vector bundle E, ((1))(D(theta)) is the c...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.70.3.797

    authors: Knapp RE

    更新日期:1973-03-01 00:00:00

  • Suppression of breast cancer growth and metastasis by a serpin myoepithelium-derived serine proteinase inhibitor expressed in the mammary myoepithelial cells.

    abstract::A serpin was identified in normal mammary gland by differential cDNA sequencing. In situ hybridization has detected this serpin exclusively in the myoepithelial cells on the normal and noninvasive mammary epithelial side of the basement membrane and thus was named myoepithelium-derived serine proteinase inhibitor (MEP...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.96.7.3700

    authors: Xiao G,Liu YE,Gentz R,Sang QA,Ni J,Goldberg ID,Shi YE

    更新日期:1999-03-30 00:00:00

  • Colony-stimulating factor (CSF) radioimmunoassay: detection of a CSF subclass stimulating macrophage production.

    abstract::Colony-stimulating factors (CSFs) stimulate the differentiation of immature precursor cells to mature granulocytes and macrophages. Purified 125I-labeled murine L cell CSF has been used to develop a radioimmunoassay (RIA) that detects a subclass of CSFs that stimulates macrophage production. Murine CSF preparations th...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.76.6.2969

    authors: Stanley ER

    更新日期:1979-06-01 00:00:00

  • Molecular and biological changes in the cold-adapted "master strain" A/AA/6/60 (H2N2) influenza virus.

    abstract::The live cold-adapted (ca) A/AA/6/60 influenza vaccine is being commercially developed for worldwide use in children and is being used as a model for other live vaccines. Although it has been proven safe and immunogenic, the molecular basis of cold adaptation has never been determined. To identify sequence changes res...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.90.13.6032

    authors: Herlocher ML,Maassab HF,Webster RG

    更新日期:1993-07-01 00:00:00

  • Identification of the endothelial cell binding site for factor IX.

    abstract::We previously demonstrated that the primary region of factor IX and IXa responsible for saturable specific binding to bovine aortic endothelial cells resides in residues 3-11 at the amino terminus of factor IX. We also demonstrated that mutations of lysine to alanine at residue 5, factor IX K5A, or valine to lysine at...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.93.20.11068

    authors: Cheung WF,van den Born J,Kühn K,Kjellén L,Hudson BG,Stafford DW

    更新日期:1996-10-01 00:00:00

  • The animal model of human amnesia: long-term memory impaired and short-term memory intact.

    abstract::Normal monkeys and monkeys with lesions of the hippocampal formation and adjacent cortex (the H+ lesion) were trained on the delayed nonmatching to sample (DNMS) task with a delay of 0.5 s between the sample and the choice. The animals with H+ lesions learned the task normally at this short delay and also exhibited th...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.91.12.5637

    authors: Alvarez P,Zola-Morgan S,Squire LR

    更新日期:1994-06-07 00:00:00

  • Emergence of self-organized multivortex states in flocks of active rollers.

    abstract::Active matter, both synthetic and biological, demonstrates complex spatiotemporal self-organization and the emergence of collective behavior. A coherent rotational motion, the vortex phase, is of great interest because of its ability to orchestrate well-organized motion of self-propelled particles over large distances...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.2000061117

    authors: Han K,Kokot G,Tovkach O,Glatz A,Aranson IS,Snezhko A

    更新日期:2020-05-05 00:00:00

  • Functional profiling of neurons through cellular neuropharmacology.

    abstract::We describe a functional profiling strategy to identify and characterize subtypes of neurons present in a peripheral ganglion, which should be extendable to neurons in the CNS. In this study, dissociated dorsal-root ganglion neurons from mice were exposed to various pharmacological agents (challenge compounds), while ...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.1118833109

    authors: Teichert RW,Smith NJ,Raghuraman S,Yoshikami D,Light AR,Olivera BM

    更新日期:2012-01-31 00:00:00

  • Nitric oxide mediates N-methyl-D-aspartate receptor-induced activation of p21ras.

    abstract::N-methyl-D-aspartate (NMDA) glutamate receptor-mediated increases in intracellular calcium are thought to play a critical role in synaptic plasticity. The mechanisms by which changes in cytoplasmic calcium transmit the glutamate signal to the nucleus, which is ultimately important for long-lasting neuronal responses, ...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.95.10.5773

    authors: Yun HY,Gonzalez-Zulueta M,Dawson VL,Dawson TM

    更新日期:1998-05-12 00:00:00

  • A sensitized genetic system for the analysis of murine B lymphocyte signal transduction pathways dependent on Bruton's tyrosine kinase.

    abstract::Modifier screens have been powerful genetic tools to define signaling pathways in lower organisms. The identification of modifier loci in mice has begun to allow a similar dissection of mammalian signaling pathways. Transgenic mice (Btk(lo)) expressing 25% of endogenous levels of Bruton's tyrosine kinase (Btk) have B ...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.110146697

    authors: Satterthwaite AB,Willis F,Kanchanastit P,Fruman D,Cantley LC,Helgason CD,Humphries RK,Lowell CA,Simon M,Leitges M,Tarakhovsky A,Tedder TF,Lesche R,Wu H,Witte ON

    更新日期:2000-06-06 00:00:00

  • Atomic structure of a nitrate-binding protein crucial for photosynthetic productivity.

    abstract::Cyanobacteria, blue-green algae, are the most abundant autotrophs in aquatic environments and form the base of all aquatic food chains by fixing carbon and nitrogen into cellular biomass. The single most important nutrient for photosynthesis and growth is nitrate, which is severely limiting in many aquatic environment...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.0602517103

    authors: Koropatkin NM,Pakrasi HB,Smith TJ

    更新日期:2006-06-27 00:00:00

  • A few selected applications of surface nonlinear optical spectroscopy.

    abstract::As a second-order nonlinear optical process, sum-frequency generation is highly surface-specific and accordingly has been developed into a very powerful and versatile surface spectroscopic tool. It has found many unique applications in different disciplines and thus provided many exciting new research opportunities in...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章,评审

    doi:10.1073/pnas.93.22.12104

    authors: Shen YR

    更新日期:1996-10-29 00:00:00

  • Tyrosine administration increases striatal dopamine release in rats with partial nigrostriatal lesions.

    abstract::Partial, unilateral nigrostriatal lesions of varying severity were produced in rats by injecting graded doses of 6-hydroxydopamine into the substantia nigra. Formation of the dopamine metabolites dihydroxyphenylacetic acid and homovanillic acid in each surviving nigrostriatal neuron (estimated by the ratios of dihydro...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.77.7.4305

    authors: Melamed E,Hefti F,Wurtman RJ

    更新日期:1980-07-01 00:00:00

  • Engineered bio-inspired coating for passive flow control.

    abstract::Flow separation and vortex shedding are some of the most common phenomena experienced by bluff bodies under relative motion with the surrounding medium. They often result in a recirculation bubble in regions with adverse pressure gradient, which typically reduces efficiency in vehicles and increases loading on structu...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.1715567115

    authors: Bocanegra Evans H,Hamed AM,Gorumlu S,Doosttalab A,Aksak B,Chamorro LP,Castillo L

    更新日期:2018-02-06 00:00:00

  • Quantifying changes in the thiol redox proteome upon oxidative stress in vivo.

    abstract::Antimicrobial levels of reactive oxygen species (ROS) are produced by the mammalian host defense to kill invading bacteria and limit bacterial colonization. One main in vivo target of ROS is the thiol group of proteins. We have developed a quantitative thiol trapping technique termed OxICAT to identify physiologically...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.0707723105

    authors: Leichert LI,Gehrke F,Gudiseva HV,Blackwell T,Ilbert M,Walker AK,Strahler JR,Andrews PC,Jakob U

    更新日期:2008-06-17 00:00:00

  • Pyochelin: novel structure of an iron-chelating growth promoter for Pseudomonas aeruginosa.

    abstract::Pyochelin, an endogenous growth promoter that solubilizes ferric iron, has been isolated from Pseudomonas aeruginosa, including clinical strains. The structure of pyochelin has been assigned as 2-(2-o-hydroxyphenyl-2-thiazolin-4-yl)-3-methylthiazolidine-4-carboxylic acid and is of a different type from those previousl...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.78.7.4256

    authors: Cox CD,Rinehart KL Jr,Moore ML,Cook JC Jr

    更新日期:1981-07-01 00:00:00

  • Synthesis of bacteriophage lambda DNA in vitro: requirement for O and P gene products.

    abstract::We have developed a cell-free system to study bacteriophage lambda DNA replication. Maximal DNA synthesis in vitro requires the four deoxynucleoside triphosphates, ATP, and exogenous lambda DNA. DNA synthesis requires the products of the phage O and P genes but is not inhibited by lambda repressor. The kinetics of syn...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.71.5.1758

    authors: Shizuya H,Richardson CC

    更新日期:1974-05-01 00:00:00

  • Fialuridine and its metabolites inhibit DNA polymerase gamma at sites of multiple adjacent analog incorporation, decrease mtDNA abundance, and cause mitochondrial structural defects in cultured hepatoblasts.

    abstract::The thymidine analog fialuridine deoxy-2-fluoro-beta-D-arabinofuranosyl)-5-iodouracil (FIAU) was toxic in trials for chronic hepatitis B infection. One mechanism postulated that defective mtDNA replication was mediated through inhibition of DNA polymerase-gamma (DNA pol-gamma), by FIAU triphosphate (FIALTP) or by trip...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.93.8.3592

    authors: Lewis W,Levine ES,Griniuviene B,Tankersley KO,Colacino JM,Sommadossi JP,Watanabe KA,Perrino FW

    更新日期:1996-04-16 00:00:00

  • Reduction of ischemia and reperfusion-induced myocardial damage by cytochrome P450 inhibitors.

    abstract::Ischemia and reperfusion both contribute to tissue damage after myocardial infarction. Although many drugs have been shown to reduce infarct size when administered before ischemia, few have been shown to be effective when administered at reperfusion. Moreover, although it is generally accepted that a burst of reactive...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.0308185100

    authors: Granville DJ,Tashakkor B,Takeuchi C,Gustafsson AB,Huang C,Sayen MR,Wentworth P Jr,Yeager M,Gottlieb RA

    更新日期:2004-02-03 00:00:00

  • Endogenous sources of variation in language acquisition.

    abstract::A fundamental question in the study of human language acquisition centers around apportioning explanatory force between the experience of the learner and the core knowledge that allows learners to represent that experience. We provide a previously unidentified kind of data identifying children's contribution to langua...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.1517094113

    authors: Han CH,Musolino J,Lidz J

    更新日期:2016-01-26 00:00:00

  • Structural studies of the scrapie prion protein by electron crystallography.

    abstract::Because the insolubility of the scrapie prion protein (PrP(Sc)) has frustrated structural studies by x-ray crystallography or NMR spectroscopy, we used electron crystallography to characterize the structure of two infectious variants of the prion protein. Isomorphous two-dimensional crystals of the N-terminally trunca...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.052703499

    authors: Wille H,Michelitsch MD,Guenebaut V,Supattapone S,Serban A,Cohen FE,Agard DA,Prusiner SB

    更新日期:2002-03-19 00:00:00

  • Dual regulation of NR2B and NR2C expression by NMDA receptor activation in mouse cerebellar granule cell cultures.

    abstract::In the developing cerebellum, switching of the subunit composition of NMDA receptors occurs in granule cells from NR2B-containing receptors to NR2C-containing ones. We investigated the mechanisms underlying switching of NR2B and NR2C subunit composition in primary cultures of mouse granule cells at the physiological K...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.0805574105

    authors: Iijima K,Abe H,Okazawa M,Moriyoshi K,Nakanishi S

    更新日期:2008-08-19 00:00:00

  • Depletion layer dynamics of polyelectrolyte solutions under Poiseuille flow.

    abstract::Complex liquids flow through channels faster than expected, an effect attributed to the formation of low-viscosity depletion layers at the boundaries. Characterization of depletion layer length scale, concentration, and dynamics has remained elusive due in large part to the lack of suitable real-space experimental tec...

    journal_title:Proceedings of the National Academy of Sciences of the United States of America

    pub_type: 杂志文章

    doi:10.1073/pnas.1900623116

    authors: Park SJ,Shakya A,King JT

    更新日期:2019-08-13 00:00:00