Statistically significant meta-analyses of clinical trials have modest credibility and inflated effects.

Abstract:

OBJECTIVE:To assess whether nominally statistically significant effects in meta-analyses of clinical trials are true and whether their magnitude is inflated. STUDY DESIGN AND SETTING:Data from the Cochrane Database of Systematic Reviews 2005 (issue 4) and 2010 (issue 1) were used. We considered meta-analyses with binary outcomes and four or more trials in 2005 with P<0.05 for the random-effects odds ratio (OR). We examined whether any of these meta-analyses had updated counterparts in 2010. We estimated the credibility (true-positive probability) under different prior assumptions and inflation in OR estimates in 2005. RESULTS:Four hundred sixty-one meta-analyses in 2005 were eligible, and 80 had additional trials included by 2010. The effect sizes (ORs) were smaller in the updating data (2005-2010) than in the respective meta-analyses in 2005 (median 0.85-fold, interquartile range [IQR]: 0.66-1.06), even more prominently for meta-analyses with less than 300 events in 2005 (median 0.67-fold, IQR: 0.54-0.96). Mean credibility of the 461 meta-analyses in 2005 was 63-84% depending on the assumptions made. Credibility estimates changed >20% in 19-31 (24-39%) of the 80 updated meta-analyses. CONCLUSIONS:Most meta-analyses with nominally significant results pertain to truly nonnull effects, but exceptions are not uncommon. The magnitude of observed effects, especially in meta-analyses with limited evidence, is often inflated.

journal_name

J Clin Epidemiol

authors

Pereira TV,Ioannidis JP

doi

10.1016/j.jclinepi.2010.12.012

subject

Has Abstract

pub_date

2011-10-01 00:00:00

pages

1060-9

issue

10

eissn

0895-4356

issn

1878-5921

pii

S0895-4356(11)00009-6

journal_volume

64

pub_type

杂志文章,评审
  • A case-control evaluation of treatment efficacy: the example of magnesium sulfate prophylaxis against eclampsia in patients with preeclampsia.

    abstract::Randomized trials are the optimal approach for evaluations of treatment efficacy but may not always be feasible. We study the adequacy of the case-control design in evaluating efficacy in a situation where the investigated therapy, namely the administration of magnesium sulfate for the prevention of eclampsia in patie...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(96)00421-0

    authors: Abi-Said D,Annegers JF,Combs-Cantrell D,Suki R,Frankowski RF,Willmore LJ

    更新日期:1997-04-01 00:00:00

  • An observational study showed that explaining randomization using gambling-related metaphors and computer-agency descriptions impeded randomized clinical trial recruitment.

    abstract:OBJECTIVES:To explore how the concept of randomization is described by clinicians and understood by patients in randomized controlled trials (RCTs) and how it contributes to patient understanding and recruitment. STUDY DESIGN AND SETTING:Qualitative analysis of 73 audio recordings of recruitment consultations from fiv...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.jclinepi.2018.02.018

    authors: Jepson M,Elliott D,Conefrey C,Wade J,Rooshenas L,Wilson C,Beard D,Blazeby JM,Birtle A,Halliday A,Stein R,Donovan JL,CSAW study group.,Chemorad study group.,POUT study group.,ACST-2 study group.,OPTIMA prelim study group.

    更新日期:2018-07-01 00:00:00

  • Administrative database code accuracy did not vary notably with changes in disease prevalence.

    abstract:OBJECTIVES:Previous mathematical analyses of diagnostic tests based on the categorization of a continuous measure have found that test sensitivity and specificity varies significantly by disease prevalence. This study determined if the accuracy of diagnostic codes varied by disease prevalence. STUDY DESIGN AND SETTING...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2016.05.009

    authors: van Walraven C,English S,Austin PC

    更新日期:2016-11-01 00:00:00

  • Rasch analysis informed modifications to the Work Instability Scale for Rheumatoid Arthritis for use in work-related upper limb disorders.

    abstract:OBJECTIVE:The Work Instability Scale for Rheumatoid Arthritis (RA-WIS) is a promising prognostic tool for future work disability outcomes. Rasch analysis was conducted to examine the psychometric performance of the RA-WIS in work-related upper limb disorders. STUDY DESIGN AND SETTING:Eligible injured workers (n=396) a...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2011.02.002

    authors: Tang K,Beaton DE,Gignac MA,Bombardier C

    更新日期:2011-11-01 00:00:00

  • ROC area discrimination (ROCAD) curve: a new method of evaluating the discriminating ability of ordinal scales.

    abstract:OBJECTIVE:The area under the receiver operating characteristic (ROC) curve has been frequently used to assess the ability of diagnostic tests to discriminate between individuals with and without a disease. In this paper, we propose to use the ROC area to evaluate the discriminating power of ordinal measures, such as ma...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2007.11.016

    authors: Kopec JA,Sayre EC

    更新日期:2008-10-01 00:00:00

  • The Italian SF-36 Health Survey: translation, validation and norming.

    abstract::This article reports on the development and validation of the Italian SF-36 Health Survey using data from seven studies in which an Italian version of the SF-36 was administered to more than 7000 subjects between 1991 and 1995. Empirical findings from a wide array of studies and diseases indicate that the performance ...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(98)00094-8

    authors: Apolone G,Mosconi P

    更新日期:1998-11-01 00:00:00

  • The Marks Asthma Quality of Life Questionnaire: further validation and examination of responsiveness to change.

    abstract::We performed analyses to examine the structure, validity, and responsiveness to change of the Marks Asthma Quality of Life Questionnaire (AQLQ), originally validated in Australia in a self-administered format, among 539 U.S. subjects with asthma. Subjects were interviewed twice by telephone over an 18-month period. Ba...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(99)00026-8

    authors: Katz PP,Eisner MD,Henke J,Shiboski S,Yelin EH,Blanc PD

    更新日期:1999-07-01 00:00:00

  • Unconditional and conditional incentives differentially improved general practitioners' participation in an online survey: randomized controlled trial.

    abstract:OBJECTIVES:To compare the impact of unconditional and conditional financial incentives on response rates among Australian general practitioners invited by mail to participate in an online survey about cancer care and to investigate possible differential response bias between incentive groups. STUDY DESIGN AND SETTING:...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,随机对照试验

    doi:10.1016/j.jclinepi.2014.09.013

    authors: Young JM,O'Halloran A,McAulay C,Pirotta M,Forsdike K,Stacey I,Currow D

    更新日期:2015-06-01 00:00:00

  • Cochrane Qualitative and Implementation Methods Group guidance series-paper 6: reporting guidelines for qualitative, implementation, and process evaluation evidence syntheses.

    abstract:OBJECTIVES:To outline contemporary and novel developments for the presentation and reporting of syntheses of qualitative, implementation, and process evaluation evidence and provide recommendations for the use of reporting guidelines. STUDY DESIGN AND SETTING:An overview of reporting guidelines for qualitative, implem...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2017.10.022

    authors: Flemming K,Booth A,Hannes K,Cargo M,Noyes J

    更新日期:2018-05-01 00:00:00

  • Diagnostic E-codes for commonly used, narrow therapeutic index medications poorly predict adverse drug events.

    abstract:OBJECTIVE:We sought to examine the validity of specific hospital discharge codes in identifying drug toxicity precipitating hospitalization, among elderly users of high-risk medications. STUDY DESIGN AND SETTING:We conducted a cross-sectional evaluation assessing the diagnostic test characteristics of International Cl...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.jclinepi.2007.08.003

    authors: Leonard CE,Haynes K,Localio AR,Hennessy S,Tjia J,Cohen A,Kimmel SE,Feldman HI,Metlay JP

    更新日期:2008-06-01 00:00:00

  • A primary care Web-based Intervention Modeling Experiment replicated behavior changes seen in earlier paper-based experiment.

    abstract:OBJECTIVES:Intervention Modeling Experiments (IMEs) are a way of developing and testing behavior change interventions before a trial. We aimed to test this methodology in a Web-based IME that replicated the trial component of an earlier, paper-based IME. STUDY DESIGN AND SETTING:Three-arm, Web-based randomized evaluat...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,随机对照试验

    doi:10.1016/j.jclinepi.2016.07.008

    authors: Treweek S,Francis JJ,Bonetti D,Barnett K,Eccles MP,Hudson J,Jones C,Pitts NB,Ricketts IW,Sullivan F,Weal M,MacLennan G

    更新日期:2016-12-01 00:00:00

  • P. C. A. Louis and the birth of clinical epidemiology.

    abstract::Pierre Charles Alexandre Louis (1787-1872) has been the direct or indirect mentor of influential U.S. and English scientists in public health, epidemiology, medicine, and biostatistics during the 19th and 20th century. Louis was primarily a clinician, but his name has been more closely associated with the history of e...

    journal_title:Journal of clinical epidemiology

    pub_type: 传,历史文章,杂志文章

    doi:10.1016/s0895-4356(96)00294-6

    authors: Morabia A

    更新日期:1996-12-01 00:00:00

  • Factors associated with errors in death certificate completion. A national study in Taiwan.

    abstract::To identify characteristics of certifying physicians and the deceased that are associated with errors in death certificate completion in Taiwan, we retrospectively reviewed 4123 systematically sampled death certificates issued in 1994. Multivariate analyses were used to assess the associations of various characteristi...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(00)00299-7

    authors: Lu TH,Shau WY,Shih TP,Lee MC,Chou MC,Lin CK

    更新日期:2001-03-01 00:00:00

  • The UK version of the Seattle Angina Questionnaire (SAQ-UK): reliability, validity and responsiveness.

    abstract::The study assesses the reliability, validity and responsiveness of the UK version of the Seattle Angina Questionnaire (SAQ-UK). The instrument was anglicised and administered by self-completed postal questionnaire to 959 patients recruited from general practices in the North East of England. A total of 655 (68.3%) pat...

    journal_title:Journal of clinical epidemiology

    pub_type: 临床试验,杂志文章,随机对照试验

    doi:10.1016/s0895-4356(01)00352-3

    authors: Garratt AM,Hutchinson A,Russell I,Network for Evidence-Based Practice in Northern and Yorkshire (NEBPINY).

    更新日期:2001-09-01 00:00:00

  • Single pivotal trials with few corroborating characteristics were used for FDA approval of cancer therapies.

    abstract:BACKGROUND AND OBJECTIVE:Novel cancer therapies are often approved with evidence from a single pivotal trial alone. There are concerns about the credibility of this evidence. Higher validity may be indicated by five methodological and statistical characteristics of pivotal trial evidence that were described by the U.S....

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2019.05.033

    authors: Ladanie A,Speich B,Briel M,Sclafani F,Bucher HC,Agarwal A,Ioannidis JPA,Pereira TV,Kasenda B,Hemkens LG

    更新日期:2019-10-01 00:00:00

  • Quality varies across clinical practice guidelines for mammography screening in women aged 40-49 years as assessed by AGREE and AMSTAR instruments.

    abstract:OBJECTIVE:To assess the quality of clinical practice guidelines providing recommendations on the frequency of mammography screening in asymptomatic, average-risk women 40-49 years of age. STUDY DESIGN AND SETTING:We searched the National Guideline Clearinghouse and MEDLINE for guidelines published from 2005 to 2010. F...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,评审

    doi:10.1016/j.jclinepi.2010.12.005

    authors: Burda BU,Norris SL,Holmer HK,Ogden LA,Smith ME

    更新日期:2011-09-01 00:00:00

  • Diagnostic interval and mortality in colorectal cancer: U-shaped association demonstrated for three different datasets.

    abstract:OBJECTIVE:To test the theory of a U-shaped association between time from the first presentation of symptoms in primary care to the diagnosis (the diagnostic interval) and mortality after diagnosis of colorectal cancer (CRC). STUDY DESIGN AND SETTING:Three population-based studies in Denmark and the United Kingdom usin...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2011.12.006

    authors: Tørring ML,Frydenberg M,Hamilton W,Hansen RP,Lautrup MD,Vedsted P

    更新日期:2012-06-01 00:00:00

  • A nested case-control study of influenza vaccination was a cost-effective alternative to a full cohort analysis.

    abstract:OBJECTIVE:In the absence of trial results that are applicable to the target population, nested case-control studies might be an alternative to full-cohort analysis. We compared relative and absolute estimates of associations in an influenza vaccine study using both designs. STUDY DESIGN AND SETTING:Data from elderly p...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2004.01.019

    authors: Hak E,Wei F,Grobbee DE,Nichol KL

    更新日期:2004-09-01 00:00:00

  • Using postal randomization to replace telephone randomization had no significant effect on recruitment of patients.

    abstract:OBJECTIVE:To test the effect of postal randomization on recruitment of patients into a randomized trial in primary care. STUDY DESIGN AND SETTING:General practices used a telephone service to randomize patients in our trial. Delays in the start of recruitment at some sites led us to modify the randomization procedure....

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.jclinepi.2007.04.003

    authors: Brealey SD,Atwell C,Bryan S,Coulton S,Cox H,Cross B,Fylan F,Garratt A,Gilbert FJ,Gillan MG,Hendry M,Hood K,Houston H,King D,Morton V,Orchard J,Robling M,Russell IT,Torgerson D,Wadsworth V,Wilkinson C

    更新日期:2007-10-01 00:00:00

  • Most systematic reviews of adverse effects did not include unpublished data.

    abstract:OBJECTIVES:We sought to identify the proportion of systematic reviews of adverse effects which search for unpublished data and the success rates of identifying unpublished data for inclusion in a systematic review. STUDY DESIGN AND SETTING:Two reviewers independently screened all records published in 2014 in the Datab...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2016.05.003

    authors: Golder S,Loke YK,Wright K,Sterrantino C

    更新日期:2016-09-01 00:00:00

  • Public and private hypotheses.

    abstract::The uses of hypothesis in scientific medicine and in clinical medicine are broadly similar; but they differ in subtle but important logical details. The key distinction is that scientific medicine deals with broad ("public") hypotheses about entire populations; and the predominant problem of inference is statistics in...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(89)90028-0

    authors: Murphy EA

    更新日期:1989-01-01 00:00:00

  • Diagnostic certainty and potential for misclassification in exocrine pancreatic cancer. PANKRAS I Project Investigations.

    abstract::Whereas over the last decade epidemiologic studies on exocrine pancreatic cancer (EPC) continued to show a remarkable heterogeneity in diagnostic criteria applied to define caseness, the actual magnitude and consequences of misclassification remain largely unexplored. The objectives were: (1) to estimate the degree of...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(94)90123-6

    authors: Porta M,Malats N,Piñol JL,Rifà J,Andreu M,Real FX

    更新日期:1994-09-01 00:00:00

  • Methods to evaluate risks for composite end points and their individual components.

    abstract:OBJECTIVE:Both randomized and observational studies commonly examine composite end points, but the literature on model development and criticism in this setting is limited. STUDY DESIGN AND SETTING:We examined approaches for evaluating heterogeneity in the effects of risk factors for different components of the end po...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2003.02.001

    authors: Glynn RJ,Rosner B

    更新日期:2004-02-01 00:00:00

  • Validation of a combined comorbidity index.

    abstract::The basic objective of this paper is to evaluate an age-comorbidity index in a cohort of patients who were originally enrolled in a prospective study to identify risk factors for peri-operative complications. Two-hundred and twenty-six patients were enrolled in the study. The participants were patients with hypertensi...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(94)90129-5

    authors: Charlson M,Szatrowski TP,Peterson J,Gold J

    更新日期:1994-11-01 00:00:00

  • A study of the noninstrumented physical examination of the knee found high observer variability.

    abstract:OBJECTIVE:This study estimated the inter- and intraobserver reliability of a set of noninstrumented physical examination measures for knee pain in older adults. STUDY DESIGN AND SETTING:Forty-five patients from primary care, and 13 patients from secondary care, were each examined by two out of a team of three physical...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2005.11.004

    authors: Wood L,Peat G,Wilkie R,Hay E,Thomas E,Sim J

    更新日期:2006-05-01 00:00:00

  • A cohort study found that white blood cell count and endocrine markers predicted preterm birth in symptomatic women.

    abstract:OBJECTIVE:This cohort study investigated potential clinical and biochemical predictors of subsequent preterm birth in women presenting with threatened preterm labor. STUDY DESIGN AND SETTING:Subjects were 218 pregnant women admitted to hospital with a diagnosis of threatened preterm labor at 22-36 weeks gestation. Exc...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2004.06.015

    authors: Campbell MK,Challis JR,DaSilva O,Bocking AD

    更新日期:2005-03-01 00:00:00

  • Mapping the expanded often inappropriate use of the Framingham Risk Score in the medical literature.

    abstract:OBJECTIVES:To systematically evaluate the use of Framingham Risk Score (FRS) in the medical literature and specifically examine the use of FRS in different populations and settings and for different outcomes than the ones originally developed for. STUDY DESIGN AND SETTING:We identified all the citations to the article...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2013.10.021

    authors: Tzoulaki I,Seretis A,Ntzani EE,Ioannidis JP

    更新日期:2014-05-01 00:00:00

  • A review of two journals found that articles using multivariable logistic regression frequently did not report commonly recommended assumptions.

    abstract:BACKGROUND AND OBJECTIVE:To examine if commonly recommended assumptions for multivariable logistic regression are addressed in two major epidemiological journals. METHODS:Ninety-nine articles from the Journal of Clinical Epidemiology and the American Journal of Epidemiology were surveyed for 10 criteria: six dealing w...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,评审

    doi:10.1016/j.jclinepi.2003.05.003

    authors: Ottenbacher KJ,Ottenbacher HR,Tooth L,Ostir GV

    更新日期:2004-11-01 00:00:00

  • Non-Cochrane vs. Cochrane reviews were twice as likely to have positive conclusion statements: cross-sectional study.

    abstract:OBJECTIVES:To determine which factors predict favorable results and positive conclusions in systematic reviews (SRs) and to assess the level of agreement between SR results and conclusions. STUDY DESIGN AND SETTING:A sample of 296 English SRs indexed in MEDLINE (November, 2004) was obtained. Two investigators independ...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2008.08.008

    authors: Tricco AC,Tetzlaff J,Pham B,Brehaut J,Moher D

    更新日期:2009-04-01 00:00:00

  • The International Continence Society (ICS) incontinence definition: is the social and hygienic aspect appropriate for etiologic research?

    abstract:OBJECTIVE:To investigate the effect of applying a problem assessment versus a pure symptom urinary incontinence (UI) caseness definition in etiologic research. SUBJECTS:A random population sample of 2613 women aged 30-59 years, who responded to a postal questionnaire. MAIN PARAMETERS: One-year period prevalence of the...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(97)00130-3

    authors: Foldspang A,Mommsen S

    更新日期:1997-09-01 00:00:00