The statistical significance of meta-analyses is frequently fragile: definition of a fragility index for meta-analyses.

Abstract:

OBJECTIVES:Meta-analyses inform clinical practice by summarizing treatment effect estimates based on results from several trials. However, the statistical significance of a meta-analysis (i.e., whether the pooled treatment effect is statistically significant or not) may rely on the outcome of only a few patients from specific trials in the meta-analysis. We aimed to evaluate the extent to which the statistical significance of meta-analyses can be changed (from statistically significant to nonsignificant, or vice versa) after modifying the event status of patients in specific arms of specific trials. METHODS:We conducted a cross-sectional analysis of meta-analyses of trials with a binary outcome from Cochrane Systematic Reviews. We defined the fragility index of meta-analyses as the minimum number of patients from one or more trials included in the meta-analysis for whom an event-status modification (i.e., changing an event to nonevent or a nonevent to event) would change the statistical significance of the pooled treatment effect. For statistically significant and nonsignificant meta-analyses, we evaluated the fragility index, the ratio between the fragility index and the total number of participants included in the trials, and the ratio between the fragility index and the total number of events. RESULTS:Our sample comprised 906 meta-analyses: 400 and 506 had statistically significant and nonsignificant pooled treatment effects, respectively. For statistically significant meta-analyses, the median fragility index was 12 (Q1-Q3: 4-33); for 29% the fragility index was 5 or less. Overall, 43% and 9% meta-analyses would have become nonsignificant if the event status was modified for less than 1% of the total participants in one or several specific trials, and for less than 1% of the total number of events, respectively. These proportions were similar for statistically nonsignificant meta-analyses. Overall, the statistical significance of 33% of all meta-analyses depended on the event status of five or fewer participants from one or more specific trials. CONCLUSION:The statistical significance of meta-analyses often depends on the outcome of a few patients. The fragility index of meta-analyses may help in interpreting the conclusions of meta-analyses.

journal_name

J Clin Epidemiol

authors

Atal I,Porcher R,Boutron I,Ravaud P

doi

10.1016/j.jclinepi.2019.03.012

subject

Has Abstract

pub_date

2019-07-01 00:00:00

pages

32-40

eissn

0895-4356

issn

1878-5921

pii

S0895-4356(18)30818-7

journal_volume

111

pub_type

杂志文章
  • Bias in estimates of confidence intervals for health outcome report cards.

    abstract::Health researchers commonly use logistic regression when profiling health providers. Data from the patients treated by the providers are used to construct models predicting the expected number of outcomes for providers and the ratio of observed to expected outcomes (O/E ratio) used as a risk-adjusted measure of provid...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(03)00048-9

    authors: Faris PD,Ghali WA,Brant R

    更新日期:2003-06-01 00:00:00

  • Larger effect sizes in nonrandomized studies are associated with higher rates of EMA licensing approval.

    abstract:OBJECTIVES:The aim of this study was to evaluate how often the European Medicines Agency (EMA) has authorized drugs based on nonrandomized studies and whether there is an association between treatment effects and EMA preference for further testing in randomized clinical trials (RCTs). STUDY DESIGN AND SETTING:We revie...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,评审

    doi:10.1016/j.jclinepi.2018.01.011

    authors: Djulbegovic B,Glasziou P,Klocksieben FA,Reljic T,VanDenBergh M,Mhaskar R,Ioannidis JPA,Chalmers I

    更新日期:2018-06-01 00:00:00

  • An e-mail survey identified unpublished studies for systematic reviews.

    abstract:BACKGROUND AND OBJECTIVE:A large number of trials remain difficult to locate or unpublished for systematic reviews. The objective of this article was to determine the usefulness of making e-mail contact with authors of clinical trials and literature reviews found in MEDLINE to identify unpublished or difficult to locat...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2005.11.022

    authors: Reveiz L,Cardona AF,Ospina EG,de Agular S

    更新日期:2006-07-01 00:00:00

  • Reducing population stratification bias: stratum matching is better than exposure.

    abstract:OBJECTIVE:Genetic studies of complex human diseases rely heavily on the epidemiologic association paradigm, particularly the population-based case-control designs. This study aims to compare the matching effectiveness in terms of bias reduction between exposure matching and stratum matching. STUDY DESIGN AND SETTING:F...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2008.02.016

    authors: Lee WC,Wang LY

    更新日期:2009-01-01 00:00:00

  • Development and validation of a multivariate predictor of mortality in very low birth weight.

    abstract::Accurate prognosis is critical to the design of all prospective research aimed at improving survival. Predictions based on birth weight, gestational age, or any other single variable, fail to take into account the potentially important contribution of other factors. In order to develop a practical and accurate multiva...

    journal_title:Journal of clinical epidemiology

    pub_type: 临床试验,杂志文章

    doi:10.1016/0895-4356(88)90079-0

    authors: Ales KL,Frayer W,Hawks G,Auld PM,Druzin ML

    更新日期:1988-01-01 00:00:00

  • Correspondence analysis is a useful tool to uncover the relationships among categorical variables.

    abstract:OBJECTIVE:Correspondence analysis (CA) is a multivariate graphical technique designed to explore the relationships among categorical variables. Epidemiologists frequently collect data on multiple categorical variables with the goal of examining associations among these variables. Nevertheless, CA appears to be an under...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2009.08.008

    authors: Sourial N,Wolfson C,Zhu B,Quail J,Fletcher J,Karunananthan S,Bandeen-Roche K,Béland F,Bergman H

    更新日期:2010-06-01 00:00:00

  • A method is presented to plan the required sample size when estimating regression-based reference limits.

    abstract:BACKGROUND AND OBJECTIVE:Reference limits are widely used in anthropometry, the behavioral sciences, medicine, and clinical chemistry. They describe the distribution of a quantitative variable in a healthy population, and are often a smooth function of age or another determinant. Thus, instead of estimating reference l...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2006.09.004

    authors: Bellera CA,Hanley JA

    更新日期:2007-06-01 00:00:00

  • Statistically significant papers in psychiatry were cited more often than others.

    abstract:OBJECTIVE:Citations by other researchers are important in the dissemination of research findings. We aimed to investigate whether preferential citation of statistically significant articles exists in the psychiatric literature. STUDY DESIGN AND SETTINGS:We analyzed all original research papers published in 1996 in fou...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2006.11.014

    authors: Nieminen P,Rucker G,Miettunen J,Carpenter J,Schumacher M

    更新日期:2007-09-01 00:00:00

  • Misclassification of exposure is high when interview data on drug use are used as a proxy measure of chronic drug use during follow-up.

    abstract:BACKGROUND AND OBJECTIVE:In many observational studies, the association between drugs and disease is analyzed with information from a baseline interview. We investigated the magnitude and direction of exposure misclassification by comparing interview data at baseline with prospectively gathered pharmacy data. METHODS:...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2003.12.017

    authors: Beiderbeck AB,Sturkenboom MC,Coebergh JW,Leufkens HG,Stricker BH

    更新日期:2004-09-01 00:00:00

  • Cochrane Qualitative and Implementation Methods Group guidance series-paper 3: methods for assessing methodological limitations, data extraction and synthesis, and confidence in synthesized qualitative findings.

    abstract::The Cochrane Qualitative and Implementation Methods Group develops and publishes guidance on the synthesis of qualitative and mixed-method implementation evidence. Choice of appropriate methodologies, methods, and tools is essential when developing a rigorous protocol and conducting the synthesis. Cochrane authors who...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2017.06.020

    authors: Noyes J,Booth A,Flemming K,Garside R,Harden A,Lewin S,Pantoja T,Hannes K,Cargo M,Thomas J

    更新日期:2018-05-01 00:00:00

  • Responsiveness was similar between direct and mapped SF-6D in colorectal cancer patients who declined.

    abstract:OBJECTIVES:To evaluate the responsiveness of generic and mapped preference-based measures based on the anchor of global change in health condition of colorectal cancer (CRC) patients. STUDY DESIGN AND SETTING:A baseline sample of 333 Chinese CRC patients was recruited between September 2009 and July 2010 and was surve...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2013.08.011

    authors: Wong CK,Mulhern B,Wan YF,Lam CL

    更新日期:2014-02-01 00:00:00

  • A stroke prediction score in the elderly: validation and Web-based application.

    abstract::The objective of this study was to construct a prediction model for predicting stroke in an elderly U.S. population, and to assess the accuracy in this population of other previously published prediction models. The subjects were participants in the Cardiovascular Health Study: 2,495 men and 3,393 women age 65 years a...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(01)00434-6

    authors: Lumley T,Kronmal RA,Cushman M,Manolio TA,Goldstein S

    更新日期:2002-02-01 00:00:00

  • Decline in mortality from heart disease in Denmark: some methodological problems.

    abstract::Mortality rates in Denmark from ischemic heart diseases (IHD), other heart diseases and unknown causes are presented for the period 1968-92. In all age groups, mortality from IHD is higher at the beginning of the period than at the end. For other heart disease, the plot of the mortality rate is U-shaped for the age gr...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(94)00158-m

    authors: Juel K,Sjøl A

    更新日期:1995-04-01 00:00:00

  • Use of goal attainment scaling in measuring clinically important change in cognitive rehabilitation patients.

    abstract::Measuring the effectiveness of cognitive rehabilitation programs poses both conceptual and practical challenges. We compared several standardized outcome measures with goal attainment scaling (GAS) to assess their sensitivity to changes in health status in patients undergoing cognitive rehabilitation. GAS is a measure...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(97)00014-0

    authors: Rockwood K,Joyce B,Stolee P

    更新日期:1997-05-01 00:00:00

  • Geriatrics index of comorbidity was the most accurate predictor of death in geriatric hospital among six comorbidity scores.

    abstract:OBJECTIVES:To compare the abilities of six validated comorbidity indices (Charlson index, cumulative illness rating scale [CIRS], index of coexistent diseases, Kaplan scale, geriatrics index of comorbidity [GIC], and chronic disease score) to predict adverse hospitalization outcomes (death during hospitalization, lengt...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2009.11.013

    authors: Zekry D,Loures Valle BH,Lardi C,Graf C,Michel JP,Gold G,Krause KH,Herrmann FR

    更新日期:2010-09-01 00:00:00

  • Clinical correlates of advanced retinopathy in type II diabetic patients: implications for screening. The CODIAB-INSERM-Zeneca Pharma Study Group.

    abstract::The clinical correlates of advanced retinopathy were determined in a sample of 427 type II diabetic outpatients, aged 35 to 74 years, recruited in eight centers from all parts of France. The presence of retinopathy was assessed by fluorescein angiography with centralized interpretation. Advanced retinopathy (prolifera...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,多中心研究

    doi:10.1016/0895-4356(95)00582-x

    authors: Delcourt C,Villatte-Cathelineau B,Vauzelle-Kervroedan F,Papoz L

    更新日期:1996-06-01 00:00:00

  • False discovery rate control is a recommended alternative to Bonferroni-type adjustments in health studies.

    abstract:OBJECTIVES:Procedures for controlling the false positive rate when performing many hypothesis tests are commonplace in health and medical studies. Such procedures, most notably the Bonferroni adjustment, suffer from the problem that error rate control cannot be localized to individual tests, and that these procedures d...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,评审

    doi:10.1016/j.jclinepi.2014.03.012

    authors: Glickman ME,Rao SR,Schultz MR

    更新日期:2014-08-01 00:00:00

  • Patient characteristics and treatments associated with a decline in upper-body function following breast cancer therapy.

    abstract::Breast cancer therapy is often followed by a decline in upper-body function. Women (303) diagnosed with stage I or II breast cancer were interviewed 5 and 21 months after surgery and their medical records were reviewed. Women with cardiopulmonary comorbidity had an odds ratio for decline at the 5-month interview of 2....

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(99)00176-6

    authors: Lash TL,Silliman RA

    更新日期:2000-06-01 00:00:00

  • In network meta-analysis, most of the information comes from indirect evidence: empirical study.

    abstract:OBJECTIVES:Network meta-analysis (NMA) may produce more precise estimates of treatment effects than pairwise meta-analysis. We examined the relative contribution of network paths of different lengths to estimates of treatment effects. STUDY DESIGN AND SETTING:We analyzed 213 published NMAs. We categorized network shap...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2020.04.009

    authors: Papakonstantinou T,Nikolakopoulou A,Egger M,Salanti G

    更新日期:2020-08-01 00:00:00

  • The number needed to treat (NNT) can be adjusted for bias when the outcome is measured with error.

    abstract:BACKGROUND AND OBJECTIVE:We consider the number needed to treat (NNT) when the event of interest is defined by dichotomizing a continuous response at a threshold level. If the response is measured with error, the resulting NNT is biased. We consider methods to reduce this bias. METHODS:Bias adjustment was studied usin...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2004.01.021

    authors: Marschner IC,Emberson J,Irwig L,Walter SD

    更新日期:2004-12-01 00:00:00

  • Recall bias in case-control studies: an empirical analysis and theoretical framework.

    abstract::Previous work has suggested that recall bias in case-control studies may be more serious when the overall study quality is lower. This paper summarizes a systematic literature search to examine the question. All relevant studies published between 1966 and 1990 were included if they met the following criteria: (1) they...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(94)00132-a

    authors: Chouinard E,Walter S

    更新日期:1995-02-01 00:00:00

  • Mortality in a population with long-term exposure to inorganic selenium via drinking water.

    abstract::We analyzed the 1986-1997 mortality in a cohort of 2065 residents of an Italian municipality which had been exposed to drinking water with a high content of inorganic selenium over a long period of time, and compared it with mortality in the remainder of the municipal population. Mortality from malignant neoplasms inc...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(00)00233-x

    authors: Vinceti M,Nacci G,Rocchi E,Cassinadri T,Vivoli R,Marchesi C,Bergomi M

    更新日期:2000-10-01 00:00:00

  • Abbreviated literature searches were viable alternatives to comprehensive searches: a meta-epidemiological study.

    abstract:OBJECTIVE:To assess the effects of abbreviated literature searches on evidence syntheses conclusions. STUDY DESIGN AND SETTING:We randomly selected 60 Cochrane reviews of clinical interventions and repeated literature searches using 14 abbreviated approaches (combinations of MEDLINE, Embase, CENTRAL with and without s...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,meta分析

    doi:10.1016/j.jclinepi.2018.05.022

    authors: Nussbaumer-Streit B,Klerings I,Wagner G,Heise TL,Dobrescu AI,Armijo-Olivo S,Stratil JM,Persad E,Lhachimi SK,Van Noord MG,Mittermayr T,Zeeb H,Hemkens L,Gartlehner G

    更新日期:2018-10-01 00:00:00

  • Modelling the relative risk of esophageal cancer in a case-control study.

    abstract::The aim of this paper was to evaluate the combined effect of alcohol, smoking habits and frequency of consumption of selected dietary items, which are the major sources of retinoids and beta-carotene in the Italian diet, on the risk of esophageal cancer (EC). The adequacy of the multiplicative structure for describing...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(92)90035-l

    authors: Valsecchi MG

    更新日期:1992-04-01 00:00:00

  • Public and private hypotheses.

    abstract::The uses of hypothesis in scientific medicine and in clinical medicine are broadly similar; but they differ in subtle but important logical details. The key distinction is that scientific medicine deals with broad ("public") hypotheses about entire populations; and the predominant problem of inference is statistics in...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(89)90028-0

    authors: Murphy EA

    更新日期:1989-01-01 00:00:00

  • An embedded mixed-methods study highlighted a lack of discussions on retention in clinical trial consultations.

    abstract:OBJECTIVES:This study investigated trial consultations to identify whether and to what extent discussions of retention are present. STUDY DESIGN AND SETTING:This embedded mixed-methods study design included a purposive sample of audio-recorded trial consultations obtained from four sites of a large multicenter UK-base...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2020.03.011

    authors: Tunji-Ajayi P,Duncan EM,Gillies K

    更新日期:2020-07-01 00:00:00

  • Statistical analysis of failure times in total joint replacement.

    abstract::Time to revision is an important criterion describing the quality of implants in total joint surgery. Estimates of failure probabilities are required to inform a patient about the risk of suffering a reoperation. Also, regression models are used for comparing different prosthesis designs. Typically, patients dying bef...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(01)00371-7

    authors: Schwarzer G,Schumacher M,Maurer TB,Ochsner PE

    更新日期:2001-10-01 00:00:00

  • Synthesis of genetic association studies for pertinent gene-disease associations requires appropriate methodological and statistical approaches.

    abstract:OBJECTIVE:The aim of the study was to consider statistical and methodological issues affecting the results of meta-analysis of genetic association studies for pertinent gene-disease associations. Although the basic statistical issues for performing meta-analysis are well described in the literature, there are remaining...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,评审

    doi:10.1016/j.jclinepi.2007.12.011

    authors: Zintzaras E,Lau J

    更新日期:2008-07-01 00:00:00

  • Lack of correlation between clinic and 24 hour ambulatory blood pressure in subjects participating in a therapeutic drug trial.

    abstract::Although previous studies document that blood pressure measured in the clinic may differ significantly from blood pressure recorded over 24 hours, the impact of these differences on patient selection for therapeutic drug trials is unknown. A placebo controlled double-blind evaluation of an experimental antihypertensiv...

    journal_title:Journal of clinical epidemiology

    pub_type: 临床试验,杂志文章

    doi:10.1016/0895-4356(89)90046-2

    authors: Gradman AH,Pangan P,Germain M

    更新日期:1989-01-01 00:00:00

  • A grounded patient-centered approach generated the personal and societal burden of osteoarthritis model.

    abstract:OBJECTIVE:To develop a conceptual model to describe the burden of osteoarthritis in individuals with this condition and on the broader community. STUDY DESIGN AND SETTING:Six concept mapping workshops were undertaken: three with patients (n = 26) and three with health professionals (n = 27) in Australia and Sweden. Th...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2013.03.012

    authors: Busija L,Buchbinder R,Osborne RH

    更新日期:2013-09-01 00:00:00