Likelihood-based random-effects meta-analysis with few studies: empirical and simulation studies.

Abstract:

BACKGROUND:Standard random-effects meta-analysis methods perform poorly when applied to few studies only. Such settings however are commonly encountered in practice. It is unclear, whether or to what extent small-sample-size behaviour can be improved by more sophisticated modeling. METHODS:We consider likelihood-based methods, the DerSimonian-Laird approach, Empirical Bayes, several adjustment methods and a fully Bayesian approach. Confidence intervals are based on a normal approximation, or on adjustments based on the Student-t-distribution. In addition, a linear mixed model and two generalized linear mixed models (GLMMs) assuming binomial or Poisson distributed numbers of events per study arm are considered for pairwise binary meta-analyses. We extract an empirical data set of 40 meta-analyses from recent reviews published by the German Institute for Quality and Efficiency in Health Care (IQWiG). Methods are then compared empirically as well as in a simulation study, based on few studies, imbalanced study sizes, and considering odds-ratio (OR) and risk ratio (RR) effect sizes. Coverage probabilities and interval widths for the combined effect estimate are evaluated to compare the different approaches. RESULTS:Empirically, a majority of the identified meta-analyses include only 2 studies. Variation of methods or effect measures affects the estimation results. In the simulation study, coverage probability is, in the presence of heterogeneity and few studies, mostly below the nominal level for all frequentist methods based on normal approximation, in particular when sizes in meta-analyses are not balanced, but improve when confidence intervals are adjusted. Bayesian methods result in better coverage than the frequentist methods with normal approximation in all scenarios, except for some cases of very large heterogeneity where the coverage is slightly lower. Credible intervals are empirically and in the simulation study wider than unadjusted confidence intervals, but considerably narrower than adjusted ones, with some exceptions when considering RRs and small numbers of patients per trial-arm. Confidence intervals based on the GLMMs are, in general, slightly narrower than those from other frequentist methods. Some methods turned out impractical due to frequent numerical problems. CONCLUSIONS:In the presence of between-study heterogeneity, especially with unbalanced study sizes, caution is needed in applying meta-analytical methods to few studies, as either coverage probabilities might be compromised, or intervals are inconclusively wide. Bayesian estimation with a sensibly chosen prior for between-trial heterogeneity may offer a promising compromise.

journal_name

BMC Med Res Methodol

authors

Seide SE,Röver C,Friede T

doi

10.1186/s12874-018-0618-3

subject

Has Abstract

pub_date

2019-01-11 00:00:00

pages

16

issue

1

issn

1471-2288

pii

10.1186/s12874-018-0618-3

journal_volume

19

pub_type

杂志文章
  • Reliability of routinely collected anthropometric measurements in primary care.

    abstract:BACKGROUND:Measuring body mass index (BMI) has been proposed as a method of screening for preventive primary care and population surveillance of childhood obesity. However, the accuracy of routinely collected measurements has been questioned. The purpose of this study was to assess the reliability of height, length and...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0726-8

    authors: Carsley S,Parkin PC,Tu K,Pullenayegum E,Persaud N,Maguire JL,Birken CS,TARGet Kids! Collaboration.

    更新日期:2019-04-24 00:00:00

  • Strategy for recruitment and factors associated with motivation and satisfaction in a randomized trial with 210 healthy volunteers without financial compensation.

    abstract:BACKGROUND:The aim was to describe a strategy for recruitment of healthy volunteers (HV) to a randomized trial that assessed the efficacy of different telephone techniques to assist HV in performing cardiac massage for vital emergency. Participation in the randomized trial was not financially compensated, however HV we...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-15-2

    authors: Luzurier Q,Damm C,Lion F,Daniel C,Pellerin L,Tavolacci MP

    更新日期:2015-01-05 00:00:00

  • Understanding the predictive value of continuous markers for censored survival data using a likelihood ratio approach.

    abstract:BACKGROUND:The likelihood ratio function (LR), the ratio of conditional probabilities of obtaining a specific marker value among those with the event of interest over those without, provides an easily interpretable way to quantify the update of the risk prediction due to the knowledge of the marker value. The LR has be...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0721-0

    authors: Smith AM,Christodouleas JP,Hwang WT

    更新日期:2019-05-22 00:00:00

  • Novel study design to assess the utility of the copd assessment test in a primary care setting.

    abstract::The quality of a consultation provided by a physician can have a profound impact on the quality of care and patient engagement in treatment decisions. When the COPD Assessment Test (CAT) was developed, one of its aims was to aid the communication between physician and patient about the impact of COPD. We developed a n...

    journal_title:BMC medical research methodology

    pub_type: 信件

    doi:10.1186/1471-2288-13-63

    authors: Gruffydd-Jones K,Marsden H,Holmes S,Kardos P,Escamilla R,Dal Negro R,Roberts J,Nadeau G,Leather D,Jones P

    更新日期:2013-05-10 00:00:00

  • Bayesian methods in clinical trials: a Bayesian analysis of ECOG trials E1684 and E1690.

    abstract:BACKGROUND:E1684 was the pivotal adjuvant melanoma trial for establishment of high-dose interferon (IFN) as effective therapy of high-risk melanoma patients. E1690 was an intriguing effort to corroborate E1684, and the differences between the outcomes of these trials have embroiled the field in controversy over the pas...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-183

    authors: Ibrahim JG,Chen MH,Chu H

    更新日期:2012-11-29 00:00:00

  • A proof of principle for using adaptive testing in Routine Outcome Monitoring: the efficiency of the Mood and Anxiety Symptoms Questionnaire -Anhedonic Depression CAT.

    abstract:BACKGROUND:In Routine Outcome Monitoring (ROM) there is a high demand for short assessments. Computerized Adaptive Testing (CAT) is a promising method for efficient assessment. In this article, the efficiency of a CAT version of the Mood and Anxiety Symptom Questionnaire, - Anhedonic Depression scale (MASQ-AD) for use ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-4

    authors: Smits N,Zitman FG,Cuijpers P,den Hollander-Gijsman ME,Carlier IV

    更新日期:2012-01-10 00:00:00

  • Positive predictive value of a case definition for diabetes mellitus using automated administrative health data in children and youth exposed to antipsychotic drugs or control medications: a Tennessee Medicaid study.

    abstract:BACKGROUND:We developed and validated an automated database case definition for diabetes in children and youth to facilitate pharmacoepidemiologic investigations of medications and the risk of diabetes. METHODS:The present study was part of an in-progress retrospective cohort study of antipsychotics and diabetes in Te...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-128

    authors: Bobo WV,Cooper WO,Stein CM,Olfson M,Mounsey J,Daugherty J,Ray WA

    更新日期:2012-08-24 00:00:00

  • Assembling and validating a heart failure-free cohort from the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study.

    abstract:BACKGROUND:Studies examining incident heart failure (HF) have been limited to select populations. To examine incident HF with broader generalizability, there is need to assemble a HF-free cohort using a geographically-diverse sample. We aimed to develop and validate a simple medication-based strategy for assembling a H...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0890-x

    authors: Goyal P,Mefford MT,Chen L,Sterling MR,Durant RW,Safford MM,Levitan EB

    更新日期:2020-03-04 00:00:00

  • Measuring health-related quality of life in chronic obstructive pulmonary disease: properties of the EQ-5D-5L and PROMIS-43 short form.

    abstract:BACKGROUND:The Patient Reported Outcomes Measurement Information System 43-item short form (PROMIS-43) and the five-level EQ-5D (EQ-5D-5L) are recently developed measures of health-related quality of life (HRQL) that have potentially broad application in evaluating treatments and capturing burden of respiratory-related...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,多中心研究

    doi:10.1186/1471-2288-14-78

    authors: Lin FJ,Pickard AS,Krishnan JA,Joo MJ,Au DH,Carson SS,Gillespie S,Henderson AG,Lindenauer PK,McBurnie MA,Mularski RA,Naureckas ET,Vollmer WM,Lee TA,CONCERT Consortium.

    更新日期:2014-06-16 00:00:00

  • Survival prediction models since liver transplantation - comparisons between Cox models and machine learning techniques.

    abstract:BACKGROUND:Predicting survival of recipients after liver transplantation is regarded as one of the most important challenges in contemporary medicine. Hence, improving on current prediction models is of great interest.Nowadays, there is a strong discussion in the medical field about machine learning (ML) and whether it...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01153-1

    authors: Kantidakis G,Putter H,Lancia C,Boer J,Braat AE,Fiocco M

    更新日期:2020-11-16 00:00:00

  • Joint modelling of time-to-event and multivariate longitudinal outcomes: recent developments and issues.

    abstract:BACKGROUND:Available methods for the joint modelling of longitudinal and time-to-event outcomes have typically only allowed for a single longitudinal outcome and a solitary event time. In practice, clinical studies are likely to record multiple longitudinal outcomes. Incorporating all sources of data will improve the p...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-016-0212-5

    authors: Hickey GL,Philipson P,Jorgensen A,Kolamunnage-Dona R

    更新日期:2016-09-07 00:00:00

  • Reference effect measures for quantifying, comparing and visualizing variation from random and fixed effects in non-normal multilevel models, with applications to site variation in medical procedure use and outcomes.

    abstract:BACKGROUND:Multilevel models for non-normal outcomes are widely used in medical and health sciences research. While methods for interpreting fixed effects are well-developed, methods to quantify and interpret random cluster variation and compare it with other sources of variation are less established. Random cluster va...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-018-0517-7

    authors: Glorioso TJ,Grunwald GK,Ho PM,Maddox TM

    更新日期:2018-07-06 00:00:00

  • Suitability of administrative claims databases for bariatric surgery research - is the glass half-full or half-empty?

    abstract:BACKGROUND:Claims databases are generally considered inadequate for obesity research due to suboptimal capture of body mass index (BMI) measurements. This might not be true for bariatric surgery because of reimbursement requirements and changes in coding systems. We assessed the availability and validity of claims-base...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01106-8

    authors: Li X,Lewis KH,Callaway K,Wharam JF,Toh S

    更新日期:2020-09-07 00:00:00

  • Consistency between stated and revealed preferences: a discrete choice experiment and a behavioural experiment on vaccination behaviour compared.

    abstract:BACKGROUND:Discrete Choice Experiments (DCEs) are increasingly used in studies in healthcare research but there is still little empirical evidence for the predictive value of these hypothetical situations in similar real life circumstances. The aim of this paper is to compare the stated preferences in a DCE and the acc...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0010-5

    authors: Lambooij MS,Harmsen IA,Veldwijk J,de Melker H,Mollema L,van Weert YW,de Wit GA

    更新日期:2015-03-12 00:00:00

  • Can a 'consent to contact' community help research teams overcome barriers to recruitment? The development and impact of the 'Research for the Future' community.

    abstract:BACKGROUND:Recruitment to health research remains a major challenge. Innovation is required to meet policy commitments to help patients take part in health research. One innovation that may help meet those policy goals is the development of 'consent to contact' systems, where people give generic consent to be contacted...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0843-4

    authors: Grady K,Gibson M,Bower P

    更新日期:2019-10-22 00:00:00

  • Structure formats of randomised controlled trial abstracts: a cross-sectional analysis of their current usage and association with methodology reporting.

    abstract:BACKGROUND:The reporting of randomised controlled trial (RCT) abstracts is of vital importance. The primary objective of this study was to investigate the association between structure format and RCT abstracts' quality of methodology reporting, informed by the current requirement and usage of structure formats by leadi...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0469-3

    authors: Hua F,Walsh T,Glenny AM,Worthington H

    更新日期:2018-01-10 00:00:00

  • Writing a discussion section: how to integrate substantive and statistical expertise.

    abstract:BACKGROUND:When discussing results medical research articles often tear substantive and statistical (methodical) contributions apart, just as if both were independent. Consequently, reasoning on bias tends to be vague, unclear and superficial. This can lead to over-generalized, too narrow and misleading conclusions, es...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,评审

    doi:10.1186/s12874-018-0490-1

    authors: Höfler M,Venz J,Trautmann S,Miller R

    更新日期:2018-04-17 00:00:00

  • Applying an intersectionality lens to the theoretical domains framework: a tool for thinking about how intersecting social identities and structures of power influence behaviour.

    abstract:BACKGROUND:A key component of the implementation process is identifying potential barriers and facilitators that need to be addressed. The Theoretical Domains Framework (TDF) is one of the most commonly used frameworks for this purpose. When applying the TDF, it is critical to understand the context in which behaviours...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01056-1

    authors: Etherington N,Rodrigues IB,Giangregorio L,Graham ID,Hoens AM,Kasperavicius D,Kelly C,Moore JE,Ponzano M,Presseau J,Sibley KM,Straus S

    更新日期:2020-06-26 00:00:00

  • Overview of systematic reviews of therapeutic ranges: methodologies and recommendations for practice.

    abstract:BACKGROUND:Many medicines are dosed to achieve a particular therapeutic range, and monitored using therapeutic drug monitoring (TDM). The evidence base for a therapeutic range can be evaluated using systematic reviews, to ensure it continues to reflect current indications, doses, routes and formulations, as well as upd...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,评审

    doi:10.1186/s12874-017-0363-z

    authors: Cooney L,Loke YK,Golder S,Kirkham J,Jorgensen A,Sinha I,Hawcutt D

    更新日期:2017-06-02 00:00:00

  • Imputation by the mean score should be avoided when validating a Patient Reported Outcomes questionnaire by a Rasch model in presence of informative missing data.

    abstract:BACKGROUND:Nowadays, more and more clinical scales consisting in responses given by the patients to some items (Patient Reported Outcomes - PRO), are validated with models based on Item Response Theory, and more specifically, with a Rasch model. In the validation sample, presence of missing data is frequent. The aim of...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-11-105

    authors: Hardouin JB,Conroy R,Sébille V

    更新日期:2011-07-14 00:00:00

  • Barriers and facilitators to patient recruitment to a cluster randomized controlled trial in primary care: lessons for future trials.

    abstract:BACKGROUND:Primary-care based randomized controlled trials (RCTs) build an important evidence base for general practice but little evidence exists about barriers to recruitment which often hamper such trials. We investigated the issues that impeded and facilitated recruitment to a clinical trial in general practice. M...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0012-3

    authors: Foster JM,Sawyer SM,Smith L,Reddel HK,Usherwood T

    更新日期:2015-03-12 00:00:00

  • The comprehensive cohort model in a pilot trial in orthopaedic trauma.

    abstract:BACKGROUND:The primary aim of this study was to provide an estimate of effect size for the functional outcome of operative versus non-operative treatment for patients with an acute rupture of the Achilles tendon using accelerated rehabilitation for both groups of patients. The secondary aim was to assess the use of a c...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,随机对照试验

    doi:10.1186/1471-2288-11-39

    authors: Kearney RS,Achten J,Parsons NR,Costa ML

    更新日期:2011-04-05 00:00:00

  • Determinants of non- response to a second assessment of lifestyle factors and body weight in the EPIC-PANACEA study.

    abstract:BACKGROUND:This paper discusses whether baseline demographic, socio-economic, health variables, length of follow-up and method of contacting the participants predict non-response to the invitation for a second assessment of lifestyle factors and body weight in the European multi-center EPIC-PANACEA study. METHODS:Over...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-148

    authors: May AM,Adema LE,Romaguera D,Vergnaud AC,Agudo A,Ekelund U,Steffen A,Orfanos P,Slimani N,Rinaldi S,Mouw T,Rohrmann S,Hermann S,Boeing H,Bergmann MM,Jakobsen MU,Overvad K,Wareham NJ,Gonzalez C,Tjonneland A,Halkjaer

    更新日期:2012-09-24 00:00:00

  • Analytical methods used in estimating the prevalence of HIV/AIDS from demographic and cross-sectional surveys with missing data: a systematic review.

    abstract:BACKGROUND:Sero- prevalence studies often have a problem of missing data. Few studies report the proportion of missing data and even fewer describe the methods used to adjust the results for missing data. The objective of this review was to determine the analytical methods used for analysis in HIV surveys with missing ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-00944-w

    authors: Mosha NR,Aluko OS,Todd J,Machekano R,Young T

    更新日期:2020-03-14 00:00:00

  • Practical considerations for sensitivity analysis after multiple imputation applied to epidemiological studies with incomplete data.

    abstract:BACKGROUND:Multiple Imputation as usually implemented assumes that data are Missing At Random (MAR), meaning that the underlying missing data mechanism, given the observed data, is independent of the unobserved data. To explore the sensitivity of the inferences to departures from the MAR assumption, we applied the meth...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-73

    authors: Héraud-Bousquet V,Larsen C,Carpenter J,Desenclos JC,Le Strat Y

    更新日期:2012-06-08 00:00:00

  • Validation of the French version of the Vulnerable Elders Survey-13 (VES-13).

    abstract:BACKGROUND:Identifying and assessing degree and type of frailty among older persons is a major challenge when targeting high risk populations to identify preventive interventions. The Vulnerable Elders Survey-(VES-13) is a simple instrument to identify frailty defined as risk for death, functional decline or institutio...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-0910-x

    authors: Belmin J,Khellaf L,Pariel S,Jarzebowski W,Valembois L,Zeisel J,Lafuente-Lafuente C

    更新日期:2020-02-05 00:00:00

  • Forecasts of COPD mortality in Australia: 2006-2025.

    abstract:BACKGROUND:Chronic Obstructive Pulmonary Disease (COPD) is currently the fifth leading cause of death in Australia, and there are marked differences in mortality trends between men and women. In this study, we have sought to model and forecast age related changes in COPD mortality over time for men and women separately...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-17

    authors: Erbas B,Ullah S,Hyndman RJ,Scollo M,Abramson M

    更新日期:2012-02-21 00:00:00

  • Comparing denominator degrees of freedom approximations for the generalized linear mixed model in analyzing binary outcome in small sample cluster-randomized trials.

    abstract:BACKGROUND:Small number of clusters and large variation of cluster sizes commonly exist in cluster-randomized trials (CRTs) and are often the critical factors affecting the validity and efficiency of statistical analyses. F tests are commonly used in the generalized linear mixed model (GLMM) to test intervention effect...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0026-x

    authors: Li P,Redden DT

    更新日期:2015-04-23 00:00:00

  • Integrating and extending cohort studies: lessons from the eXtending Treatments, Education and Networks in Depression (xTEND) study.

    abstract:BACKGROUND:Epidemiologic studies often struggle to adequately represent populations and outcomes of interest. Differences in methodology, data analysis and research questions often mean that reviews and synthesis of the existing literature have significant limitations. The current paper details our experiences in combi...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-13-122

    authors: Allen J,Inder KJ,Lewin TJ,Attia JR,Kay-Lambkin FJ,Baker AL,Hazell T,Kelly BJ

    更新日期:2013-10-05 00:00:00

  • Participant recruitment in sensitive surveys: a comparative trial of 'opt in' versus 'opt out' approaches.

    abstract:BACKGROUND:Although in health services survey research we strive for a high response rate, this must be balanced against the need to recruit participants ethically and considerately, particularly in surveys with a sensitive nature. In survey research there are no established recommendations to guide recruitment approac...

    journal_title:BMC medical research methodology

    pub_type: 临床试验,杂志文章

    doi:10.1186/1471-2288-13-3

    authors: Hunt KJ,Shlomo N,Addington-Hall J

    更新日期:2013-01-11 00:00:00