Evaluating treatments in health care: the instability of a one-legged stool.

Abstract:

BACKGROUND:Both scientists and the public routinely refer to randomized controlled trials (RCTs) as being the 'gold standard' of scientific evidence. Although there is no question that placebo-controlled RCTs play a significant role in the evaluation of new pharmaceutical treatments, especially when it is important to rule out placebo effects, they have many inherent limitations which constrain their ability to inform medical decision making. The purpose of this paper is to raise questions about over-reliance on RCTs and to point out an additional perspective for evaluating healthcare evidence, as embodied in the Hill criteria. The arguments presented here are generally relevant to all areas of health care, though mental health applications provide the primary context for this essay. DISCUSSION:This article first traces the history of RCTs, and then evaluates five of their major limitations: they often lack external validity, they have the potential for increasing health risk in the general population, they are no less likely to overestimate treatment effects than many other methods, they make a relatively weak contribution to clinical practice, and they are excessively expensive (leading to several additional vulnerabilities in the quality of evidence produced). Next, the nine Hill criteria are presented and discussed as a richer approach to the evaluation of health care treatments. Reliance on these multi-faceted criteria requires more analytical thinking than simply examining RCT data, but will also enhance confidence in the evaluation of novel treatments. SUMMARY:Excessive reliance on RCTs tends to stifle funding of other types of research, and publication of other forms of evidence. We call upon our research and clinical colleagues to consider additional methods of evaluating data, such as the Hill criteria. Over-reliance on RCTs is similar to resting all of health care evidence on a one-legged stool.

journal_name

BMC Med Res Methodol

authors

Kaplan BJ,Giesbrecht G,Shannon S,McLeod K

doi

10.1186/1471-2288-11-65

subject

Has Abstract

pub_date

2011-05-11 00:00:00

pages

65

issn

1471-2288

pii

1471-2288-11-65

journal_volume

11

pub_type

杂志文章
  • Using the Bernoulli trial approaches for detecting ordered alternatives.

    abstract:BACKGROUND:Diagnostic problems in clinical trials are sometimes ordinal. For example, colon tumor staging was performed according to the TNM classification. However, clinical data are limited by markedly small sample sizes in some stage. METHODS:We propose a distribution-free test for detecting ordered alternatives in...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-13-148

    authors: Chang CH,Chin CC,Yu WW,Huang YY

    更新日期:2013-12-05 00:00:00

  • Exploratory trials, confirmatory observations: a new reasoning model in the era of patient-centered medicine.

    abstract:BACKGROUND:The prevailing view in therapeutic clinical research today is that observational studies are useful for generating new hypotheses and that controlled experiments (i.e., randomized clinical trials, RCTs) are the most appropriate method for assessing and confirming the efficacy of interventions. DISCUSSION:Th...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-11-57

    authors: Sacristán JA

    更新日期:2011-04-25 00:00:00

  • Protecting patient privacy when sharing patient-level data from clinical trials.

    abstract:BACKGROUND:Greater transparency and, in particular, sharing of patient-level data for further scientific research is an increasingly important topic for the pharmaceutical industry and other organisations who sponsor and conduct clinical trials as well as generally in the interests of patients participating in studies....

    journal_title:BMC medical research methodology

    pub_type: 信件

    doi:10.1186/s12874-016-0169-4

    authors: Tucker K,Branson J,Dilleen M,Hollis S,Loughlin P,Nixon MJ,Williams Z

    更新日期:2016-07-08 00:00:00

  • Development of an integrative coding framework for evaluating context within implementation science.

    abstract:BACKGROUND:This research aims to explore an identified gap in implementation science methodology, that is, how to assess context in implementation research. Context is among the strongest influences on implementation success but is a construct that is poorly understood and reported within the literature. Consequently, ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01044-5

    authors: Rogers L,De Brún A,McAuliffe E

    更新日期:2020-06-15 00:00:00

  • On the censored cost-effectiveness analysis using copula information.

    abstract:BACKGROUND:Information and theory beyond copula concepts are essential to understand the dependence relationship between several marginal covariates distributions. In a therapeutic trial data scheme, most of the time, censoring occurs. That could lead to a biased interpretation of the dependence relationship between ma...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0305-9

    authors: Fontaine C,Daurès JP,Landais P

    更新日期:2017-02-15 00:00:00

  • Testing the treatment effect on competing causes of death in oncology clinical trials.

    abstract:BACKGROUND:Chemotherapy is expected to reduce cancer deaths (CD), while possibly being harmful in terms of non-cancer deaths (NCD) because of toxicity. Peto's log-rank test is popular in the medical literature, but its operating characteristics are barely known. We compared this test to the most common ones in the stat...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-14-72

    authors: Rotolo F,Michiels S

    更新日期:2014-05-29 00:00:00

  • Is a large scale community programme as effective as a community rehabilitation programme delivered in the setting of a clinical trial?

    abstract:BACKGROUND:The rationale for commissioning community pulmonary rehabilitation programmes is based on evidence from randomised clinical trials. However, there are a number of reasons why similar programmes might be less effective outside the environment of a clinical trial. These include a less highly selected patient g...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-13-103

    authors: Goyder EC,Strong M,Green A,Holmes MW,Miles G,Reddington O,Lawson R,Lee A,Basran G

    更新日期:2013-08-13 00:00:00

  • Reported frequency of physical activity in a large epidemiological study: relationship to specific activities and repeatability over time.

    abstract:BACKGROUND:How overall physical activity relates to specific activities and how reported activity changes over time may influence interpretation of observed associations between physical activity and health. We examine the relationships between various physical activities self-reported at different times in a large coh...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-11-97

    authors: Armstrong ME,Cairns BJ,Green J,Reeves GK,Beral V,Million Women Study Collaborators.

    更新日期:2011-06-22 00:00:00

  • Development and preliminary validation of a brief nurses' perceived professional benefit questionnaire (NPPBQ).

    abstract:BACKGROUND:With the increased empirical interest in the positive significance of improving nurses' sense of professional benefits, there is a requirement for measures of nurses' perceived professional benefit (NPPB). Our objective was to develop and psychometrically test a brief Nurses' Perceived Professional Benefit Q...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,多中心研究

    doi:10.1186/s12874-020-0908-4

    authors: Hu Y,Hu J,Li L,Zhao B,Liu X,Li F

    更新日期:2020-01-30 00:00:00

  • Does the association between adherence to statin medications and mortality depend on measurement approach? A retrospective cohort study.

    abstract:BACKGROUND:The aim of this study was to examine the relationship between mortality and statin adherence using two different approaches to adherence measurement (summary versus repeated-measures). METHODS:A retrospective cohort study was conducted using administrative data from Saskatchewan, Canada between 1994 and 200...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0339-z

    authors: Alsabbagh MW,Eurich D,Lix LM,Wilson TW,Blackburn DF

    更新日期:2017-04-20 00:00:00

  • The use of relative incidence ratios in self-controlled case series studies: an overview.

    abstract:BACKGROUND:The self-controlled case series (SCCS) is a useful design for investigating associations between outcomes and transient exposures. The SCCS design controls for all fixed covariates, but effect modification can still occur. This can be evaluated by including interaction terms in the model which, when exponent...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-016-0225-0

    authors: Hawken S,Potter BK,Little J,Benchimol EI,Mahmud S,Ducharme R,Wilson K

    更新日期:2016-09-23 00:00:00

  • Prediction intervals for future BMI values of individual children: a non-parametric approach by quantile boosting.

    abstract:BACKGROUND:The construction of prediction intervals (PIs) for future body mass index (BMI) values of individual children based on a recent German birth cohort study with n = 2007 children is problematic for standard parametric approaches, as the BMI distribution in childhood is typically skewed depending on age. METHO...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-6

    authors: Mayr A,Hothorn T,Fenske N

    更新日期:2012-01-25 00:00:00

  • Recruiting and motivating black subjects to complete a lengthy survey in a large cohort study: an exploration of different strategies.

    abstract:BACKGROUND:The effectiveness of multiple innovative recruitment strategies for enrolling Black/African American participants to the Adventist Health Study-2 (AHS-2) is described. The study's focus is diet and breast, prostate and colon cancer. METHODS:Promotions centered on trust, relationship building and incentives ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-14-46

    authors: Herring P,Butler T,Hall S,Bennett H,Montgomery SB,Fraser G

    更新日期:2014-04-03 00:00:00

  • Evaluation of a modified version of the Posttraumatic Growth Inventory-Short Form.

    abstract:BACKGROUND:Posttraumatic growth is the positive change resulting from traumatic experiences and is typically assessed with retrospective measures like the Posttraumatic Growth Inventory (PTGI). The PTGI was designed to include reference to a specific traumatic event, making it difficult to implement, without change, in...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0344-2

    authors: Kaur N,Porter B,LeardMann CA,Tobin LE,Lemus H,Luxton DD,Millennium Cohort Study Team.

    更新日期:2017-04-20 00:00:00

  • Can electronic search engines optimize screening of search results in systematic reviews: an empirical study.

    abstract:BACKGROUND:Most electronic search efforts directed at identifying primary studies for inclusion in systematic reviews rely on the optimal Boolean search features of search interfaces such as DIALOG and Ovid. Our objective is to test the ability of an Ultraseek search engine to rank MEDLINE records of the included studi...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-6-7

    authors: Sampson M,Barrowman NJ,Moher D,Clifford TJ,Platt RW,Morrison A,Klassen TP,Zhang L

    更新日期:2006-02-24 00:00:00

  • Awareness of wearing an accelerometer does not affect physical activity in youth.

    abstract:BACKGROUND:This study aimed to investigate whether awareness of being monitored by an accelerometer has an effect on physical activity in young people. METHODS:Eighty healthy participants aged 10-18 years were randomized between blinded and nonblinded groups. The blinded participants were informed that we were testing...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,随机对照试验

    doi:10.1186/s12874-017-0378-5

    authors: Vanhelst J,Béghin L,Drumez E,Coopman S,Gottrand F

    更新日期:2017-07-11 00:00:00

  • Accounting for misclassification bias of binary outcomes due to underscreening: a sensitivity analysis.

    abstract:BACKGROUND:Diagnostic tests are performed in a subset of the population who are at higher risk, resulting in undiagnosed cases among those who do not receive the test. This poses a challenge for estimating the prevalence of the disease in the study population, and also for studying the risk factors for the disease. ME...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0447-9

    authors: Zhang N,Cheng S,Ambroggio L,Florin TA,Macaluso M

    更新日期:2017-12-12 00:00:00

  • The fading of reported effectiveness. A meta-analysis of randomised controlled trials.

    abstract:BACKGROUND:The "real" effect size of a medical therapy is constant over time. In contrast, the effect size reported in randomised controlled trials (RCTs) may change over time because the sum of all kinds of bias influencing the reported effectiveness is not necessarily constant. As this would affect the validity of me...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,meta分析

    doi:10.1186/1471-2288-6-25

    authors: Gehr BT,Weiss C,Porzsolt F

    更新日期:2006-05-11 00:00:00

  • Handling coarsened age information in the analysis of emergency department presentations.

    abstract:BACKGROUND:Administrative databases offer vast amounts of data that provide opportunities for cost-effective insights. They simultaneously pose significant challenges to statistical analysis such as the redaction of data because of privacy policies and the provision of data that may not be at the level of detail requir...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01181-x

    authors: Rosychuk RJ,Bachman JWN,Chen A,Hu XJ

    更新日期:2020-12-07 00:00:00

  • Mechanisms and pathways to impact in public health research: a preliminary analysis of research funded by the National Institute for Health Research (NIHR).

    abstract:BACKGROUND:The mechanisms and pathways to impacts from public health research in the UK have not been widely studied. Through the lens of one funder (NIHR), our aims are to map the diversity of public health research, in terms of funding mechanisms, disciplinary contributions, and public health impacts, identify exampl...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-0905-7

    authors: Boulding H,Kamenetzky A,Ghiga I,Ioppolo B,Herrera F,Parks S,Manville C,Guthrie S,Hinrichs-Krapels S

    更新日期:2020-02-19 00:00:00

  • Simulation-based estimation of mean and standard deviation for meta-analysis via Approximate Bayesian Computation (ABC).

    abstract:BACKGROUND:When conducting a meta-analysis of a continuous outcome, estimated means and standard deviations from the selected studies are required in order to obtain an overall estimate of the mean effect and its confidence interval. If these quantities are not directly reported in the publications, they must be estima...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0055-5

    authors: Kwon D,Reis IM

    更新日期:2015-08-12 00:00:00

  • Effects of the search technique on the measurement of the change in quality of randomized controlled trials over time in the field of brain injury.

    abstract:BACKGROUND:To determine if the search technique that is used to sample randomized controlled trial (RCT) manuscripts from a field of medical science can influence the measurement of the change in quality over time in that field. METHODS:RCT manuscripts in the field of brain injury were identified using two readily-ava...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-5-7

    authors: Borsody MK,Yamada C

    更新日期:2005-02-07 00:00:00

  • Predictor characteristics necessary for building a clinically useful risk prediction model: a simulation study.

    abstract:BACKGROUND:Compelled by the intuitive appeal of predicting each individual patient's risk of an outcome, there is a growing interest in risk prediction models. While the statistical methods used to build prediction models are increasingly well understood, the literature offers little insight to researchers seeking to g...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-016-0223-2

    authors: Schummers L,Himes KP,Bodnar LM,Hutcheon JA

    更新日期:2016-09-21 00:00:00

  • Empirical study of correlated survival times for recurrent events with proportional hazards margins and the effect of correlation and censoring.

    abstract:BACKGROUND:In longitudinal studies where subjects experience recurrent incidents over a period of time, such as respiratory infections, fever or diarrhea, statistical methods are required to take into account the within-subject correlation. METHODS:For repeated events data with censored failure, the independent increm...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-13-95

    authors: Villegas R,Julià O,Ocaña J

    更新日期:2013-07-24 00:00:00

  • Using the Beta distribution in group-based trajectory models.

    abstract:BACKGROUND:We demonstrate an application of Group-Based Trajectory Modeling (GBTM) based on the beta distribution. It is offered as an alternative to the normal distribution for modeling continuous longitudinal data that are poorly fit by the normal distribution even with censoring. The primary advantage of the beta di...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-018-0620-9

    authors: Elmer J,Jones BL,Nagin DS

    更新日期:2018-11-26 00:00:00

  • Does the provision of a DVD-based audio-visual presentation improve recruitment in a clinical trial? A randomised trial of DVD trial invitations.

    abstract:BACKGROUND:Recruitment to clinical trials can be challenging. Methods that improve the efficiency of trial recruitment are needed to increase successful study completions. The aim of this study was to ascertain whether sending an audio-visual presentation on a digital versatile disc (DVD), along with usual study invita...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,多中心研究,随机对照试验

    doi:10.1186/s12874-019-0663-6

    authors: Rogers A,Flynn RWV,Mackenzie IS,MacDonald TM

    更新日期:2019-01-30 00:00:00

  • Getting messier with TIDieR: embracing context and complexity in intervention reporting.

    abstract:BACKGROUND:The Template for Intervention Description and Replication (TIDieR) checklist and guide was developed by an international team of experts to promote full and accurate description of trial interventions. It is now widely used in health research. The aim of this paper is to describe the experience of using TIDi...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0461-y

    authors: Cotterill S,Knowles S,Martindale AM,Elvey R,Howard S,Coupe N,Wilson P,Spence M

    更新日期:2018-01-18 00:00:00

  • Using a social marketing framework to evaluate recruitment of a prospective study of genetic counseling and testing for the deaf community.

    abstract:BACKGROUND:Recruiting deaf and hard-of-hearing participants, particularly sign language-users, for genetics health service research is challenging due to communication barriers, mistrust toward genetics, and researchers' unfamiliarity with deaf people. Feelings of social exclusion and lack of social cohesion between re...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-13-145

    authors: Kobayashi Y,Boudreault P,Hill K,Sinsheimer JS,Palmer CG

    更新日期:2013-11-25 00:00:00

  • Centralising and optimising decentralised stroke care systems: a simulation study on short-term costs and effects.

    abstract:BACKGROUND:Centralisation of thrombolysis may offer substantial benefits. The aim of this study was to assess short term costs and effects of centralisation of thrombolysis and optimised care in a decentralised system. METHODS:Using simulation modelling, three scenarios to improve decentralised settings in the North o...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,多中心研究

    doi:10.1186/s12874-016-0275-3

    authors: Lahr MM,van der Zee DJ,Luijckx GJ,Vroomen PC,Buskens E

    更新日期:2017-01-10 00:00:00

  • Methods to increase response rates to a population-based maternity survey: a comparison of two pilot studies.

    abstract:BACKGROUND:Surveys are established methods for collecting population data that are unavailable from other sources; however, response rates to surveys are declining. A number of methods have been identified to increase survey returns yet response rates remain low. This paper evaluates the impact of five selected methods...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0702-3

    authors: Harrison S,Henderson J,Alderdice F,Quigley MA

    更新日期:2019-03-20 00:00:00