The thresholds for statistical and clinical significance - a five-step procedure for evaluation of intervention effects in randomised clinical trials.

Abstract:

BACKGROUND:Thresholds for statistical significance are insufficiently demonstrated by 95% confidence intervals or P-values when assessing results from randomised clinical trials. First, a P-value only shows the probability of getting a result assuming that the null hypothesis is true and does not reflect the probability of getting a result assuming an alternative hypothesis to the null hypothesis is true. Second, a confidence interval or a P-value showing significance may be caused by multiplicity. Third, statistical significance does not necessarily result in clinical significance. Therefore, assessment of intervention effects in randomised clinical trials deserves more rigour in order to become more valid. METHODS:Several methodologies for assessing the statistical and clinical significance of intervention effects in randomised clinical trials were considered. Balancing simplicity and comprehensiveness, a simple five-step procedure was developed. RESULTS:For a more valid assessment of results from a randomised clinical trial we propose the following five-steps: (1) report the confidence intervals and the exact P-values; (2) report Bayes factor for the primary outcome, being the ratio of the probability that a given trial result is compatible with a 'null' effect (corresponding to the P-value) divided by the probability that the trial result is compatible with the intervention effect hypothesised in the sample size calculation; (3) adjust the confidence intervals and the statistical significance threshold if the trial is stopped early or if interim analyses have been conducted; (4) adjust the confidence intervals and the P-values for multiplicity due to number of outcome comparisons; and (5) assess clinical significance of the trial results. CONCLUSIONS:If the proposed five-step procedure is followed, this may increase the validity of assessments of intervention effects in randomised clinical trials.

journal_name

BMC Med Res Methodol

authors

Jakobsen JC,Gluud C,Winkel P,Lange T,Wetterslev J

doi

10.1186/1471-2288-14-34

subject

Has Abstract

pub_date

2014-03-04 00:00:00

pages

34

issn

1471-2288

pii

1471-2288-14-34

journal_volume

14

pub_type

杂志文章
  • Dichotomisation using a distributional approach when the outcome is skewed.

    abstract:BACKGROUND:Dichotomisation of continuous outcomes has been rightly criticised by statisticians because of the loss of information incurred. However to communicate a comparison of risks, dichotomised outcomes may be necessary. Peacock et al. developed a distributional approach to the dichotomisation of normally distribu...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0028-8

    authors: Sauzet O,Ofuya M,Peacock JL

    更新日期:2015-04-24 00:00:00

  • Using an onset-anchored Bayesian hierarchical model to improve predictions for amyotrophic lateral sclerosis disease progression.

    abstract:BACKGROUND:Amyotrophic Lateral Sclerosis (ALS), also known as Lou Gehrig's disease, is a rare disease with extreme between-subject variability, especially with respect to rate of disease progression. This makes modelling a subject's disease progression, which is measured by the ALS Functional Rating Scale (ALSFRS), ver...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-018-0479-9

    authors: Karanevich AG,Statland JM,Gajewski BJ,He J

    更新日期:2018-02-06 00:00:00

  • Measurement and control of bias in patient reported outcomes using multidimensional item response theory.

    abstract:BACKGROUND:Patient-reported outcome (PRO) measures play a key role in the advancement of patient-centered care research. The accuracy of inferences, relevance of predictions, and the true nature of the associations made with PRO data depend on the validity of these measures. Errors inherent to self-report measures can ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-016-0161-z

    authors: Dowling NM,Bolt DM,Deng S,Li C

    更新日期:2016-05-26 00:00:00

  • Likelihood-based random-effects meta-analysis with few studies: empirical and simulation studies.

    abstract:BACKGROUND:Standard random-effects meta-analysis methods perform poorly when applied to few studies only. Such settings however are commonly encountered in practice. It is unclear, whether or to what extent small-sample-size behaviour can be improved by more sophisticated modeling. METHODS:We consider likelihood-based...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-018-0618-3

    authors: Seide SE,Röver C,Friede T

    更新日期:2019-01-11 00:00:00

  • Industry-supported meta-analyses compared with meta-analyses with non-profit or no support: differences in methodological quality and conclusions.

    abstract:BACKGROUND:Studies have shown that industry-sponsored meta-analyses of drugs lack scientific rigour and have biased conclusions. However, these studies have been restricted to certain medical specialities. We compared all industry-supported meta-analyses of drug-drug comparisons with those without industry support. ME...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-8-60

    authors: Jørgensen AW,Maric KL,Tendal B,Faurschou A,Gøtzsche PC

    更新日期:2008-09-09 00:00:00

  • Applying an intersectionality lens to the theoretical domains framework: a tool for thinking about how intersecting social identities and structures of power influence behaviour.

    abstract:BACKGROUND:A key component of the implementation process is identifying potential barriers and facilitators that need to be addressed. The Theoretical Domains Framework (TDF) is one of the most commonly used frameworks for this purpose. When applying the TDF, it is critical to understand the context in which behaviours...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01056-1

    authors: Etherington N,Rodrigues IB,Giangregorio L,Graham ID,Hoens AM,Kasperavicius D,Kelly C,Moore JE,Ponzano M,Presseau J,Sibley KM,Straus S

    更新日期:2020-06-26 00:00:00

  • Use of re-randomized data in meta-analysis.

    abstract:BACKGROUND:Outcomes collected in randomized clinical trials are observations of random variables that should be independent and identically distributed. However, in some trials, the patients are randomized more than once thus violating both of these assumptions. The probability of an event is not always the same when a...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-5-17

    authors: Hozo I,Djulbegovic B,Clark O,Lyman GH

    更新日期:2005-05-10 00:00:00

  • Accounting for misclassification bias of binary outcomes due to underscreening: a sensitivity analysis.

    abstract:BACKGROUND:Diagnostic tests are performed in a subset of the population who are at higher risk, resulting in undiagnosed cases among those who do not receive the test. This poses a challenge for estimating the prevalence of the disease in the study population, and also for studying the risk factors for the disease. ME...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0447-9

    authors: Zhang N,Cheng S,Ambroggio L,Florin TA,Macaluso M

    更新日期:2017-12-12 00:00:00

  • Centralising and optimising decentralised stroke care systems: a simulation study on short-term costs and effects.

    abstract:BACKGROUND:Centralisation of thrombolysis may offer substantial benefits. The aim of this study was to assess short term costs and effects of centralisation of thrombolysis and optimised care in a decentralised system. METHODS:Using simulation modelling, three scenarios to improve decentralised settings in the North o...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,多中心研究

    doi:10.1186/s12874-016-0275-3

    authors: Lahr MM,van der Zee DJ,Luijckx GJ,Vroomen PC,Buskens E

    更新日期:2017-01-10 00:00:00

  • An evaluation of DistillerSR's machine learning-based prioritization tool for title/abstract screening - impact on reviewer-relevant outcomes.

    abstract:BACKGROUND:Systematic reviews often require substantial resources, partially due to the large number of records identified during searching. Although artificial intelligence may not be ready to fully replace human reviewers, it may accelerate and reduce the screening burden. Using DistillerSR (May 2020 release), we eva...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01129-1

    authors: Hamel C,Kelly SE,Thavorn K,Rice DB,Wells GA,Hutton B

    更新日期:2020-10-15 00:00:00

  • Measuring sidewalk distances using Google Earth.

    abstract:BACKGROUND:Physical activity is an important determinant of health. Walking is the most common physical activity performed by adults and the presence of sidewalks along roads is a determinant of walking. Geographic information systems (GIS) can be used to measure sidewalks; however, GIS sidewalk data are difficult to a...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-39

    authors: Janssen I,Rosu A

    更新日期:2012-03-29 00:00:00

  • Empirical study of correlated survival times for recurrent events with proportional hazards margins and the effect of correlation and censoring.

    abstract:BACKGROUND:In longitudinal studies where subjects experience recurrent incidents over a period of time, such as respiratory infections, fever or diarrhea, statistical methods are required to take into account the within-subject correlation. METHODS:For repeated events data with censored failure, the independent increm...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-13-95

    authors: Villegas R,Julià O,Ocaña J

    更新日期:2013-07-24 00:00:00

  • Maximising response to postal questionnaires--a systematic review of randomised trials in health research.

    abstract:BACKGROUND:Postal self-completion questionnaires offer one of the least expensive modes of collecting patient based outcomes in health care research. The purpose of this review is to assess the efficacy of methods of increasing response to postal questionnaires in health care studies on patient populations. METHODS:Th...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,评审

    doi:10.1186/1471-2288-6-5

    authors: Nakash RA,Hutton JL,Jørstad-Stein EC,Gates S,Lamb SE

    更新日期:2006-02-23 00:00:00

  • Cross-cultural validation of the Functional Vision Questionnaire for Children and Young People (FVQ_CYP) with visual impairment in the Dutch population: challenges and opportunities.

    abstract:BACKGROUND:To assess cross-cultural validity between Dutch and English versions of the FVQ_CYP, a patient-reported outcome measure developed in the United Kingdom (UK) for children and adolescents with (severe) visual impairment or blindness (VI for brevity) to measure functional vision. METHODS:The 36-item FVQ_CYP wa...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0875-9

    authors: Elsman EBM,Tadić V,Peeters CFW,van Rens GHMB,Rahi JS,van Nispen RMA

    更新日期:2019-12-03 00:00:00

  • A systematic review to investigate the measurement properties of goal attainment scaling, towards use in drug trials.

    abstract:BACKGROUND:One of the main challenges for drug evaluation in rare diseases is the often heterogeneous course of these diseases. Traditional outcome measures may not be applicable for all patients, when they are in different stages of their disease. For instance, in Duchenne Muscular Dystrophy, the Six Minute Walk Test ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,评审

    doi:10.1186/s12874-016-0205-4

    authors: Gaasterland CM,Jansen-van der Weide MC,Weinreich SS,van der Lee JH

    更新日期:2016-08-17 00:00:00

  • Comparing survival curves based on medians.

    abstract:BACKGROUND:Although some nonparametric methods have been proposed in the literature to test for the equality of median survival times for censored data in medical research, in general they have inflated type I error rates, which make their use limited in practice, especially when the sample sizes are small. METHODS:In...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-016-0133-3

    authors: Chen Z,Zhang G

    更新日期:2016-03-16 00:00:00

  • Quasi-linear Cox proportional hazards model with cross- L1 penalty.

    abstract:BACKGROUND:To accurately predict the response to treatment, we need a stable and effective risk score that can be calculated from patient characteristics. When we evaluate such risks from time-to-event data with right-censoring, Cox's proportional hazards model is the most popular for estimating the linear risk score. ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01063-2

    authors: Omae K,Eguchi S

    更新日期:2020-07-06 00:00:00

  • Attempting rigour and replicability in thematic analysis of qualitative research data; a case study of codebook development.

    abstract:BACKGROUND:Navigating the world of qualitative thematic analysis can be challenging. This is compounded by the fact that detailed descriptions of methods are often omitted from qualitative discussions. While qualitative research methodologies are now mature, there often remains a lack of fine detail in their descriptio...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0707-y

    authors: Roberts K,Dowell A,Nie JB

    更新日期:2019-03-28 00:00:00

  • Evaluation of a modified version of the Posttraumatic Growth Inventory-Short Form.

    abstract:BACKGROUND:Posttraumatic growth is the positive change resulting from traumatic experiences and is typically assessed with retrospective measures like the Posttraumatic Growth Inventory (PTGI). The PTGI was designed to include reference to a specific traumatic event, making it difficult to implement, without change, in...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0344-2

    authors: Kaur N,Porter B,LeardMann CA,Tobin LE,Lemus H,Luxton DD,Millennium Cohort Study Team.

    更新日期:2017-04-20 00:00:00

  • Adjusting for under-identification of Aboriginal and/or Torres Strait Islander births in time series produced from birth records: using record linkage of survey data and administrative data sources.

    abstract:BACKGROUND:Statistical time series derived from administrative data sets form key indicators in measuring progress in addressing disadvantage in Aboriginal and Torres Strait Islander populations in Australia. However, inconsistencies in the reporting of Indigenous status can cause difficulties in producing reliable ind...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-90

    authors: Lawrence D,Christensen D,Mitrou F,Draper G,Davis G,McKeown S,McAullay D,Pearson G,Zubrick SR

    更新日期:2012-07-02 00:00:00

  • A random effects meta-analysis model with Box-Cox transformation.

    abstract:BACKGROUND:In a random effects meta-analysis model, true treatment effects for each study are routinely assumed to follow a normal distribution. However, normality is a restrictive assumption and the misspecification of the random effects distribution may result in a misleading estimate of overall mean for the treatmen...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0376-7

    authors: Yamaguchi Y,Maruo K,Partlett C,Riley RD

    更新日期:2017-07-19 00:00:00

  • Suitability of measures of self-reported medication adherence for routine clinical use: a systematic review.

    abstract:BACKGROUND:There is a recognised need to build primary care medication adherence services which are tailored to patients' needs. Continuous quality improvement of such services requires a regular working method of measuring adherence in order to monitor effectiveness. Self report has been considered the method of choic...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,评审

    doi:10.1186/1471-2288-11-149

    authors: Garfield S,Clifford S,Eliasson L,Barber N,Willson A

    更新日期:2011-11-03 00:00:00

  • Consistency between stated and revealed preferences: a discrete choice experiment and a behavioural experiment on vaccination behaviour compared.

    abstract:BACKGROUND:Discrete Choice Experiments (DCEs) are increasingly used in studies in healthcare research but there is still little empirical evidence for the predictive value of these hypothetical situations in similar real life circumstances. The aim of this paper is to compare the stated preferences in a DCE and the acc...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0010-5

    authors: Lambooij MS,Harmsen IA,Veldwijk J,de Melker H,Mollema L,van Weert YW,de Wit GA

    更新日期:2015-03-12 00:00:00

  • Simulation-based estimation of mean and standard deviation for meta-analysis via Approximate Bayesian Computation (ABC).

    abstract:BACKGROUND:When conducting a meta-analysis of a continuous outcome, estimated means and standard deviations from the selected studies are required in order to obtain an overall estimate of the mean effect and its confidence interval. If these quantities are not directly reported in the publications, they must be estima...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0055-5

    authors: Kwon D,Reis IM

    更新日期:2015-08-12 00:00:00

  • Clinical trial simulation to evaluate power to compare the antiviral effectiveness of two hepatitis C protease inhibitors using nonlinear mixed effect models: a viral kinetic approach.

    abstract:BACKGROUND:Models of hepatitis C virus (HCV) kinetics are increasingly used to estimate and to compare in vivo drug's antiviral effectiveness of new potent anti-HCV agents. Viral kinetic parameters can be estimated using non-linear mixed effect models (NLMEM). Here we aimed to evaluate the performance of this approach ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-13-60

    authors: Laouénan C,Guedj J,Mentré F

    更新日期:2013-04-25 00:00:00

  • Prediction intervals for future BMI values of individual children: a non-parametric approach by quantile boosting.

    abstract:BACKGROUND:The construction of prediction intervals (PIs) for future body mass index (BMI) values of individual children based on a recent German birth cohort study with n = 2007 children is problematic for standard parametric approaches, as the BMI distribution in childhood is typically skewed depending on age. METHO...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-6

    authors: Mayr A,Hothorn T,Fenske N

    更新日期:2012-01-25 00:00:00

  • Joint modelling of time-to-event and multivariate longitudinal outcomes: recent developments and issues.

    abstract:BACKGROUND:Available methods for the joint modelling of longitudinal and time-to-event outcomes have typically only allowed for a single longitudinal outcome and a solitary event time. In practice, clinical studies are likely to record multiple longitudinal outcomes. Incorporating all sources of data will improve the p...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-016-0212-5

    authors: Hickey GL,Philipson P,Jorgensen A,Kolamunnage-Dona R

    更新日期:2016-09-07 00:00:00

  • Sample size calculations for cluster randomised controlled trials with a fixed number of clusters.

    abstract:BACKGROUND:Cluster randomised controlled trials (CRCTs) are frequently used in health service evaluation. Assuming an average cluster size, required sample sizes are readily computed for both binary and continuous outcomes, by estimating a design effect or inflation factor. However, where the number of clusters are fix...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-11-102

    authors: Hemming K,Girling AJ,Sitch AJ,Marsh J,Lilford RJ

    更新日期:2011-06-30 00:00:00

  • A random effects variance shift model for detecting and accommodating outliers in meta-analysis.

    abstract:BACKGROUND:Meta-analysis typically involves combining the estimates from independent studies in order to estimate a parameter of interest across a population of studies. However, outliers often occur even under the random effects model. The presence of such outliers could substantially alter the conclusions in a meta-a...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-11-19

    authors: Gumedze FN,Jackson D

    更新日期:2011-02-16 00:00:00

  • Reference effect measures for quantifying, comparing and visualizing variation from random and fixed effects in non-normal multilevel models, with applications to site variation in medical procedure use and outcomes.

    abstract:BACKGROUND:Multilevel models for non-normal outcomes are widely used in medical and health sciences research. While methods for interpreting fixed effects are well-developed, methods to quantify and interpret random cluster variation and compare it with other sources of variation are less established. Random cluster va...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-018-0517-7

    authors: Glorioso TJ,Grunwald GK,Ho PM,Maddox TM

    更新日期:2018-07-06 00:00:00