A system for rating the stability and strength of medical evidence.

Abstract:

BACKGROUND:Methods for describing one's confidence in the available evidence are useful for end-users of evidence reviews. Analysts inevitably make judgments about the quality, quantity consistency, robustness, and magnitude of effects observed in the studies identified. The subjectivity of these judgments in several areas underscores the need for transparency in judgments. DISCUSSION:This paper introduces a new system for rating medical evidence. The system requires explicit judgments and provides explicit rules for balancing these judgments. Unlike other systems for rating the strength of evidence, our system draws a distinction between two types of conclusions: quantitative and qualitative. A quantitative conclusion addresses the question, "How well does it work?", whereas a qualitative conclusion addresses the question, "Does it work?" In our system, quantitative conclusions are tied to stability ratings, and qualitative conclusions are tied to strength ratings. Our system emphasizes extensive a priori criteria for judgments to reduce the potential for bias. Further, the system makes explicit the impact of heterogeneity testing, meta-analysis, and sensitivity analyses on evidence ratings. This article provides details of our system, including graphical depictions of how the numerous judgments that an analyst makes can be combined. We also describe two worked examples of how the system can be applied to both interventional and diagnostic technologies. SUMMARY:Although explicit judgments and formal combination rules are two important steps on the path to a comprehensive system for rating medical evidence, many additional steps must also be taken. Foremost among these are the distinction between quantitative and qualitative conclusions, an extensive set of a priori criteria for making judgments, and the direct impact of analytic results on evidence ratings. These attributes form the basis for a logically consistent system that can improve the usefulness of evidence reviews.

journal_name

BMC Med Res Methodol

authors

Treadwell JR,Tregear SJ,Reston JT,Turkelson CM

doi

10.1186/1471-2288-6-52

subject

Has Abstract

pub_date

2006-10-19 00:00:00

pages

52

issn

1471-2288

pii

1471-2288-6-52

journal_volume

6

pub_type

杂志文章
  • Correction for retest effects across repeated measures of cognitive functioning: a longitudinal cohort study of postoperative delirium.

    abstract:BACKGROUND:Few studies have compared methods to correct for retest effects or practice effects in settings where an acute event could influence test performance, such as major surgery. Our goal in this study was to evaluate the use of different methods to correct for the effects of practice or retest on repeated test a...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-018-0530-x

    authors: Racine AM,Gou Y,Fong TG,Marcantonio ER,Schmitt EM,Travison TG,Inouye SK,Jones RN

    更新日期:2018-07-03 00:00:00

  • Linkage of the CHHiP randomised controlled trial with primary care data: a study investigating ways of supplementing cancer trials and improving evidence-based practice.

    abstract:BACKGROUND:Randomised controlled trials (RCTs) are the gold standard for evidence-based practice. However, RCTs can have limitations. For example, translation of findings into practice can be limited by design features, such as inclusion criteria, not accurately reflecting clinical populations. In addition, it is expen...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01078-9

    authors: Lemanska A,Byford RC,Cruickshank C,Dearnaley DP,Ferreira F,Griffin C,Hall E,Hinton W,de Lusignan S,Sherlock J,Faithfull S

    更新日期:2020-07-25 00:00:00

  • Psychometric analysis of the brief symptom inventory 18 (BSI-18) in a representative German sample.

    abstract:BACKGROUND:The BSI-18 contains the three six-item scales somatization, depression, and anxiety as well as the Global Severity Index (GSI), including all 18 items. The BSI-18 is the latest and shortest of the multidimensional versions of the Symptom-Checklist 90-R, but its psychometric properties have not been sufficien...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-016-0283-3

    authors: Franke GH,Jaeger S,Glaesmer H,Barkmann C,Petrowski K,Braehler E

    更新日期:2017-01-26 00:00:00

  • Adjusting for under-identification of Aboriginal and/or Torres Strait Islander births in time series produced from birth records: using record linkage of survey data and administrative data sources.

    abstract:BACKGROUND:Statistical time series derived from administrative data sets form key indicators in measuring progress in addressing disadvantage in Aboriginal and Torres Strait Islander populations in Australia. However, inconsistencies in the reporting of Indigenous status can cause difficulties in producing reliable ind...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-90

    authors: Lawrence D,Christensen D,Mitrou F,Draper G,Davis G,McKeown S,McAullay D,Pearson G,Zubrick SR

    更新日期:2012-07-02 00:00:00

  • Writing a discussion section: how to integrate substantive and statistical expertise.

    abstract:BACKGROUND:When discussing results medical research articles often tear substantive and statistical (methodical) contributions apart, just as if both were independent. Consequently, reasoning on bias tends to be vague, unclear and superficial. This can lead to over-generalized, too narrow and misleading conclusions, es...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,评审

    doi:10.1186/s12874-018-0490-1

    authors: Höfler M,Venz J,Trautmann S,Miller R

    更新日期:2018-04-17 00:00:00

  • Evaluation of a modified version of the Posttraumatic Growth Inventory-Short Form.

    abstract:BACKGROUND:Posttraumatic growth is the positive change resulting from traumatic experiences and is typically assessed with retrospective measures like the Posttraumatic Growth Inventory (PTGI). The PTGI was designed to include reference to a specific traumatic event, making it difficult to implement, without change, in...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0344-2

    authors: Kaur N,Porter B,LeardMann CA,Tobin LE,Lemus H,Luxton DD,Millennium Cohort Study Team.

    更新日期:2017-04-20 00:00:00

  • Bayesian methods in clinical trials: a Bayesian analysis of ECOG trials E1684 and E1690.

    abstract:BACKGROUND:E1684 was the pivotal adjuvant melanoma trial for establishment of high-dose interferon (IFN) as effective therapy of high-risk melanoma patients. E1690 was an intriguing effort to corroborate E1684, and the differences between the outcomes of these trials have embroiled the field in controversy over the pas...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-183

    authors: Ibrahim JG,Chen MH,Chu H

    更新日期:2012-11-29 00:00:00

  • Pragmatic controlled clinical trials in primary care: the struggle between external and internal validity.

    abstract:BACKGROUND:Controlled clinical trials of health care interventions are either explanatory or pragmatic. Explanatory trials test whether an intervention is efficacious; that is, whether it can have a beneficial effect in an ideal situation. Pragmatic trials measure effectiveness; they measure the degree of beneficial ef...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-3-28

    authors: Godwin M,Ruhland L,Casson I,MacDonald S,Delva D,Birtwhistle R,Lam M,Seguin R

    更新日期:2003-12-22 00:00:00

  • Validation of the French version of the Vulnerable Elders Survey-13 (VES-13).

    abstract:BACKGROUND:Identifying and assessing degree and type of frailty among older persons is a major challenge when targeting high risk populations to identify preventive interventions. The Vulnerable Elders Survey-(VES-13) is a simple instrument to identify frailty defined as risk for death, functional decline or institutio...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-0910-x

    authors: Belmin J,Khellaf L,Pariel S,Jarzebowski W,Valembois L,Zeisel J,Lafuente-Lafuente C

    更新日期:2020-02-05 00:00:00

  • Defeat and entrapment: more than meets the eye? Applying network analysis to estimate dimensions of highly correlated constructs.

    abstract:BACKGROUND:Defeat and entrapment have been shown to be of central relevance to the development of different disorders. However, it remains unclear whether they represent two distinct constructs or one overall latent variable. One reason for the unclarity is that traditional factor analytic techniques have trouble estim...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-018-0470-5

    authors: Forkmann T,Teismann T,Stenzel JS,Glaesmer H,de Beurs D

    更新日期:2018-01-25 00:00:00

  • Lifetime body size and reproductive factors: comparisons of data recorded prospectively with self reports in middle age.

    abstract:BACKGROUND:Data on lifetime exposures are often self-reported in epidemiologic studies, sometimes many years after the relevant age. Validity of self-reported data is usually inferred from their agreement with measured values, but few studies directly quantify the likely effects of reporting errors in body size and rep...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-11-7

    authors: Cairns BJ,Liu B,Clennell S,Cooper R,Reeves GK,Beral V,Kuh D

    更新日期:2011-01-17 00:00:00

  • Reliability, validity and administrative burden of the community reintegration of injured service members computer adaptive test (CRIS-CAT)".

    abstract:BACKGROUND:The Computer Adaptive Test version of the Community Reintegration of Injured Service Members measure (CRIS-CAT) consists of three scales measuring Extent of, Perceived Limitations in, and Satisfaction with community integration. The CRIS-CAT was developed using item response theory methods. The purposes of t...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-145

    authors: Resnik L,Borgia M,Ni P,Pirraglia PA,Jette A

    更新日期:2012-09-17 00:00:00

  • Power and sample size determination for the group comparison of patient-reported outcomes using the Rasch model: impact of a misspecification of the parameters.

    abstract:BACKGROUND:Patient-reported outcomes (PRO) are important as endpoints in clinical trials and epidemiological studies. Guidelines for the development of PRO instruments and analysis of PRO data have emphasized the need to report methods used for sample size planning. The Raschpower procedure has been proposed for sample...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0011-4

    authors: Blanchin M,Guilleux A,Perrot B,Bonnaud-Antignac A,Hardouin JB,Sébille V

    更新日期:2015-03-15 00:00:00

  • Protocol for a systematic review and individual patient data meta-analysis of prognostic factors of foot ulceration in people with diabetes: the international research collaboration for the prediction of diabetic foot ulcerations (PODUS).

    abstract:BACKGROUND:Diabetes-related lower limb amputations are associated with considerable morbidity and mortality and are usually preceded by foot ulceration. The available systematic reviews of aggregate data are compromised because the primary studies report both adjusted and unadjusted estimates. As adjusted meta-analyses...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,meta分析

    doi:10.1186/1471-2288-13-22

    authors: Crawford F,Anandan C,Chappell FM,Murray GD,Price JF,Sheikh A,Simpson CR,Maxwell M,Stansby GP,Young MJ,Abbott CA,Boulton AJ,Boyko EJ,Kastenbauer T,Leese GP,Monami M,Monteiro-Soares M,Rith-Najarian SJ,Veves A,Coates N

    更新日期:2013-02-15 00:00:00

  • Implementation of clinical research trials using web-based and mobile devices: challenges and solutions.

    abstract:BACKGROUND:With the increasing implementation of web-based, mobile health interventions in clinical trials, it is crucial for researchers to address the security and privacy concerns of patient information according to high ethical standards. The full process of meeting these standards is often made more complicated du...

    journal_title:BMC medical research methodology

    pub_type: 社论

    doi:10.1186/s12874-017-0324-6

    authors: Eagleson R,Altamirano-Diaz L,McInnis A,Welisch E,De Jesus S,Prapavessis H,Rombeek M,Seabrook JA,Park T,Norozi K

    更新日期:2017-03-17 00:00:00

  • A mixed methods case study investigating how randomised controlled trials (RCTs) are reported, understood and interpreted in practice.

    abstract:BACKGROUND:While randomised controlled trials (RCTs) provide high-quality evidence to guide practice, much routine care is not based upon available RCTs. This disconnect between evidence and practice is not sufficiently well understood. This case study explores this relationship using a novel approach. Better understan...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01009-8

    authors: Byrne BE,Rooshenas L,Lambert HS,Blazeby JM

    更新日期:2020-05-12 00:00:00

  • Overview of systematic reviews of therapeutic ranges: methodologies and recommendations for practice.

    abstract:BACKGROUND:Many medicines are dosed to achieve a particular therapeutic range, and monitored using therapeutic drug monitoring (TDM). The evidence base for a therapeutic range can be evaluated using systematic reviews, to ensure it continues to reflect current indications, doses, routes and formulations, as well as upd...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,评审

    doi:10.1186/s12874-017-0363-z

    authors: Cooney L,Loke YK,Golder S,Kirkham J,Jorgensen A,Sinha I,Hawcutt D

    更新日期:2017-06-02 00:00:00

  • European Medicines Agency Policy 0070: an exploratory review of data utility in clinical study reports for academic research.

    abstract:BACKGROUND:Clinical study reports (CSRs) have been increasingly utilised within academic research in recent years. European Medicines Agency (EMA) Policy 0070 'Phase 1,' which came into effect in January 2015, requires the publication of regulatory documents such as CSRs from central applications in an anonymised forma...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章,评审

    doi:10.1186/s12874-019-0836-3

    authors: Ferran JM,Nevitt SJ

    更新日期:2019-11-05 00:00:00

  • Consistency between stated and revealed preferences: a discrete choice experiment and a behavioural experiment on vaccination behaviour compared.

    abstract:BACKGROUND:Discrete Choice Experiments (DCEs) are increasingly used in studies in healthcare research but there is still little empirical evidence for the predictive value of these hypothetical situations in similar real life circumstances. The aim of this paper is to compare the stated preferences in a DCE and the acc...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-015-0010-5

    authors: Lambooij MS,Harmsen IA,Veldwijk J,de Melker H,Mollema L,van Weert YW,de Wit GA

    更新日期:2015-03-12 00:00:00

  • Norm values and psychometric properties for the German health regulatory focus scale - results of a representative survey.

    abstract:BACKGROUND:The health regulatory focus is an application of Higgins' regulatory focus theory to a health-specific context. It explains individual differences in health motivation, strategies, and behavior. Previous research found the Health Regulatory Focus Scale (HRFS) to be a reliable and valid measure for the constr...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-00927-x

    authors: Schmalbach B,Zenger M,Brähler E,Petrowski K

    更新日期:2020-03-02 00:00:00

  • Assembling and validating a heart failure-free cohort from the Reasons for Geographic and Racial Differences in Stroke (REGARDS) study.

    abstract:BACKGROUND:Studies examining incident heart failure (HF) have been limited to select populations. To examine incident HF with broader generalizability, there is need to assemble a HF-free cohort using a geographically-diverse sample. We aimed to develop and validate a simple medication-based strategy for assembling a H...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0890-x

    authors: Goyal P,Mefford MT,Chen L,Sterling MR,Durant RW,Safford MM,Levitan EB

    更新日期:2020-03-04 00:00:00

  • Prediction intervals for future BMI values of individual children: a non-parametric approach by quantile boosting.

    abstract:BACKGROUND:The construction of prediction intervals (PIs) for future body mass index (BMI) values of individual children based on a recent German birth cohort study with n = 2007 children is problematic for standard parametric approaches, as the BMI distribution in childhood is typically skewed depending on age. METHO...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-12-6

    authors: Mayr A,Hothorn T,Fenske N

    更新日期:2012-01-25 00:00:00

  • Structural issues affecting mixed methods studies in health research: a qualitative study.

    abstract:BACKGROUND:Health researchers undertake studies which combine qualitative and quantitative methods. Little attention has been paid to the structural issues affecting this mixed methods approach. We explored the facilitators and barriers to undertaking mixed methods studies in health research. METHODS:Face-to-face semi...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-9-82

    authors: O'Cathain A,Nicholl J,Murphy E

    更新日期:2009-12-09 00:00:00

  • Pan-Canadian assessment of pandemic immunization data collection: study methodology.

    abstract:BACKGROUND:The collection of individual-level pandemic (H1N1) 2009 influenza immunization data was considered important to facilitate optimal vaccine delivery and accurate assessment of vaccine coverage. These data are also critical for research aimed at evaluating the new vaccine's safety and effectiveness. Systems us...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-10-51

    authors: Pereira JA,Quach S,Heidebrecht C,Foisy J,Quan S,Finkelstein M,Sikora CA,Bettinger JA,Buckeridge DL,McCarthy A,Deeks S,Kwong JC,Public Health Agency of Canada\/Canadian Institutes of Health Research Influenza Research Network Vacc

    更新日期:2010-06-08 00:00:00

  • Strengthening policy coding methodologies to improve COVID-19 disease modeling and policy responses: a proposed coding framework and recommendations.

    abstract:BACKGROUND:In recent months, multiple efforts have sought to characterize COVID-19 social distancing policy responses. These efforts have used various coding frameworks, but many have relied on coding methodologies that may not adequately describe the gradient in social distancing policies as states "re-open." METHODS...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-01174-w

    authors: Lane J,Garrison MM,Kelley J,Sarma P,Katz A

    更新日期:2020-12-08 00:00:00

  • An international survey of physicians regarding clinical trials: a comparison between Kyoto University Hospital and Seoul National University Hospital.

    abstract:BACKGROUND:International clinical trials are now rapidly expanding into Asia. However, the proportion of global trials is higher in South Korea compared to Japan despite implementation of similar governmental support in both countries. The difference in clinical trial environment might influence the respective physicia...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-13-130

    authors: Ito-Ihara T,Hong JH,Kim OJ,Sumi E,Kim SY,Tanaka S,Narita K,Hatta T,Choi EK,Choi KJ,Miyagawa T,Minami M,Murayama T,Yokode M

    更新日期:2013-10-25 00:00:00

  • An information-motivation-behavioural skills analysis of long-lasting insecticidal net use among pregnant women in a hospital in North-Eastern Nigeria.

    abstract:BACKGROUND:Sleeping under a long-lasting insecticidal net (LLIN) is recommended for all pregnant women in sub-Saharan Africa, due to the high prevalence of malaria infection and its associated complications in the region. Despite this, LLIN use has still remained sub-optimal among pregnant women in Maiduguri, Nigeria. ...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-019-0803-z

    authors: Balami AD,Said SM,Zulkefli NAM,Bachok N,Balami EL

    更新日期:2019-07-18 00:00:00

  • Accounting for misclassification bias of binary outcomes due to underscreening: a sensitivity analysis.

    abstract:BACKGROUND:Diagnostic tests are performed in a subset of the population who are at higher risk, resulting in undiagnosed cases among those who do not receive the test. This poses a challenge for estimating the prevalence of the disease in the study population, and also for studying the risk factors for the disease. ME...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-017-0447-9

    authors: Zhang N,Cheng S,Ambroggio L,Florin TA,Macaluso M

    更新日期:2017-12-12 00:00:00

  • Mechanisms and pathways to impact in public health research: a preliminary analysis of research funded by the National Institute for Health Research (NIHR).

    abstract:BACKGROUND:The mechanisms and pathways to impacts from public health research in the UK have not been widely studied. Through the lens of one funder (NIHR), our aims are to map the diversity of public health research, in terms of funding mechanisms, disciplinary contributions, and public health impacts, identify exampl...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/s12874-020-0905-7

    authors: Boulding H,Kamenetzky A,Ghiga I,Ioppolo B,Herrera F,Parks S,Manville C,Guthrie S,Hinrichs-Krapels S

    更新日期:2020-02-19 00:00:00

  • Does updating improve the methodological and reporting quality of systematic reviews?

    abstract:BACKGROUND:Systematic reviews (SRs) must be of high quality. The purpose of our research was to compare the methodological and reporting quality of original versus updated Cochrane SRs to determine whether updating had improved these two quality dimensions. METHODS:We identified updated Cochrane SRs published in issue...

    journal_title:BMC medical research methodology

    pub_type: 杂志文章

    doi:10.1186/1471-2288-6-27

    authors: Shea B,Boers M,Grimshaw JM,Hamel C,Bouter LM

    更新日期:2006-06-13 00:00:00