A simulation study into the performance of "optimal" diagnostic thresholds in the population:"Large" effect sizes are not enough.

Abstract:

OBJECTIVES:Many diagnostic studies are aimed at defining "optimal" thresholds. Here, we evaluate the performance of empirically defined optimal thresholds (1) in the sample in which they were defined and (2) in the population from which the sample was drawn. STUDY DESIGN AND SETTING:We simulated test results for 120,000 samples varying the number of people without a disease (n between 20 and 500), number of people with a disease (m between 20 and 500), the magnitude of the difference between group means [effect size (ES) between 0.5 and 4], and distributions (normal and log-normal). The thresholds associated with the maximal Youden index were defined as optimal. Performance was defined as the percentage of correct classifications in the sample and when applied to the whole population. RESULTS:At the population level, the thresholds defined for the four ESs (0.5, 0.8, 2, and 4) yielded a median of 59%, 65%, 83%, and 97% correct classifications, respectively. At the sample level, the samples with similar characteristics yielded widely varying estimates of the performance that were systematically higher than at the population level. CONCLUSION:Researchers need to be careful defining cut points for mean differences that are traditionally considered "large" (ES = 0.8). The diagnostic utility of optimal thresholds needs to be assessed in prospective studies.

journal_name

J Clin Epidemiol

authors

Hirschfeld G,do Brasil PE

doi

10.1016/j.jclinepi.2013.07.018

subject

Has Abstract

pub_date

2014-04-01 00:00:00

pages

449-53

issue

4

eissn

0895-4356

issn

1878-5921

pii

S0895-4356(13)00348-X

journal_volume

67

pub_type

杂志文章
  • Development and validation of a multivariate predictor of mortality in very low birth weight.

    abstract::Accurate prognosis is critical to the design of all prospective research aimed at improving survival. Predictions based on birth weight, gestational age, or any other single variable, fail to take into account the potentially important contribution of other factors. In order to develop a practical and accurate multiva...

    journal_title:Journal of clinical epidemiology

    pub_type: 临床试验,杂志文章

    doi:10.1016/0895-4356(88)90079-0

    authors: Ales KL,Frayer W,Hawks G,Auld PM,Druzin ML

    更新日期:1988-01-01 00:00:00

  • Measuring health status: what are the necessary measurement properties?

    abstract::The different measurement properties necessary for instruments whose goal is to detect differences between subjects at a single point in time (discriminative instruments) and those whose goal is to detect longitudinal change within subjects (evaluative instruments) is becoming increasingly recognized. Up to now, requi...

    journal_title:Journal of clinical epidemiology

    pub_type: 评论,杂志文章

    doi:10.1016/0895-4356(92)90194-r

    authors: Guyatt GH,Kirshner B,Jaeschke R

    更新日期:1992-12-01 00:00:00

  • Using ordinal logistic regression to estimate the likelihood of colorectal neoplasia.

    abstract::The utility of ordinal logistic regression in the prediction of colorectal neoplasia was demonstrated in a group of 461 consecutive patients undergoing colonoscopy in a community practice. One hundred twenty-nine patients had adenomatous polyps and 34 had colorectal adenocarcinoma. An ordinal logistic regression model...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(91)90159-7

    authors: Brazer SR,Pancotto FS,Long TT 3rd,Harrell FE Jr,Lee KL,Tyor MP,Pryor DB

    更新日期:1991-01-01 00:00:00

  • A case-control study found that low albumin and smoking were associated with aortic dissection.

    abstract:OBJECTIVE:Hypertension is a risk factor for aortic dissection. Besides hypertension, associated factors for incidence of aortic dissection are not clear. The aim of this study was to examine whether other factors exist by an epidemiologic case-control approach. STUDY DESIGN AND SETTING:240 consecutive cases suffering ...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2003.09.010

    authors: Takeuchi T,Adachi H,Ohuchida M,Nakamura T,Satoh A,Jacobs DR Jr,Imaizumi T

    更新日期:2004-04-01 00:00:00

  • Time-related issues with application to health gains and losses.

    abstract::Time-related aspects of health have attracted increasing interest, and it has become evident that many medical situations concern the exchange of present-day costs for future benefits. Traditional decision analytic paradigms weigh the probability of outcomes and the value of outcomes. Such analyses are incomplete if t...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(02)00447-x

    authors: Ortendahl M,Fries JF

    更新日期:2002-09-01 00:00:00

  • Redundant meta-analyses are common in genetic epidemiology.

    abstract:OBJECTIVES:The massive growth in the publication of meta-analyses may cause redundancy and wasted efforts. We performed a metaepidemiologic study to evaluate the extent of potential redundancy in published meta-analyses in genetic epidemiology. STUDY DESIGN AND SETTING:Using a sample of 38 index meta-analyses of genet...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,评审

    doi:10.1016/j.jclinepi.2020.05.035

    authors: Sigurdson MK,Khoury MJ,Ioannidis JPA

    更新日期:2020-11-01 00:00:00

  • Factors associated with errors in death certificate completion. A national study in Taiwan.

    abstract::To identify characteristics of certifying physicians and the deceased that are associated with errors in death certificate completion in Taiwan, we retrospectively reviewed 4123 systematically sampled death certificates issued in 1994. Multivariate analyses were used to assess the associations of various characteristi...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(00)00299-7

    authors: Lu TH,Shau WY,Shih TP,Lee MC,Chou MC,Lin CK

    更新日期:2001-03-01 00:00:00

  • Using reference values to define disease based on the lower limit of normal biased the population attributable fraction, but not the population excess risk: the example of chronic airflow obstruction.

    abstract:BACKGROUND:The impact of disease on population health is most commonly estimated by the population attributable fraction (PAF), or less commonly by the excess risk, an alternative measure that estimates the absolute risk of disease in the population that can be ascribed to the exposure. Using chronic airflow obstructio...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2017.10.020

    authors: Burney P,Minelli C

    更新日期:2018-01-01 00:00:00

  • A structured review of randomized controlled trials of weight loss showed little improvement in health-related quality of life.

    abstract:OBJECTIVE:To estimate the effect of weight-loss interventions on health-related quality of life (HrQoL) in randomized controlled trials (RCTs); to conduct a meta-analysis of weight-loss treatment on depressive symptoms; and, to examine methodological and presentation issues that compromise study validity. STUDY DESIGN...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,meta分析,评审

    doi:10.1016/j.jclinepi.2004.10.015

    authors: Maciejewski ML,Patrick DL,Williamson DF

    更新日期:2005-06-01 00:00:00

  • Female specialists were more likely to respond to a postal questionnaire about drug treatments for Alzheimer disease.

    abstract:OBJECTIVE:To assess demographic predictors of response for specialists who were mailed a postal questionnaire on drug treatments for Alzheimer disease. STUDY DESIGN AND SETTING:The questionnaire was sent to 317 specialists in Quebec, Canada. Demographic variables included specialty, urban/rural practice, language, sex...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2003.11.008

    authors: Oremus M,Wolfson C

    更新日期:2004-06-01 00:00:00

  • Measuring importance of outcomes to patients: a cross-sectional survey for the German anal cancer guideline.

    abstract:OBJECTIVE:We aimed to generate evidence on patients' values and preferences to inform the development of the German national Evidence-based Anal Cancer Guideline. STUDY DESIGN AND SETTING:We developed a list of health outcomes based on a systematic search. We then asked anal cancer patients and experts of the guidelin...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2020.09.026

    authors: Werner RN,Gaskins M,Dressler C,Nast A,Schaefer C,Aigner F,Siegel R

    更新日期:2021-01-01 00:00:00

  • US Food and Drug Administration documents can provide unpublished evidence relevant to systematic reviews.

    abstract:OBJECTIVES:A key systematic review (SR) methodology is comprehensive searching. The Drug Effectiveness Review Project (DERP) SRs search US Food and Drug Administration (FDA) documents to identify unpublished evidence. This study evaluates the success of those efforts. STUDY DESIGN AND SETTING:We examined DERP reports ...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,评审

    doi:10.1016/j.jclinepi.2013.05.006

    authors: McDonagh MS,Peterson K,Balshem H,Helfand M

    更新日期:2013-10-01 00:00:00

  • Using postal randomization to replace telephone randomization had no significant effect on recruitment of patients.

    abstract:OBJECTIVE:To test the effect of postal randomization on recruitment of patients into a randomized trial in primary care. STUDY DESIGN AND SETTING:General practices used a telephone service to randomize patients in our trial. Delays in the start of recruitment at some sites led us to modify the randomization procedure....

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.jclinepi.2007.04.003

    authors: Brealey SD,Atwell C,Bryan S,Coulton S,Cox H,Cross B,Fylan F,Garratt A,Gilbert FJ,Gillan MG,Hendry M,Hood K,Houston H,King D,Morton V,Orchard J,Robling M,Russell IT,Torgerson D,Wadsworth V,Wilkinson C

    更新日期:2007-10-01 00:00:00

  • A critical review of methods for the assessment of patient-level interactions in individual participant data meta-analysis of randomized trials, and guidance for practitioners.

    abstract:OBJECTIVE:Treatments may be more effective in some patients than others, and individual participant data (IPD) meta-analysis of randomized trials provides perhaps the best method of investigating treatment-covariate interactions. Various methods are used; we provide a comprehensive critique and develop guidance on meth...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章,评审

    doi:10.1016/j.jclinepi.2010.11.016

    authors: Fisher DJ,Copas AJ,Tierney JF,Parmar MK

    更新日期:2011-09-01 00:00:00

  • Text message reminders to improve questionnaire response rates.

    abstract:OBJECTIVES:To assess whether short message service (SMS) text messages sent as prenotification or postnotification reminders improve questionnaire response rates in randomized controlled trial (RCT) follow-up. STUDY DESIGN AND SETTING:Three "trials within a trial" assessed text message notifications sent before or aft...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2016.05.011

    authors: Keding A,Brabyn S,MacPherson H,Richmond SJ,Torgerson DJ

    更新日期:2016-11-01 00:00:00

  • Measuring quality of life in the frail elderly.

    abstract::This study was carried out to develop and test the Geriatric Quality of Life Questionnaire (GQLQ), a health-related qualify of life (HRQL) questionnaire designed for the frail elderly. One hundred patients were asked to identify areas of their daily lives affected by their health. The GQLQ, developed on the basis of t...

    journal_title:Journal of clinical epidemiology

    pub_type: 临床试验,杂志文章,随机对照试验

    doi:10.1016/0895-4356(93)90143-o

    authors: Guyatt GH,Eagle DJ,Sackett B,Willan A,Griffith L,McIlroy W,Patterson CJ,Turpie I

    更新日期:1993-12-01 00:00:00

  • Coronary events in Khartoum, Sudan.

    abstract::The aim of the study was to determine coronary events in a defined Sudanese population. All coronary events occurring in Khartoum, the capital of Sudan, were registered during the calendar year 1989. The cold pursuit method of identifying coronary events by surveillance of routinely collected data was adopted. A total...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(96)00116-3

    authors: Khalil SI,Ibrahim-Khalil M,el Hag M,el Shafie S,Mahgoub F,Ali EN

    更新日期:1996-09-01 00:00:00

  • The qualification period.

    abstract::For randomized trials that do not require prompt onset of therapy, a pre-admission preparation, called the qualification period, can deal with certain customary difficulties in design, analysis, and ethics. For the patients' baseline state, the qualification period can be used to "wash out" effects of previous treatme...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(91)90208-q

    authors: Knipschild P,Leffers P,Feinstein AR

    更新日期:1991-01-01 00:00:00

  • Does a history of hypertension influence the prognosis among diabetics with acute chest pain?

    abstract::The purpose of this study was to relate the 1-year risk of death and development of acute myocardial infarction among diabetics with acute chest pain to whether they had a history of hypertension or not. All patients with a history of diabetes mellitus who, during 21 months, were admitted to the Emergency Room in Sahl...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(94)90174-0

    authors: Karlson BW,Herlitz J,Hjalmarson A

    更新日期:1994-07-01 00:00:00

  • Screening for coronary heart disease in elderly men based on current and past cholesterol levels.

    abstract::Efficient use of cholesterol measurements to screen for coronary heart disease in the elderly is not well defined. The purpose of this report is to examine such screening based on national guidelines in a sample of older men. Since relations between cholesterol and coronary heart disease are better established in thos...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(99)00130-4

    authors: Hakim AA,Curb JD,Burchfiel CM,Rodriguez BL,Sharp DS,Yano K,Abbott RD

    更新日期:1999-12-01 00:00:00

  • Multilevel modeling was a convenient alternative to common regression designs in longitudinal suicide research.

    abstract:BACKGROUND AND OBJECTIVE:The prospective investigation of repetitive nonfatal suicidal behavior is associated with two methodological problems. Due to the commonly used definitions of nonfatal suicidal behavior, clinical samples usually consist of patients with a considerable between-person variability. Second, repeate...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2005.10.010

    authors: Antretter E,Dunkel D,Osvath P,Voros V,Fekete S,Haring C

    更新日期:2006-06-01 00:00:00

  • Quantifying the potential benefit of CA 125 screening for ovarian cancer.

    abstract::We develop a stochastic model of screening for ovarian cancer with serum levels of the CA 125 radioimmunoassay, a tumor-specific marker. The natural history of the disease is constructed using a four stage model characterized by a multivariate distribution for duration in each stage. Preserving its important features,...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/0895-4356(91)90075-k

    authors: Skates SJ,Singer DE

    更新日期:1991-01-01 00:00:00

  • Some methodologic issues in analyzing data from a randomized adolescent tobacco and alcohol use prevention trial.

    abstract::Three issues concerning the design and analysis of randomized behavioral intervention studies are illustrated and discussed within the framework of a tobacco and alcohol prevention trial among migrant Latino adolescents. The first issue arises when subjects are randomized in clusters rather than individually. Because ...

    journal_title:Journal of clinical epidemiology

    pub_type: 临床试验,杂志文章,多中心研究,随机对照试验

    doi:10.1016/s0895-4356(03)00012-x

    authors: Slymen DJ,Elder JP,Litrownik AJ,Ayala GX,Campbell NR

    更新日期:2003-04-01 00:00:00

  • Designing pragmatic trials-what can we learn from lessons learned?

    abstract::Pragmatic trials aim to inform clinical decision making by measuring the effect of a treatment in clinical practice. The purpose of the PRECIS-2 tool is to support in designing a truly pragmatic trial. We comment on a study by Forbes et al., who assessed the applicability of the PRECIS-2 tool. The tool will prove part...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2017.06.006

    authors: Groenwold RHH,Dekkers OM

    更新日期:2017-10-01 00:00:00

  • Trade-off between benefit and harm is crucial in health screening recommendations. Part I: general principles.

    abstract::Health screening is defined as the use of a test or a series of tests to detect unrecognized health risks or preclinical disease in apparently healthy populations to permit prevention and timely intervention. A health screening strategy consists of the sequence of a screening test, confirmatory test(s), and finally, t...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2010.09.009

    authors: Dans LF,Silvestre MA,Dans AL

    更新日期:2011-03-01 00:00:00

  • Calibration of clinical prediction rules does not just assess bias.

    abstract:OBJECTIVES:Calibration is often thought to assess the bias of a clinical prediction rule. In particular, if the rule is based on a linear logistic model, it is often assumed that an overestimation of all coefficients results in a calibration slope less than 1 and an underestimation in a slope larger than 1. STUDY DESI...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2013.06.003

    authors: Vach W

    更新日期:2013-11-01 00:00:00

  • Conflict of interest as ethical shorthand: understanding the range and nature of "non-financial conflict of interest" in biomedicine.

    abstract:OBJECTIVES:The aim of the study was to identify the range of issues labeled as "non-financial conflicts of interest" in biomedicine, articulate the associated concerns, and analyze the implications of defining these issues as conflicts of interest. STUDY DESIGN AND SETTING:This was a qualitative study, triangulating d...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2019.12.014

    authors: Grundy Q,Mayes C,Holloway K,Mazzarello S,Thombs BD,Bero L

    更新日期:2020-04-01 00:00:00

  • Colorectal cancer deaths as determined by expert committee and from death certificate: a comparison. The Minnesota Study.

    abstract::For deaths during the first 13 years of follow-up of the Minnesota Colon Cancer Control Study, an expert committee using numerous medical documents, and a nosologist using only the death certificate independently determined whether colorectal cancer caused the death and, if not, whether the disease was present at deat...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/s0895-4356(99)00016-5

    authors: Ederer F,Geisser MS,Mongin SJ,Church TR,Mandel JS

    更新日期:1999-05-01 00:00:00

  • In network meta-analysis, most of the information comes from indirect evidence: empirical study.

    abstract:OBJECTIVES:Network meta-analysis (NMA) may produce more precise estimates of treatment effects than pairwise meta-analysis. We examined the relative contribution of network paths of different lengths to estimates of treatment effects. STUDY DESIGN AND SETTING:We analyzed 213 published NMAs. We categorized network shap...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2020.04.009

    authors: Papakonstantinou T,Nikolakopoulou A,Egger M,Salanti G

    更新日期:2020-08-01 00:00:00

  • Including papers in languages other than English in systematic reviews: important, feasible, yet often omitted.

    abstract::A truly international and systematic review (SR) identifies and synthesizes relevant evidence regardless of the geographical provenance or language of publication. Despite recommendations, international SRs continue to exclude papers in languages other than English (LOE) at searching, screening, and analysis phases ei...

    journal_title:Journal of clinical epidemiology

    pub_type: 杂志文章

    doi:10.1016/j.jclinepi.2019.03.004

    authors: Walpole SC

    更新日期:2019-07-01 00:00:00