What do we mean by validating a prognostic model?

Abstract:

:Prognostic models are used in medicine for investigating patient outcome in relation to patient and disease characteristics. Such models do not always work well in practice, so it is widely recommended that they need to be validated. The idea of validating a prognostic model is generally taken to mean establishing that it works satisfactorily for patients other than those from whose data it was derived. In this paper we examine what is meant by validation and review why it is necessary. We consider how to validate a model and suggest that it is desirable to consider two rather different aspects - statistical and clinical validity - and examine some general approaches to validation. We illustrate the issues using several case studies.

journal_name

Stat Med

journal_title

Statistics in medicine

authors

Altman DG,Royston P

doi

10.1002/(sici)1097-0258(20000229)19:4<453::aid-sim

subject

Has Abstract

pub_date

2000-02-29 00:00:00

pages

453-73

issue

4

eissn

0277-6715

issn

1097-0258

pii

10.1002/(SICI)1097-0258(20000229)19:4<453::AID-SIM

journal_volume

19

pub_type

杂志文章
  • Reclassification of predictions for uncovering subgroup specific improvement.

    abstract::Risk prediction models play an important role in prevention and treatment of several diseases. Models that are in clinical use are often refined and improved. In many instances, the most efficient way to improve a successful model is to identify subgroups for which there is a specific biological rationale for improvem...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.6077

    authors: Biswas S,Arun B,Parmigiani G

    更新日期:2014-05-20 00:00:00

  • Beta-binomial/Poisson regression models for repeated bivariate counts.

    abstract::We analyze data obtained from a study designed to evaluate training effects on the performance of certain motor activities of Parkinson's disease patients. Maximum likelihood methods were used to fit beta-binomial/Poisson regression models tailored to evaluate the effects of training on the numbers of attempted and su...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.3303

    authors: Lora MI,Singer JM

    更新日期:2008-07-30 00:00:00

  • On the estimation of total variability in assay validation.

    abstract::In the pharmaceutical industry, an assay method is considered validated if the accuracy and precision for an assay meet some acceptable limits. This paper discusses the assessment of assay precision in terms of the estimation of total variability of an assay from a one-way random effects model which is often considere...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.4780101006

    authors: Chow SC,Tse SK

    更新日期:1991-10-01 00:00:00

  • Changes in clinical trials mandated by the advent of meta-analysis.

    abstract::Service on the Data Monitoring Committee of the CPEP (Calcium for Pre-eclampsia Prevention) has led us to four conclusions about clinical trials which we should like to present to this gathering of biostatisticians for their reactions: (i) meta-analyses of the pertinent published trials of the same therapy should alwa...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/(SICI)1097-0258(19960630)15:12<1263::AID-S

    authors: Chalmers TC,Lau J

    更新日期:1996-06-30 00:00:00

  • A Markov mixed effect regression model for drug compliance.

    abstract::Patient compliance (adherence) with prescribed medication is often erratic, while clinical outcomes are causally linked to actual, rather than nominal medication dosage. We propose here a hierarchical Markov model for patient compliance. At the first stage, conditional upon individual random effects and a set of indiv...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/(sici)1097-0258(19981030)17:20<2313::aid-s

    authors: Girard P,Blaschke TF,Kastrissios H,Sheiner LB

    更新日期:1998-10-30 00:00:00

  • Simultaneous estimation of intrarater and interrater agreement for multiple raters under order restrictions for a binary trait.

    abstract::It is valuable in many studies to assess both intrarater and interrater agreement. Most measures of intrarater agreement do not adjust for unequal estimates of prevalence between the separate rating occasions for a given rater and measures of interrater agreement typically ignore data from the second set of assessment...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.1138

    authors: Lester Kirchner H,Lemke JH

    更新日期:2002-06-30 00:00:00

  • The power of focused tests to detect disease clustering.

    abstract::Statistical tests have been proposed for determining whether incident cases of adverse health effects are 'clustered' together. Several procedures, termed 'focused', specifically analyse disease surveillance data around pre-specified putative sources of environmental hazard. Little has been done to compare the perform...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.4780142103

    authors: Waller LA,Lawson AB

    更新日期:1995-11-15 00:00:00

  • Issues in applied statistics for public health bioterrorism surveillance using multiple data streams: research needs.

    abstract::The objective of this report is to provide a basis to inform decisions about priorities for developing statistical research initiatives in the field of public health surveillance for emerging threats. Rapid information system advances have created a vast opportunity of secondary data sources for information to enhance...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.2793

    authors: Rolka H,Burkom H,Cooper GF,Kulldorff M,Madigan D,Wong WK

    更新日期:2007-04-15 00:00:00

  • A linear exponent AR(1) family of correlation structures.

    abstract::In repeated measures settings, modeling the correlation pattern of the data can be immensely important for proper analyses. Accurate inference requires proper choice of the correlation model. Optimal efficiency of the estimation procedure demands a parsimonious parameterization of the correlation structure, with suffi...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.3928

    authors: Simpson SL,Edwards LJ,Muller KE,Sen PK,Styner MA

    更新日期:2010-07-30 00:00:00

  • A model for cross-over trials evaluating therapeutic preferences.

    abstract::A preference trial is a special form of cross-over trial where clinical conditions determine when patients change treatment, in a prescribed order. This can be modelled using a geometric distribution. The model can be simply fitted using standard logistic regression methodology. The procedure is applied to a trial stu...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/(SICI)1097-0258(19960229)15:4<443::AID-SIM

    authors: Lindsey JK,Jones B

    更新日期:1996-02-28 00:00:00

  • A spatial scan statistic for ordinal data.

    abstract::Spatial scan statistics are widely used for count data to detect geographical disease clusters of high or low incidence, mortality or prevalence and to evaluate their statistical significance. Some data are ordinal or continuous in nature, however, so that it is necessary to dichotomize the data to use a traditional s...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.2607

    authors: Jung I,Kulldorff M,Klassen AC

    更新日期:2007-03-30 00:00:00

  • Mixture distributions in multi-state modelling: some considerations in a study of psoriatic arthritis.

    abstract::In many studies, interest lies in determining whether members of the study population will undergo a particular event of interest. Such scenarios are often termed 'mover-stayer' scenarios, and interest lies in modelling two sub-populations of 'movers' (those who have a propensity to undergo the event of interest) and ...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.5529

    authors: O'Keeffe AG,Tom BD,Farewell VT

    更新日期:2013-02-20 00:00:00

  • Empirical evaluation of statistical models for counts or rates.

    abstract::We consider methods for selecting the joint specification of the mean and variance functions in statistical models for rates or counts. Based on analyses of diagnosis-specific hospital discharge rates in Michigan, we show that a Poisson model with an extra variance component for the systematic variation is superior to...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.4780100908

    authors: Wolfe RA,Petroni GR,McLaughlin CG,McMahon LF Jr

    更新日期:1991-09-01 00:00:00

  • Testing the equality of two Poisson means using the rate ratio.

    abstract::In this article, we investigate procedures for comparing two independent Poisson variates that are observed over unequal sampling frames (i.e. time intervals, populations, areas or any combination thereof). We consider two statistics (with and without the logarithmic transformation) for testing the equality of two Poi...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.1949

    authors: Ng HK,Tang ML

    更新日期:2005-03-30 00:00:00

  • Testing conditional independence in sets of I × J tables by means of moment and correlation score tests with application to HPV vaccine.

    abstract::A new testing approach is described for improving statistical tests of independence in sets of tables stratified on one or more relevant factors in case of categorical (nominal or ordinal) variables. Common tests of independence that exploit the ordinality of one of the variables use a restricted-alternative approach....

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.7006

    authors: Iannario M,Lang JB

    更新日期:2016-11-10 00:00:00

  • Likelihood-based methods for regression analysis with binary exposure status assessed by pooling.

    abstract::The need for resource-intensive laboratory assays to assess exposures in many epidemiologic studies provides ample motivation to consider study designs that incorporate pooled samples. In this paper, we consider the case in which specimens are combined for the purpose of determining the presence or absence of a pool-w...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.4426

    authors: Lyles RH,Tang L,Lin J,Zhang Z,Mukherjee B

    更新日期:2012-09-28 00:00:00

  • Discriminant analysis when all variables are ordered.

    abstract::Determination of the equation that relates an ordered dependent variable to ordered independent variables is sought. One solution, non-parametric discriminant analysis (NPD), involves obtaining the best monotonic step function by means of a computer search procedure. Although one can use alternative selection criteria...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.4780110804

    authors: Johnston B,Seshia SS

    更新日期:1992-06-15 00:00:00

  • Reflecting on "A Statistician in Medicine" in 2020.

    abstract::In this commentary, we revisit Sir Austin Bradford Hill's seminal Alfred Watson Memorial Lecture in 1962 through the eyes of two practicing biostatisticians of the current era. We summarize some eternal takeaway messages from Hill's lecture regarding observations and experiments translated through the modern lexicon o...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.8830

    authors: Dempsey W,Mukherjee B

    更新日期:2021-01-15 00:00:00

  • Emerging and recurrent issues in drug development.

    abstract::This paper reviews several emerging and recurrent issues relating to the drug development process. These emerging issues include changes to the FDA regulatory environment, internationalization of drug development, advances in computer technology and visualization tools, and efforts to incorporate meta-analysis methodo...

    journal_title:Statistics in medicine

    pub_type: 杂志文章,评审

    doi:10.1002/(sici)1097-0258(19990915/30)18:17/18<2301:

    authors: Anello C

    更新日期:1999-09-15 00:00:00

  • Methods for analyzing data from probabilistic linkage strategies based on partially identifying variables.

    abstract::In record linkage studies, unique identifiers are often not available, and therefore, the linkage procedure depends on combinations of partially identifying variables with low discriminating power. As a consequence, wrongly linked covariate and outcome pairs will be created and bias further analysis of the linked data...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.5498

    authors: Hof MH,Zwinderman AH

    更新日期:2012-12-30 00:00:00

  • Joint modeling of repeated multivariate cognitive measures and competing risks of dementia and death: a latent process and latent class approach.

    abstract::Joint models initially dedicated to a single longitudinal marker and a single time-to-event need to be extended to account for the rich longitudinal data of cohort studies. Multiple causes of clinical progression are indeed usually observed, and multiple longitudinal markers are collected when the true latent trait of...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.6731

    authors: Proust-Lima C,Dartigues JF,Jacqmin-Gadda H

    更新日期:2016-02-10 00:00:00

  • Methods for dose finding studies in cancer clinical trials: a review and results of a Monte Carlo study.

    abstract::We discuss some of the statistical approaches to the design and analysis of phase I clinical trials in cancer. An attempt is made to identify the issues, particular to this type of trial, that should be addressed by an appropriate methodology. A brief review of schemes currently in use is provided together with our vi...

    journal_title:Statistics in medicine

    pub_type: 杂志文章,评审

    doi:10.1002/sim.4780101104

    authors: O'Quigley J,Chevret S

    更新日期:1991-11-01 00:00:00

  • Inference for multimarker adaptive enrichment trials.

    abstract::Identification of treatment selection biomarkers has become very important in cancer drug development. Adaptive enrichment designs have been developed for situations where a unique treatment selection biomarker is not apparent based on the mechanism of action of the drug. With such designs, the eligibility rules may b...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.7422

    authors: Simon R,Simon N

    更新日期:2017-11-20 00:00:00

  • A frailty model for recurrent events during alternating restraint and non-restraint time periods.

    abstract::We consider recurrent events of the same type that occur during alternating restraint and non-restraint time periods. This research is motivated by a study on juvenile recidivism, where the probationers were followed for re-offenses during alternating placement periods and free-time periods. During the placement perio...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.7150

    authors: Li X,Chen Y,Li R

    更新日期:2017-02-20 00:00:00

  • Minimum sample size for developing a multivariable prediction model: PART II - binary and time-to-event outcomes.

    abstract::When designing a study to develop a new prediction model with binary or time-to-event outcomes, researchers should ensure their sample size is adequate in terms of the number of participants (n) and outcome events (E) relative to the number of predictor parameters (p) considered for inclusion. We propose that the mini...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.7992

    authors: Riley RD,Snell KI,Ensor J,Burke DL,Harrell FE Jr,Moons KG,Collins GS

    更新日期:2019-03-30 00:00:00

  • The Integrated Calibration Index (ICI) and related metrics for quantifying the calibration of logistic regression models.

    abstract::Assessing the calibration of methods for estimating the probability of the occurrence of a binary outcome is an important aspect of validating the performance of risk-prediction algorithms. Calibration commonly refers to the agreement between predicted and observed probabilities of the outcome. Graphical methods are a...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.8281

    authors: Austin PC,Steyerberg EW

    更新日期:2019-09-20 00:00:00

  • Recommended tests for association in 2 x 2 tables.

    abstract::The asymptotic Pearson's chi-squared test and Fisher's exact test have long been the most used for testing association in 2x2 tables. Unconditional tests preserve the significance level and generally are more powerful than Fisher's exact test for moderate to small samples, but previously were disadvantaged by being co...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.3531

    authors: Lydersen S,Fagerland MW,Laake P

    更新日期:2009-03-30 00:00:00

  • Determining the value of additional surrogate exposure data for improving the estimate of an odds ratio.

    abstract::We consider the design of both cohort and case-control studies in which an initial ('stage 1') sample of complete data on an error-free disease indicator (D), a correct ('gold standard') dichotomous exposure measurement (X) and an error-prone exposure measurement (Z) are available. We calculate the amount of additiona...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.4780142307

    authors: Dahm PF,Gail MH,Rosenberg PS,Pee D

    更新日期:1995-12-15 00:00:00

  • A Bayesian analysis of mixture structural equation models with non-ignorable missing responses and covariates.

    abstract::In behavioral, biomedical, and social-psychological sciences, it is common to encounter latent variables and heterogeneous data. Mixture structural equation models (SEMs) are very useful methods to analyze these kinds of data. Moreover, the presence of missing data, including both missing responses and missing covaria...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.3915

    authors: Cai JH,Song XY,Hser YI

    更新日期:2010-08-15 00:00:00

  • Power and sample size determination for group comparison of patient-reported outcomes using polytomous Rasch models.

    abstract::The analysis of patient-reported outcomes or other psychological traits can be realized using the Rasch measurement model. When the objective of a study is to compare groups of individuals, it is important, before the study, to define a sample size such that the group comparison test will attain a given power. The Ras...

    journal_title:Statistics in medicine

    pub_type: 杂志文章

    doi:10.1002/sim.6478

    authors: Hardouin JB,Blanchin M,Feddag ML,Le Néel T,Perrot B,Sébille V

    更新日期:2015-07-20 00:00:00