A score regression approach to assess calibration of continuous probabilistic predictions.

Abstract:

:Calibration, the statistical consistency of forecast distributions and the observations, is a central requirement for probabilistic predictions. Calibration of continuous forecasts is typically assessed using the probability integral transform histogram. In this article, we propose significance tests based on scoring rules to assess calibration of continuous predictive distributions. For an ideal normal forecast we derive the first two moments of two commonly used scoring rules: the logarithmic and the continuous ranked probability score. This naturally leads to the construction of two unconditional tests for normal predictions. More generally, we propose a novel score regression approach, where the individual scores are regressed on suitable functions of the predictive variance. This conditional approach is applicable even for certain nonnormal predictions based on the Dawid-Sebastiani score. Two case studies illustrate that the score regression approach has typically more power in detecting miscalibrated forecasts than the other approaches considered, including a recently proposed technique based on conditional exceedance probability curves.

journal_name

Biometrics

journal_title

Biometrics

authors

Held L,Rufibach K,Balabdaoui F

doi

10.1111/j.1541-0420.2010.01406.x

subject

Has Abstract

pub_date

2010-12-01 00:00:00

pages

1295-305

issue

4

eissn

0006-341X

issn

1541-0420

pii

BIOM1406

journal_volume

66

pub_type

杂志文章
  • Additive gamma frailty models with applications to competing risks in related individuals.

    abstract::Epidemiological studies of related individuals are often complicated by the fact that follow-up on the event type of interest is incomplete due to the occurrence of other events. We suggest a class of frailty models with cause-specific hazards for correlated competing events in related individuals. The frailties are b...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/biom.12326

    authors: Eriksson F,Scheike T

    更新日期:2015-09-01 00:00:00

  • Biased and unbiased estimation in longitudinal studies with informative visit processes.

    abstract::The availability of data in longitudinal studies is often driven by features of the characteristics being studied. For example, clinical databases are increasingly being used for research to address longitudinal questions. Because visit times in such data are often driven by patient characteristics that may be related...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/biom.12501

    authors: McCulloch CE,Neuhaus JM,Olin RL

    更新日期:2016-12-01 00:00:00

  • Comments about Joint Modeling of Cluster Size and Binary and Continuous Subunit-Specific Outcomes.

    abstract::In longitudinal studies and in clustered situations often binary and continuous response variables are observed and need to be modeled together. In a recent publication Dunson, Chen, and Harry (2003, Biometrics 59, 521-530) (DCH) propose a Bayesian approach for joint modeling of cluster size and binary and continuous ...

    journal_title:Biometrics

    pub_type: 评论,杂志文章

    doi:10.1111/j.1541-020X.2005.00409_1.x

    authors: Gueorguieva RV

    更新日期:2005-09-01 00:00:00

  • A Bayesian approach to the analysis of quantal bioassay studies using nonparametric mixture models.

    abstract::We develop a Bayesian nonparametric mixture modeling framework for quantal bioassay settings. The approach is built upon modeling dose-dependent response distributions. We adopt a structured nonparametric prior mixture model, which induces a monotonicity restriction for the dose-response curve. Particular emphasis is ...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/biom.12120

    authors: Fronczyk K,Kottas A

    更新日期:2014-03-01 00:00:00

  • Confidence intervals and P-values for meta-analysis with publication bias.

    abstract::We study publication bias in meta-analysis by supposing there is a population (y, sigma) of studies which give treatment effect estimates y approximately N(theta, sigma(2)). A selection function describes the probability that each study is selected for review. The overall estimate of theta depends on the studies selec...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.1541-0420.2006.00705.x

    authors: Henmi M,Copas JB,Eguchi S

    更新日期:2007-06-01 00:00:00

  • A novel statistical method for modeling covariate effects in bisulfite sequencing derived measures of DNA methylation.

    abstract::Identifying disease-associated changes in DNA methylation can help us gain a better understanding of disease etiology. Bisulfite sequencing allows the generation of high-throughput methylation profiles at single-base resolution of DNA. However, optimally modeling and analyzing these sparse and discrete sequencing data...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/biom.13307

    authors: Zhao K,Oualkacha K,Lakhal-Chaieb L,Labbe A,Klein K,Ciampi A,Hudson M,Colmegna I,Pastinen T,Zhang T,Daley D,Greenwood CMT

    更新日期:2020-05-21 00:00:00

  • Multiple imputation for model checking: completed-data plots with missing and latent data.

    abstract::In problems with missing or latent data, a standard approach is to first impute the unobserved data, then perform all statistical analyses on the completed dataset--corresponding to the observed data and imputed unobserved data--using standard procedures for complete-data inference. Here, we extend this approach to mo...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.0006-341X.2005.031010.x

    authors: Gelman A,Van Mechelen I,Verbeke G,Heitjan DF,Meulders M

    更新日期:2005-03-01 00:00:00

  • Estimation of individual genetic effects from binary observations on relatives applied to a family history of respiratory illnesses and chronic lung disease of newborns.

    abstract::This paper considers methods for estimating the relationship between a binary response Y and the genetic effects responsible for a second binary trait Z. The responses Y are observed only for target individuals, and the responses Z are observed only for the relatives of these targets. The analysis consists of two part...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.0006-341x.2000.00808.x

    authors: Houwing-Duistermaat JJ,van Houwelingen HC,de Winter JP

    更新日期:2000-09-01 00:00:00

  • Sample size methods for estimating HIV incidence from cross-sectional surveys.

    abstract::Understanding HIV incidence, the rate at which new infections occur in populations, is critical for tracking and surveillance of the epidemic. In this article, we derive methods for determining sample sizes for cross-sectional surveys to estimate incidence with sufficient precision. We further show how to specify samp...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/biom.12336

    authors: Konikoff J,Brookmeyer R

    更新日期:2015-12-01 00:00:00

  • Adjusted regression trend test for a multicenter clinical trial.

    abstract::Studies using a series of increasing doses of a compound, including a zero dose control, are often conducted to study the effect of the compound on the response of interest. For a one-way design, Tukey et al. (1985, Biometrics 41, 295-301) suggested assessing trend by examining the slopes of regression lines under ari...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.0006-341x.1999.00460.x

    authors: Quan H,Capizzi T

    更新日期:1999-06-01 00:00:00

  • On assessing interrater agreement for multiple attribute responses.

    abstract::New methods are developed for assessing the extent of interrater agreement when each unit to be rated is characterized by a (possibly empty) subset of a specified set of distinct nominal attributes. For such multiple attribute response data, a two-rater concordance statistic is derived, and associated statistical infe...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:

    authors: Kupper LL,Hafner KB

    更新日期:1989-09-01 00:00:00

  • Testing for Hardy-Weinberg equilibrium.

    abstract::The class of admissible tests for Hardy-Weinberg equilibrium in a multi-allelic system is characterized. The standard goodness-of-fit chi-square tests is shown to be admissible for systems of two or more alleles. The conditional probability distribution required to determine the exact significance level of this test i...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:

    authors: Ledwina T,Gnot S

    更新日期:1980-03-01 00:00:00

  • Two-stage designs for gene-disease association studies with sample size constraints.

    abstract::Gene-disease association studies based on case-control designs may often be used to identify candidate polymorphisms (markers) conferring disease risk. If a large number of markers are studied, genotyping all markers on all samples is inefficient in resource utilization. Here, we propose an alternative two-stage metho...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.0006-341X.2004.00207.x

    authors: Satagopan JM,Venkatraman ES,Begg CB

    更新日期:2004-09-01 00:00:00

  • Statistical significance for hierarchical clustering.

    abstract::Cluster analysis has proved to be an invaluable tool for the exploratory and unsupervised analysis of high-dimensional datasets. Among methods for clustering, hierarchical approaches have enjoyed substantial popularity in genomics and other fields for their ability to simultaneously uncover multiple layers of clusteri...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/biom.12647

    authors: Kimes PK,Liu Y,Neil Hayes D,Marron JS

    更新日期:2017-09-01 00:00:00

  • Bootstrap confidence intervals for adaptive cluster sampling.

    abstract::Consider a collection of spatially clustered objects where the clusters are geographically rare. Of interest is estimation of the total number of objects on the site from a sample of plots of equal size. Under these spatial conditions, adaptive cluster sampling of plots is generally useful in improving efficiency in e...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.0006-341x.2000.00503.x

    authors: Christman MC,Pontius JS

    更新日期:2000-06-01 00:00:00

  • Modeling longitudinal data with nonparametric multiplicative random effects jointly with survival data.

    abstract::In clinical studies, longitudinal biomarkers are often used to monitor disease progression and failure time. Joint modeling of longitudinal and survival data has certain advantages and has emerged as an effective way to mutually enhance information. Typically, a parametric longitudinal model is assumed to facilitate t...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.1541-0420.2007.00896.x

    authors: Ding J,Wang JL

    更新日期:2008-06-01 00:00:00

  • Robust tests for treatment effects based on censored recurrent event data observed over multiple periods.

    abstract::We derive semiparametric methods for estimating and testing treatment effects when censored recurrent event data are available over multiple periods. These methods are based on estimating functions motivated by a working "mixed-Poisson" assumption under which conditioning can eliminate subject-specific random effects....

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.1541-0420.2005.00357.x

    authors: Cook RJ,Wei W,Yi GY

    更新日期:2005-09-01 00:00:00

  • Capture-recapture estimation using finite mixtures of arbitrary dimension.

    abstract::Reversible jump Markov chain Monte Carlo (RJMCMC) methods are used to fit Bayesian capture-recapture models incorporating heterogeneity in individuals and samples. Heterogeneity in capture probabilities comes from finite mixtures and/or fixed sample effects allowing for interactions. Estimation by RJMCMC allows automa...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.1541-0420.2009.01289.x

    authors: Arnold R,Hayakawa Y,Yip P

    更新日期:2010-06-01 00:00:00

  • A model-based approach for making ecological inference from distance sampling data.

    abstract::We consider a fully model-based approach for the analysis of distance sampling data. Distance sampling has been widely used to estimate abundance (or density) of animals or plants in a spatially explicit study area. There is, however, no readily available method of making statistical inference on the relationships bet...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.1541-0420.2009.01265.x

    authors: Johnson DS,Laake JL,Ver Hoef JM

    更新日期:2010-03-01 00:00:00

  • A unified parametric regression model for recapture studies with random removals in continuous time.

    abstract::Conditional likelihood based on counting processes are combined with a Horvitz-Thompson estimator to yield a population size estimator that is more efficient than the existing ones. Random removals are allowed in the recapturing process. Simulation studies are shown to assess the performance of the proposed estimators...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.0006-341x.2002.00192.x

    authors: Yip PS,Wang Y

    更新日期:2002-03-01 00:00:00

  • Maximum likelihood estimation for N-mixture models.

    abstract::The focus of this article is on the nature of the likelihood associated with N-mixture models for repeated count data. It is shown that the infinite sum embedded in the likelihood associated with the Poisson mixing distribution can be expressed in terms of a hypergeometric function and, thence, in closed form. The res...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/biom.12521

    authors: Haines LM

    更新日期:2016-12-01 00:00:00

  • The estimation of maternal genetic variances.

    abstract::The estimation of maternal genetic variances by a multivariate maximum likelihood method is discussed. As an illustration the method is applied to data on Tribolium using a model based on partitioning the maternal genetic effect into additive and dominance components. An alternative model due to Falconer (1965) is als...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:

    authors: Thompson R

    更新日期:1976-12-01 00:00:00

  • Nonparametric discrete survival function estimation with uncertain endpoints using an internal validation subsample.

    abstract::When a true survival endpoint cannot be assessed for some subjects, an alternative endpoint that measures the true endpoint with error may be collected, which often occurs when obtaining the true endpoint is too invasive or costly. We develop an estimated likelihood function for the situation where we have both uncert...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/biom.12316

    authors: Zee J,Xie SX

    更新日期:2015-09-01 00:00:00

  • Doubly robust estimator for net survival rate in analyses of cancer registry data.

    abstract::Cancer population studies based on cancer registry databases are widely conducted to address various research questions. In general, cancer registry databases do not collect information on cause of death. The net survival rate is defined as the survival rate if a subject would not die for any causes other than cancer....

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/biom.12568

    authors: Komukai S,Hattori S

    更新日期:2017-03-01 00:00:00

  • Semiparametric maximum likelihood for nonlinear regression with measurement errors.

    abstract::This article demonstrates semiparametric maximum likelihood estimation of a nonlinear growth model for fish lengths using imprecisely measured ages. Data on the species corvina reina, found in the Gulf of Nicoya, Costa Rica, consist of lengths and imprecise ages for 168 fish and precise ages for a subset of 16 fish. T...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.0006-341x.2002.00448.x

    authors: Suh EY,Schafer DW

    更新日期:2002-06-01 00:00:00

  • Bayesian influence measures for joint models for longitudinal and survival data.

    abstract::This article develops a variety of influence measures for carrying out perturbation (or sensitivity) analysis to joint models of longitudinal and survival data (JMLS) in Bayesian analysis. A perturbation model is introduced to characterize individual and global perturbations to the three components of a Bayesian model...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.1541-0420.2012.01745.x

    authors: Zhu H,Ibrahim JG,Chi YY,Tang N

    更新日期:2012-09-01 00:00:00

  • Breeding return times and abundance in capture-recapture models.

    abstract::For many long-lived animal species, individuals do not breed every year, and are often not accessible during non-breeding periods. Individuals exhibit site fidelity if they return to the same breeding colony or spawning ground when they breed. If capture and recapture is only possible at the breeding site, temporary e...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/biom.12094

    authors: Pledger S,Baker E,Scribner K

    更新日期:2013-12-01 00:00:00

  • Numerical discretization-based estimation methods for ordinary differential equation models via penalized spline smoothing with applications in biomedical research.

    abstract::Differential equations are extensively used for modeling dynamics of physical processes in many scientific fields such as engineering, physics, and biomedical sciences. Parameter estimation of differential equation models is a challenging problem because of high computational cost and high-dimensional parameter space....

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.1541-0420.2012.01752.x

    authors: Wu H,Xue H,Kumar A

    更新日期:2012-06-01 00:00:00

  • A moving blocks empirical likelihood method for longitudinal data.

    abstract::In the analysis of longitudinal or panel data, neglecting the serial correlations among the repeated measurements within subjects may lead to inefficient inference. In particular, when the number of repeated measurements is large, it may be desirable to model the serial correlations more generally. An appealing approa...

    journal_title:Biometrics

    pub_type: 杂志文章,随机对照试验

    doi:10.1111/biom.12317

    authors: Qiu J,Wu L

    更新日期:2015-09-01 00:00:00

  • Multiple imputation methods for estimating regression coefficients in the competing risks model with missing cause of failure.

    abstract::We propose a method to estimate the regression coefficients in a competing risks model where the cause-specific hazard for the cause of interest is related to covariates through a proportional hazards relationship and when cause of failure is missing for some individuals. We use multiple imputation procedures to imput...

    journal_title:Biometrics

    pub_type: 杂志文章

    doi:10.1111/j.0006-341x.2001.01191.x

    authors: Lu K,Tsiatis AA

    更新日期:2001-12-01 00:00:00