A method to increase the power of multiple testing procedures through sample splitting.

Abstract:

:Consider the standard multiple testing problem where many hypotheses are to be tested, each hypothesis is associated with a test statistic, and large test statistics provide evidence against the null hypotheses. One proposal to provide probabilistic control of Type-I errors is the use of procedures ensuring that the expected number of false positives does not exceed a user-supplied threshold. Among such multiple testing procedures, we derive the most powerful method, meaning the test statistic cutoffs that maximize the expected number of true positives. Unfortunately, these optimal cutoffs depend on the true unknown data generating distribution, so could never be used in a practical setting. We instead consider splitting the sample so that the optimal cutoffs are estimated from a portion of the data, and then testing on the remaining data using these estimated cutoffs. When the null distributions for all test statistics are the same, the obvious way to control the expected number of false positives would be to use a common cutoff for all tests. In this work, we consider the common cutoff method as a benchmark multiple testing procedure. We show that in certain circumstances the use of estimated optimal cutoffs via sample splitting can dramatically outperform this benchmark method, resulting in increased true discoveries, while retaining Type-I error control. This paper is an updated version of the work presented in Rubin et al. (2005), later expanded upon by Wasserman and Roeder (2006).

authors

Rubin D,Dudoit S,van der Laan M

doi

10.2202/1544-6115.1148

subject

Has Abstract

pub_date

2006-01-01 00:00:00

pages

Article19

eissn

2194-6302

issn

1544-6115

journal_volume

5

pub_type

杂志文章
  • Fully Bayesian mixture model for differential gene expression: simulations and model checks.

    abstract::We present a Bayesian hierarchical model for detecting differentially expressed genes using a mixture prior on the parameters representing differential effects. We formulate an easily interpretable 3-component mixture to classify genes as over-expressed, under-expressed and non-differentially expressed, and model gene...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1314

    authors: Lewin A,Bochkina N,Richardson S

    更新日期:2007-01-01 00:00:00

  • Genetic linkage analysis in the presence of germline mosaicism.

    abstract::Germline mosaicism is a genetic condition in which some germ cells of an individual contain a mutation. This condition violates the assumptions underlying classic genetic analysis and may lead to failure of such analysis. In this work we extend the statistical model used for genetic linkage analysis in order to incorp...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1709

    authors: Weissbrod O,Geiger D

    更新日期:2011-10-04 00:00:00

  • Sparse inverse of covariance matrix of QTL effects with incomplete marker data.

    abstract::Gametic models for fitting breeding values at QTL as random effects in outbred populations have become popular because they require few assumptions about the number and distribution of QTL alleles segregating. The covariance matrix of the gametic effects has an inverse that is sparse and can be constructed rapidly by ...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1048

    authors: Thallman RM,Hanford KJ,Kachman SD,Van Vleck LD

    更新日期:2004-01-01 00:00:00

  • Addressing the shortcomings of three recent Bayesian methods for detecting interspecific recombination in DNA sequence alignments.

    abstract::We address a potential shortcoming of three probabilistic models for detecting interspecific recombination in DNA sequence alignments: the multiple change-point model (MCP) of Suchard et al. (2003), the dual multiple change-point model (DMCP) of Minin et al. (2005), and the phylogenetic factorial hidden Markov model (...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1399

    authors: Husmeier D,Mantzaris AV

    更新日期:2008-01-01 00:00:00

  • Empirical bayes microarray ANOVA and grouping cell lines by equal expression levels.

    abstract::In the exploding field of gene expression techniques such as DNA microarrays, there are still few general probabilistic methods for analysis of variance. Linear models and ANOVA are heavily used tools in many other disciplines of scientific research. The usual F-statistic is unsatisfactory for microarray data, which e...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1125

    authors: Lönnstedt I,Rimini R,Nilsson P

    更新日期:2005-01-01 00:00:00

  • Approximate maximum likelihood estimation for population genetic inference.

    abstract::In many population genetic problems, parameter estimation is obstructed by an intractable likelihood function. Therefore, approximate estimation methods have been developed, and with growing computational power, sampling-based methods became popular. However, these methods such as Approximate Bayesian Computation (ABC...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2017-0016

    authors: Bertl J,Ewing G,Kosiol C,Futschik A

    更新日期:2017-11-27 00:00:00

  • Weighted-LASSO for structured network inference from time course data.

    abstract::We present a weighted-LASSO method to infer the parameters of a first-order vector auto-regressive model that describes time course expression data generated by directed gene-to-gene regulation networks. These networks are assumed to own prior internal structures of connectivity which drive the inference method. This ...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1519

    authors: Charbonnier C,Chiquet J,Ambroise C

    更新日期:2010-01-01 00:00:00

  • Combining dependent p-values by gamma distributions.

    abstract::Combining correlated p-values from multiple hypothesis testing is a most frequently used method for integrating information in genetic and genomic data analysis. However, most existing methods for combining independent p-values from individual component problems into a single unified p-value are unsuitable for the cor...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2019-0057

    authors: Chien LC

    更新日期:2020-11-06 00:00:00

  • Approximating the variance of the conditional probability of the state of a hidden Markov model.

    abstract::In a hidden Markov model, one "estimates" the state of the hidden Markov chain at t by computing via the forwards-backwards algorithm the conditional distribution of the state vector given the observed data. The covariance matrix of this conditional distribution measures the information lost by failure to observe dire...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章,评审

    doi:10.2202/1544-6115.1296

    authors: Siegmund DO,Yakir B

    更新日期:2007-01-01 00:00:00

  • Variance and covariance heterogeneity analysis for detection of metabolites associated with cadmium exposure.

    abstract::In this study, we propose a novel statistical framework for detecting progressive changes in molecular traits as response to a pathogenic stimulus. In particular, we propose to employ Bayesian hierarchical models to analyse changes in mean level, variance and correlation of metabolic traits in relation to covariates. ...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2013-0041

    authors: Salamanca BV,Ebbels TM,Iorio MD

    更新日期:2014-04-01 00:00:00

  • Predicting protein concentrations with ELISA microarray assays, monotonic splines and Monte Carlo simulation.

    abstract::Making sound proteomic inferences using ELISA microarray assay requires both an accurate prediction of protein concentration and a credible estimate of its error. We present a method using monotonic spline statistical models (MS), penalized constrained least squares fitting (PCLS) and Monte Carlo simulation (MC) to pr...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1364

    authors: Daly DS,Anderson KK,White AM,Gonzalez RM,Varnum SM,Zangar RC

    更新日期:2008-01-01 00:00:00

  • On an extended interpretation of linkage disequilibrium in genetic case-control association studies.

    abstract::We are concerned with statistical inference for 2 × C × K contingency tables in the context of genetic case-control association studies. Multivariate methods based on asymptotic Gaussianity of vectors of test statistics require information about the asymptotic correlation structure among these test statistics under th...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2015-0024

    authors: Dickhaus T,Stange J,Demirhan H

    更新日期:2015-11-01 00:00:00

  • Multiple testing in candidate gene situations: a comparison of classical, discrete, and resampling-based procedures.

    abstract::In candidate gene association studies, usually several elementary hypotheses are tested simultaneously using one particular set of data. The data normally consist of partly correlated SNP information. Every SNP can be tested for association with the disease, e.g., using the Cochran-Armitage test for trend. To account ...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1729

    authors: Elsäβer A,Victor A,Hommel G

    更新日期:2011-01-01 00:00:00

  • Modeling, simulation and analysis of methylation profiles from reduced representation bisulfite sequencing experiments.

    abstract::The ENCODE project has funded the generation of a diverse collection of methylation profiles using reduced representation bisulfite sequencing (RRBS) technology, enabling the analysis of epigenetic variation on a genomic scale at single-site resolution. A standard application of RRBS experiments is in the location of ...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2013-0027

    authors: Lacey MR,Baribault C,Ehrlich M

    更新日期:2013-12-01 00:00:00

  • Mapping quantitative trait loci in a non-equilibrium population.

    abstract::The genetic control of a complex trait can be studied by testing and mapping the genotypes of the underlying quantitative trait loci (QTLs) through their associations with observable marker genotypes. All existing statistical methods for QTL mapping assume an equilibrium population, allowing marker-QTL associations to...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1578

    authors: Wu S,Yang J,Wu R

    更新日期:2010-01-01 00:00:00

  • Genetic association test based on principal component analysis.

    abstract::Many gene- and pathway-based association tests have been proposed in the literature. Among them, the SKAT is widely used, especially for rare variants association studies. In this paper, we investigate the connection between SKAT and a principal component analysis. This investigation leads to a procedure that encompas...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2016-0061

    authors: Chen Z,Han S,Wang K

    更新日期:2017-07-26 00:00:00

  • Buckley-James boosting for survival analysis with high-dimensional biomarker data.

    abstract::There has been increasing interest in predicting patients' survival after therapy by investigating gene expression microarray data. In the regression and classification models with high-dimensional genomic data, boosting has been successfully applied to build accurate predictive models and conduct variable selection s...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1550

    authors: Wang Z,Wang CY

    更新日期:2010-01-01 00:00:00

  • Accommodating uncertainty in a tree set for function estimation.

    abstract::Multiple branching trees have been used to model the acquisition of HIV drug resistance mutations, and several different algorithms have been developed to construct the tree set that best describes the data. These algorithms have mainly focused on the structure of the tree set. The focal point of this paper is estimat...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1324

    authors: Healy BC,DeGruttola VG,Hu C

    更新日期:2008-01-01 00:00:00

  • Accounting for undetected compounds in statistical analyses of mass spectrometry 'omic studies.

    abstract::Mass spectrometry is an important high-throughput technique for profiling small molecular compounds in biological samples and is widely used to identify potential diagnostic and prognostic compounds associated with disease. Commonly, this data generated by mass spectrometry has many missing values resulting when a com...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2013-0021

    authors: Taylor SL,Leiserowitz GS,Kim K

    更新日期:2013-12-01 00:00:00

  • Asymptotic optimality of likelihood-based cross-validation.

    abstract::Likelihood-based cross-validation is a statistical tool for selecting a density estimate based on n i.i.d. observations from the true density among a collection of candidate density estimators. General examples are the selection of a model indexing a maximum likelihood estimator, and the selection of a bandwidth index...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1036

    authors: van der Laan MJ,Dudoit S,Keles S

    更新日期:2004-01-01 00:00:00

  • Empirical bayes estimation of a sparse vector of gene expression changes.

    abstract::Gene microarray technology is often used to compare the expression of thousand of genes in two different cell lines. Typically, one does not expect measurable changes in transcription amounts for a large number of genes; furthermore, the noise level of array experiments is rather high in relation to the available numb...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1132

    authors: Erickson S,Sabatti C

    更新日期:2005-01-01 00:00:00

  • Transmission disequilibrium test power and sample size in the presence of locus heterogeneity.

    abstract::Locus heterogeneity is one of the most important issues in gene mapping and can cause significant reductions in statistical power for gene mapping, yet no research to date has provided power and sample size calculations for family-based association methods in the presence of locus heterogeneity. The purpose of this re...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1501

    authors: Chen C,Yang G,Buyske S,Matise T,Finch SJ,Gordon D

    更新日期:2009-01-01 00:00:00

  • TopKLists: a comprehensive R package for statistical inference, stochastic aggregation, and visualization of multiple omics ranked lists.

    abstract::High-throughput sequencing techniques are increasingly affordable and produce massive amounts of data. Together with other high-throughput technologies, such as microarrays, there are an enormous amount of resources in databases. The collection of these valuable data has been routine for more than a decade. Despite di...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2014-0093

    authors: Schimek MG,Budinská E,Kugler KG,Švendová V,Ding J,Lin S

    更新日期:2015-06-01 00:00:00

  • A multiple testing approach to high-dimensional association studies with an application to the detection of associations between risk factors of heart disease and genetic polymorphisms.

    abstract::We present an approach to association studies involving a dozen or so ;response' variables and a few hundred ;explanatory' variables which emphasizes transparency, simplicity, and protection against spurious results. The methods proposed are largely non-parametric, and they are systematically rounded-off by the Benjam...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1420

    authors: Ferreira JA,Berkhof J,Souverein O,Zwinderman K

    更新日期:2009-01-01 00:00:00

  • Surveying the manifold divergence of an entire protein class for statistical clues to underlying biochemical mechanisms.

    abstract::Certain residues have no known function yet are co-conserved across distantly related protein families and diverse organisms, suggesting that they perform critical roles associated with as-yet-unidentified molecular properties and mechanisms. This raises the question of how to obtain additional clues regarding these m...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1666

    authors: Neuwald AF

    更新日期:2011-01-01 00:00:00

  • Comparison and visualisation of agreement for paired lists of rankings.

    abstract::Output from analysis of a high-throughput 'omics' experiment very often is a ranked list. One commonly encountered example is a ranked list of differentially expressed genes from a gene expression experiment, with a length of many hundreds of genes. There are numerous situations where interest is in the comparison of ...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2016-0036

    authors: Donald MR,Wilson SR

    更新日期:2017-03-01 00:00:00

  • Reproducibility of biomarker identifications from mass spectrometry proteomic data in cancer studies.

    abstract::Reproducibility of disease signatures and clinical biomarkers in multi-omics disease analysis has been a key challenge due to a multitude of factors. The heterogeneity of the limited sample, various biological factors such as environmental confounders, and the inherent experimental and technical noises, compounded wit...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2018-0039

    authors: Liang Y,Kelemen A,Kelemen A

    更新日期:2019-05-11 00:00:00

  • Node sampling for protein complex estimation in bait-prey graphs.

    abstract::In cellular biology, node-and-edge graph or "network" data collection often uses bait-prey technologies such as co-immunoprecipitation (CoIP). Bait-prey technologies assay relationships or "interactions" between protein pairs, with CoIP specifically measuring protein complex co-membership. Analyses of CoIP data freque...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2015-0007

    authors: Scholtens DM,Spencer BD

    更新日期:2015-08-01 00:00:00

  • Detecting outlier samples in microarray data.

    abstract::In this paper, we address the problem of detecting outlier samples with highly different expression patterns in microarray data. Although outliers are not common, they appear even in widely used benchmark data sets and can negatively affect microarray data analysis. It is important to identify outliers in order to exp...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.2202/1544-6115.1426

    authors: Shieh AD,Hung YS

    更新日期:2009-01-01 00:00:00

  • Fast approximate inference for variable selection in Dirichlet process mixtures, with an application to pan-cancer proteomics.

    abstract::The Dirichlet Process (DP) mixture model has become a popular choice for model-based clustering, largely because it allows the number of clusters to be inferred. The sequential updating and greedy search (SUGS) algorithm (Wang & Dunson, 2011) was proposed as a fast method for performing approximate Bayesian inference ...

    journal_title:Statistical applications in genetics and molecular biology

    pub_type: 杂志文章

    doi:10.1515/sagmb-2018-0065

    authors: Crook OM,Gatto L,Kirk PDW

    更新日期:2019-12-12 00:00:00