Biased binomial assessment of cross-validated estimation of classification accuracies illustrated in diagnosis predictions.

Abstract:

:Multivariate classification is used in neuroimaging studies to infer brain activation or in medical applications to infer diagnosis. Their results are often assessed through either a binomial or a permutation test. Here, we simulated classification results of generated random data to assess the influence of the cross-validation scheme on the significance of results. Distributions built from classification of random data with cross-validation did not follow the binomial distribution. The binomial test is therefore not adapted. On the contrary, the permutation test was unaffected by the cross-validation scheme. The influence of the cross-validation was further illustrated on real-data from a brain-computer interface experiment in patients with disorders of consciousness and from an fMRI study on patients with Parkinson disease. Three out of 16 patients with disorders of consciousness had significant accuracy on binomial testing, but only one showed significant accuracy using permutation testing. In the fMRI experiment, the mental imagery of gait could discriminate significantly between idiopathic Parkinson's disease patients and healthy subjects according to the permutation test but not according to the binomial test. Hence, binomial testing could lead to biased estimation of significance and false positive or negative results. In our view, permutation testing is thus recommended for clinical application of classification with cross-validation.

journal_name

Neuroimage Clin

journal_title

NeuroImage. Clinical

authors

Noirhomme Q,Lesenfants D,Gomez F,Soddu A,Schrouff J,Garraux G,Luxen A,Phillips C,Laureys S

doi

10.1016/j.nicl.2014.04.004

subject

Has Abstract

pub_date

2014-04-13 00:00:00

pages

687-94

issn

2213-1582

pii

S2213-1582(14)00048-5

journal_volume

4

pub_type

杂志文章
  • Preterm birth and structural brain alterations in early adulthood.

    abstract::Alterations in cortical development and impaired neurodevelopmental outcomes have been described following very preterm (VPT) birth in childhood and adolescence, but only a few studies to date have investigated grey matter (GM) and white matter (WM) maturation in VPT samples in early adult life. Using voxel-based morp...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2014.08.005

    authors: Nosarti C,Nam KW,Walshe M,Murray RM,Cuddy M,Rifkin L,Allin MP

    更新日期:2014-08-13 00:00:00

  • Dysfunction of ventrolateral prefrontal cortex underlying social anxiety disorder: A multi-channel NIRS study.

    abstract::Social anxiety disorder (SAD) is characterized by strong fear and anxiety during social interactions. Although ventrolateral prefrontal cortex (VLPFC) activity in response to emotional stimuli is related to pathological anxiety, little is known about the relationship between VLPFC activity and social anxiety. This stu...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2015.05.011

    authors: Yokoyama C,Kaiya H,Kumano H,Kinou M,Umekage T,Yasuda S,Takei K,Nishikawa M,Sasaki T,Nishimura Y,Hara N,Inoue K,Kaneko Y,Suzuki S,Tanii H,Okada M,Okazaki Y

    更新日期:2015-05-28 00:00:00

  • The diffeomorphometry of temporal lobe structures in preclinical Alzheimer's disease.

    abstract::This paper examines morphometry of MRI biomarkers derived from the network of temporal lobe structures including the amygdala, entorhinal cortex and hippocampus in subjects with preclinical Alzheimer's disease (AD). Based on template-centered population analysis, it is demonstrated that the structural markers of the a...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2013.09.001

    authors: Miller MI,Younes L,Ratnanather JT,Brown T,Trinh H,Postell E,Lee DS,Wang MC,Mori S,O'Brien R,Albert M,BIOCARD Research Team.

    更新日期:2013-09-16 00:00:00

  • Measuring longitudinal myelin water fraction in new multiple sclerosis lesions.

    abstract:OBJECTIVES:Investigating the potential of myelin repair strategies in multiple sclerosis (MS) requires an understanding of myelin dynamics during lesion evolution. The objective of this study is to longitudinally measure myelin water fraction (MWF), an MRI biomarker of myelin, in new MS lesions and to identify factors ...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2015.09.003

    authors: Vargas WS,Monohan E,Pandya S,Raj A,Vartanian T,Nguyen TD,Hurtado Rúa SM,Gauthier SA

    更新日期:2015-09-12 00:00:00

  • Draining the pond and catching the fish: Uncovering the ecosystem of auditory verbal hallucinations.

    abstract::The various models proposed for the mediation of auditory verbal hallucinations (AVH) implicate a considerable number of brain areas and mechanisms. To establish which of those mechanisms are actually involved in the mediation of AVH, we developed a novel method to analyze functional MRI data, which allows for the det...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2018.09.016

    authors: Looijestijn J,Blom JD,Hoek HW,Renken R,Liemburg E,Sommer IEC,Aleman A,Goekoop R

    更新日期:2018-01-01 00:00:00

  • Fibromyalgia interacts with age to change the brain.

    abstract::Although brain plasticity in the form of gray matter increases and decreases has been observed in chronic pain, factors determining the patterns of directionality are largely unknown. Here we tested the hypothesis that fibromyalgia interacts with age to produce distinct patterns of gray matter differences, specificall...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2013.08.015

    authors: Ceko M,Bushnell MC,Fitzcharles MA,Schweinhardt P

    更新日期:2013-09-06 00:00:00

  • Dysfunction between dorsal caudate and salience network associated with impaired cognitive flexibility in obsessive-compulsive disorder: A resting-state fMRI study.

    abstract:BACKGROUND:Impaired cognitive flexibility has been implicated in the genetic basis of obsessive-compulsive disorder (OCD). Recent endophenotype studies of OCD showed neural inefficiency in the cognitive control network and interference by the limbic network of the cognitive control network. Exploring the relationship b...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2019.102004

    authors: Tomiyama H,Nakao T,Murayama K,Nemoto K,Ikari K,Yamada S,Kuwano M,Hasuzawa S,Togao O,Hiwatashi A,Kanba S

    更新日期:2019-01-01 00:00:00

  • Brain putamen volume changes in newly-diagnosed patients with obstructive sleep apnea.

    abstract::Obstructive sleep apnea (OSA) is accompanied by cognitive, motor, autonomic, learning, and affective abnormalities. The putamen serves several of these functions, especially motor and autonomic behaviors, but whether global and specific sub-regions of that structure are damaged is unclear. We assessed global and regio...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2014.01.009

    authors: Kumar R,Farahvar S,Ogren JA,Macey PM,Thompson PM,Woo MA,Yan-Go FL,Harper RM

    更新日期:2014-01-31 00:00:00

  • Insight and emotion regulation in schizophrenia: A brain activation and functional connectivity study.

    abstract:BACKGROUND:Insight is impaired in the majority of schizophrenia patients. The exact neural correlates of impaired insight remain unclear. We assume that the ability to regulate emotions contributes to having good clinical insight, as patients should be able to regulate their emotional state in such a way that they can ...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2018.09.009

    authors: Larabi DI,van der Meer L,Pijnenborg GHM,Ćurčić-Blake B,Aleman A

    更新日期:2018-01-01 00:00:00

  • Evidence for decreased Neurologic Pain Signature activation following thoracic spinal manipulation in healthy volunteers and participants with neck pain.

    abstract:BACKGROUND CONTEXT:Spinal manipulation (SM) is a common treatment for neck and back pain, theorized to mechanically affect the spine leading to therapeutic mechanical changes. The link between specific mechanical effects and clinical improvement is not well supported. SM's therapeutic action may instead be partially me...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章,随机对照试验

    doi:10.1016/j.nicl.2019.102042

    authors: Weber Ii KA,Wager TD,Mackey S,Elliott JM,Liu WC,Sparks CL

    更新日期:2019-01-01 00:00:00

  • Alzheimer's disease patients activate attention networks in a short-term memory task.

    abstract::Network functioning during cognitive tasks is of major interest in Alzheimer's disease (AD). Cognitive functioning in AD includes variable performance in short-term memory (STM). In most studies, the verbal STM functioning in AD patients has been interpreted within the phonological loop subsystem of Baddeley's working...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2019.101892

    authors: Kurth S,Bahri MA,Collette F,Phillips C,Majerus S,Bastin C,Salmon E

    更新日期:2019-01-01 00:00:00

  • Impact of schizophrenia on anterior and posterior hippocampus during memory for complex scenes.

    abstract:OBJECTIVES:Hippocampal dysfunction has been proposed as a mechanism for memory deficits in schizophrenia. Available evidence suggests that the anterior and posterior hippocampus could be differentially affected. Accordingly, we used fMRI to test the hypothesis that activity in posterior hippocampus is disproportionatel...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2016.11.017

    authors: Ragland JD,Layher E,Hannula DE,Niendam TA,Lesh TA,Solomon M,Carter CS,Ranganath C

    更新日期:2016-11-17 00:00:00

  • Brain metabolic maps in Mild Cognitive Impairment predict heterogeneity of progression to dementia.

    abstract::[(18)F]FDG-PET imaging has been recognized as a crucial diagnostic marker in Mild Cognitive Impairment (MCI), supporting the presence or the exclusion of Alzheimer's Disease (AD) pathology. A clinical heterogeneity, however, underlies MCI definition. In this study, we aimed to evaluate the predictive role of single-su...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2014.12.004

    authors: Cerami C,Della Rosa PA,Magnani G,Santangelo R,Marcone A,Cappa SF,Perani D

    更新日期:2014-12-05 00:00:00

  • Cortical thickness changes in the non-lesioned hemisphere associated with non-paretic arm immobilization in modified CI therapy.

    abstract::Recent evidence suggests that immobilization of the upper limb for 2-3 weeks induces changes in cortical thickness as well as motor performance. In constraint induced (CI) therapy, one of the most effective interventions for hemiplegia, the non-paretic arm is constrained to enforce the use of the paretic arm in the ho...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2013.05.005

    authors: Sterr A,Dean PJ,Vieira G,Conforto AB,Shen S,Sato JR

    更新日期:2013-05-21 00:00:00

  • Prion disease diagnosis using subject-specific imaging biomarkers within a multi-kernel Gaussian process.

    abstract::Prion diseases are a group of rare neurodegenerative conditions characterised by a high rate of progression and highly heterogeneous phenotypes. Whilst the most common form of prion disease occurs sporadically (sporadic Creutzfeldt-Jakob disease, sCJD), other forms are caused by prion protein gene mutations, or exposu...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2019.102051

    authors: Canas LS,Sudre CH,De Vita E,Nihat A,Mok TH,Slattery CF,Paterson RW,Foulkes AJM,Hyare H,Cardoso MJ,Thornton J,Schott JM,Barkhof F,Collinge J,Ourselin S,Mead S,Modat M

    更新日期:2019-01-01 00:00:00

  • Hypo-activity of the dorsolateral prefrontal cortex relates to increased reaction time variability in patients with schizophrenia.

    abstract::Increased reaction time intra-subject variability (RT-ISV) in fast decision tasks has been confirmed in patients with schizophrenia and has been hypothesized to result from a deficit in the control of attention. Here, an attentional task and functional brain imaging were used to probe the neural correlates of increase...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2019.101853

    authors: Panagiotaropoulou G,Thrapsanioti E,Pappa E,Grigoras C,Mylonas D,Karavasilis E,Velonakis G,Kelekis N,Smyrnis N

    更新日期:2019-01-01 00:00:00

  • Sensory and cross-network contributions to response inhibition in patients with schizophrenia.

    abstract::Patients with schizophrenia show response inhibition deficits equal to or greater than those seen in impulse-control disorders, and these deficits contribute to poor outcome. However, little is known about the circuit abnormalities underlying this impairment. To address this, we examined stop signal task performance i...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2018.01.001

    authors: Hoptman MJ,Parker EM,Nair-Collins S,Dias EC,Ross ME,DiCostanzo JN,Sehatpour P,Javitt DC

    更新日期:2018-01-06 00:00:00

  • Identifying commonality and specificity across psychosis sub-groups via classification based on features from dynamic connectivity analysis.

    abstract::It is difficult to distinguish schizophrenia (SZ), schizoaffective disorder (SAD), and bipolar disorder with psychosis (BPP) as their clinical diagnoses rely on symptoms that overlap. In this paper, we investigate if there is biological evidence to support the symptom-based clinical categories by looking across the th...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2020.102284

    authors: Du Y,Hao H,Wang S,Pearlson GD,Calhoun VD

    更新日期:2020-01-01 00:00:00

  • Multiparametric graph theoretical analysis reveals altered structural and functional network topology in Alzheimer's disease.

    abstract::Alzheimer's disease (AD), an irreversible neurodegenerative disease, is the most common type of dementia in elderly people. This present study incorporated multiple structural and functional connectivity metrics into a graph theoretical analysis framework and investigated alterations in brain network topology in patie...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2019.101680

    authors: Lin SY,Lin CP,Hsieh TJ,Lin CF,Chen SH,Chao YP,Chen YS,Hsu CC,Kuo LW

    更新日期:2019-01-01 00:00:00

  • Osmotic release oral system-methylphenidate improves neural activity during low reward processing in children and adolescents with attention-deficit/hyperactivity disorder.

    abstract::Attention-deficit/hyperactivity disorder (ADHD) is neurobehavioral disorder characterized by inattention, hyperactivity/impulsivity and impaired reward system function, such as delay aversion and low reward sensitivity. The pharmacological treatment for ADHD includes methylphenidate (MPH), or osmotic release oral syst...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2013.03.004

    authors: Mizuno K,Yoneda T,Komi M,Hirai T,Watanabe Y,Tomoda A

    更新日期:2013-03-16 00:00:00

  • High-resolution metabolic imaging of high-grade gliomas using 7T-CRT-FID-MRSI.

    abstract:OBJECTIVES:Successful neurosurgical intervention in gliomas depends on the precision of the preoperative definition of the tumor and its margins since a safe maximum resection translates into a better patient outcome. Metabolic high-resolution imaging might result in improved presurgical tumor characterization, and thu...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2020.102433

    authors: Hangel G,Cadrien C,Lazen P,Furtner J,Lipka A,Hečková E,Hingerl L,Motyka S,Gruber S,Strasser B,Kiesel B,Mischkulnig M,Preusser M,Roetzer T,Wöhrer A,Widhalm G,Rössler K,Trattnig S,Bogner W

    更新日期:2020-01-01 00:00:00

  • Morphometric MRI as a diagnostic biomarker of frontotemporal dementia: A systematic review to determine clinical applicability.

    abstract::Frontotemporal dementia (FTD) is difficult to diagnose, due to its heterogeneous nature and overlap in symptoms with primary psychiatric disorders. Brain MRI for atrophy is a key biomarker but lacks sensitivity in the early stage. Morphometric MRI-based measures and machine learning techniques are a promising tool to ...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2018.08.028

    authors: McCarthy J,Collins DL,Ducharme S

    更新日期:2018-01-01 00:00:00

  • Data fusion detects consistent relations between non-lesional white matter myelin, executive function, and clinical characteristics in multiple sclerosis.

    abstract::We examined the influence of dysfunctional, non-lesional white matter on cognitive performance in multiple sclerosis (MS). Forty-six MS subjects were assessed using MRI-based myelin water imaging (MWI), and average myelin water fraction (MWF) values across 20 white matter regions of interest (ROIs) were determined. A ...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2019.101926

    authors: Baumeister TR,Lin SJ,Vavasour I,Kolind S,Kosaka B,Li DKB,Traboulsee A,MacKay A,McKeown MJ

    更新日期:2019-01-01 00:00:00

  • Exposing asymmetric gray matter vulnerability in amyotrophic lateral sclerosis.

    abstract::Limb weakness in amyotrophic lateral sclerosis (ALS) is typically asymmetric. Previous studies have identified an effect of limb dominance on onset and spread of weakness, however relative atrophy of dominant and non-dominant brain regions has not been investigated. Our objective was to use voxel-based morphometry (VB...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2015.03.006

    authors: Devine MS,Pannek K,Coulthard A,McCombe PA,Rose SE,Henderson RD

    更新日期:2015-03-14 00:00:00

  • Regional expression of genes mediating trans-synaptic alpha-synuclein transfer predicts regional atrophy in Parkinson disease.

    abstract::Multiple genes have been implicated in Parkinson disease pathogenesis, but the relationship between regional expression of these genes and regional dysfunction across the brain is unknown. We address this question by joint analysis of high resolution magnetic resonance imaging data from the Parkinson's Progression Mar...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2018.01.009

    authors: Freeze B,Acosta D,Pandya S,Zhao Y,Raj A

    更新日期:2018-01-31 00:00:00

  • Relation of 18-F-Dopa PET with hypokinesia-rigidity, tremor and freezing in Parkinson's disease.

    abstract:INTRODUCTION:In this retrospective study concerning patients with Parkinson's disease (PD) scanned with 18-F-Dopa PET (N = 129), we looked for an association between reduced 18-F-Dopa uptake and the key PD symptoms tremor and hypokinesia-rigidity. We hypothesized to find a stronger correlation between dopaminergic depl...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2016.01.010

    authors: Pikstra ARA,van der Hoorn A,Leenders KL,de Jong BM

    更新日期:2016-01-12 00:00:00

  • Identifying motor functional neurological disorder using resting-state functional connectivity.

    abstract:BACKGROUND:Motor functional neurological disorder (mFND) is a clinical diagnosis with reliable features; however, patients are reluctant to accept the diagnosis and physicians themselves bear doubts on potential misdiagnoses. The identification of a positive biomarker could help limiting unnecessary costs of multiple r...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2017.10.012

    authors: Wegrzyk J,Kebets V,Richiardi J,Galli S,de Ville DV,Aybek S

    更新日期:2017-10-12 00:00:00

  • Disease-related cortical thinning in presymptomatic granulin mutation carriers.

    abstract::Mutations in the granulin gene (GRN) cause familial frontotemporal dementia. Understanding the structural brain changes in presymptomatic GRN carriers would enforce the use of neuroimaging biomarkers for early diagnosis and monitoring. We studied 100 presymptomatic GRN mutation carriers and 94 noncarriers from the Gen...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2020.102540

    authors: Borrego-Écija S,Sala-Llonch R,van Swieten J,Borroni B,Moreno F,Masellis M,Tartaglia C,Graff C,Galimberti D,Laforce R Jr,Rowe JB,Finger E,Vandenberghe R,Tagliavini F,de Mendonça A,Santana I,Synofzik M,Ducharme S,Levin

    更新日期:2020-12-29 00:00:00

  • Sleep onset uncovers thalamic abnormalities in patients with idiopathic generalised epilepsy.

    abstract::The thalamus is crucial for sleep regulation and the pathophysiology of idiopathic generalised epilepsy (IGE), and may serve as the underlying basis for the links between the two. We investigated this using EEG-fMRI and a specific emphasis on the role and functional connectivity (FC) of the thalamus. We defined three ...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章

    doi:10.1016/j.nicl.2017.07.008

    authors: Bagshaw AP,Hale JR,Campos BM,Rollings DT,Wilson RS,Alvim MKM,Coan AC,Cendes F

    更新日期:2017-07-12 00:00:00

  • Relevance of brain lesion location for cognition in vascular mild cognitive impairment.

    abstract:BACKGROUND:Vascular mild cognitive impairment (VMCI) is a potentially transitional state between normal aging and vascular dementia. The presence of macroscopic white matter lesions (WML) of moderate or severe extension on brain MRI is the hallmark of the VMCI. OBJECTIVE:To assess the clinical relevance of the frequen...

    journal_title:NeuroImage. Clinical

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.nicl.2019.101789

    authors: Giorgio A,Di Donato I,De Leucio A,Zhang J,Salvadori E,Poggesi A,Diciotti S,Cosottini M,Ciulli S,Inzitari D,Pantoni L,Mascalchi M,Federico A,Dotti MT,De Stefano N,VMCI-Tuscany Study Group.

    更新日期:2019-01-01 00:00:00