Transferring and generalizing deep-learning-based neural encoding models across subjects.

Abstract:

:Recent studies have shown the value of using deep learning models for mapping and characterizing how the brain represents and organizes information for natural vision. However, modeling the relationship between deep learning models and the brain (or encoding models), requires measuring cortical responses to large and diverse sets of natural visual stimuli from single subjects. This requirement limits prior studies to few subjects, making it difficult to generalize findings across subjects or for a population. In this study, we developed new methods to transfer and generalize encoding models across subjects. To train encoding models specific to a target subject, the models trained for other subjects were used as the prior models and were refined efficiently using Bayesian inference with a limited amount of data from the target subject. To train encoding models for a population, the models were progressively trained and updated with incremental data from different subjects. For the proof of principle, we applied these methods to functional magnetic resonance imaging (fMRI) data from three subjects watching tens of hours of naturalistic videos, while a deep residual neural network driven by image recognition was used to model visual cortical processing. Results demonstrate that the methods developed herein provide an efficient and effective strategy to establish both subject-specific and population-wide predictive models of cortical representations of high-dimensional and hierarchical visual features.

journal_name

Neuroimage

journal_title

NeuroImage

authors

Wen H,Shi J,Chen W,Liu Z

doi

10.1016/j.neuroimage.2018.04.053

subject

Has Abstract

pub_date

2018-08-01 00:00:00

pages

152-163

eissn

1053-8119

issn

1095-9572

pii

S1053-8119(18)30363-X

journal_volume

176

pub_type

杂志文章
  • Interacting sources of interference during sensorimotor integration processes.

    abstract::Every day, a multitude of interfering sensory inputs needs to be integrated and adequately processed using response selection processes. Interference effects are typically investigated using classical paradigms like the Flanker and Simon task. The sources of interference for Flanker and Simon effect are known to be di...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2015.09.075

    authors: Mückschel M,Stock AK,Dippel G,Chmielewski W,Beste C

    更新日期:2016-01-15 00:00:00

  • Age-related changes in reappraisal of appetitive cravings during adolescence.

    abstract::The ability to regulate temptation and manage appetitive cravings is an important aspect of healthy adolescent development, but the neural systems underlying this process are understudied. In the present study, 60 healthy females evenly distributed from 10 to 23years of age used reappraisal to regulate the desire to c...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2014.12.037

    authors: Giuliani NR,Pfeifer JH

    更新日期:2015-03-01 00:00:00

  • Small animal, whole brain fMRI: innocuous and nociceptive forepaw stimulation.

    abstract::Supra-spinal pain processing involves a number of extensive networks. An examination of these networks using small animal functional magnetic resonance imaging (fMRI) is difficult. While prior studies have successfully delineated regions consistent with known pain processing pathways, they have been restricted to acqu...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2006.12.014

    authors: Lowe AS,Beech JS,Williams SC

    更新日期:2007-04-01 00:00:00

  • Investigation into the neural correlates of emotional augmentation of clinical pain.

    abstract::Although depressive mood is an important psychological determinate of chronic pain, the neural circuitry that mediates its influence on the pain experience is largely unknown. We used functional magnetic resonance imaging (FMRI) to investigate the neurophysiological interactions between depressive symptoms and disease...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2007.12.016

    authors: Schweinhardt P,Kalk N,Wartolowska K,Chessell I,Wordsworth P,Tracey I

    更新日期:2008-04-01 00:00:00

  • Multi-session statistics on beamformed MEG data.

    abstract::Beamforming has been widely adopted as a source reconstruction technique in the analysis of magnetoencephalography data. Most beamforming implementations incorporate a spatially-varying rescaling (which we term weights normalisation) to correct for the inherent depth bias in raw beamformer estimates. Here, we demonstr...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2013.12.026

    authors: Luckhoo HT,Brookes MJ,Woolrich MW

    更新日期:2014-07-15 00:00:00

  • Early electrophysiological responses to multiple face orientations correlate with individual discrimination performance in humans.

    abstract::Picture-plane inversion dramatically impairs face recognition. Behavioral and event-related potential (ERP) studies suggest that this effect takes place during perceptual encoding of the face stimulus. However, the relationship between early electrophysiological responses to upright and inverted faces and the behavior...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2007.04.016

    authors: Jacques C,Rossion B

    更新日期:2007-07-01 00:00:00

  • Physical and physiological consequences of passive intra-oral shimming.

    abstract::Imaging the human orbitofrontal cortex (OFC) with fMRI is problematic due to the proximity of this region to the air-filled sinuses, which causes susceptibility artifacts. Placing a strongly diamagnetic material into the mouth ('mouthshim') of a human volunteer can significantly reduce the artifacts in this region. Us...

    journal_title:NeuroImage

    pub_type: 临床试验,杂志文章

    doi:10.1016/j.neuroimage.2005.07.006

    authors: Osterbauer RA,Wilson JL,Calvert GA,Jezzard P

    更新日期:2006-01-01 00:00:00

  • Hard to wake up? The cerebral correlates of sleep inertia assessed using combined behavioral, EEG and fMRI measures.

    abstract::The first minutes following awakening from sleep are typically marked by reduced vigilance, increased sleepiness and impaired performance, a state referred to as sleep inertia. Although the behavioral aspects of sleep inertia are well documented, its cerebral correlates remain poorly understood. The present study aime...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2018.09.033

    authors: Vallat R,Meunier D,Nicolas A,Ruby P

    更新日期:2019-01-01 00:00:00

  • Surface based analysis of diffusion orientation for identifying architectonic domains in the in vivo human cortex.

    abstract::Diffusion tensor MRI is sensitive to the coherent structure of brain tissue and is commonly used to study large-scale white matter structure. Diffusion in gray matter is more isotropic, however, several groups have observed coherent patterns of diffusion anisotropy within the cerebral cortical gray matter. We extend t...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2012.11.065

    authors: McNab JA,Polimeni JR,Wang R,Augustinack JC,Fujimoto K,Stevens A,Triantafyllou C,Janssens T,Farivar R,Folkerth RD,Vanduffel W,Wald LL

    更新日期:2013-04-01 00:00:00

  • Interaction of hippocampal volume and N-acetylaspartate concentration deficits in schizophrenia: a combined MRI and 1H-MRS study.

    abstract:BACKGROUND:Volume deficits assessed with magnetic resonance imaging (MRI) and neurochemical dysfunctions (N-acetylaspartate, NAA) diagnosed using proton MR spectroscopy ((1)H-MRS) are reliable observations in the hippocampus of schizophrenic patients. NAA is an important cerebral amino acid in the synthesis pathways of...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2010.06.006

    authors: Klär AA,Ballmaier M,Leopold K,Häke I,Schaefer M,Brühl R,Schubert F,Gallinat J

    更新日期:2010-10-15 00:00:00

  • Investigation of the neurovascular coupling in positive and negative BOLD responses in human brain at 7 T.

    abstract::Decreases in stimulus-dependent blood oxygenation level dependent (BOLD) signal and their underlying neurovascular origins have recently gained considerable interest. In this study a multi-echo, BOLD-corrected vascular space occupancy (VASO) functional magnetic resonance imaging (fMRI) technique was used to investigat...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2014.04.022

    authors: Huber L,Goense J,Kennerley AJ,Ivanov D,Krieger SN,Lepsien J,Trampel R,Turner R,Möller HE

    更新日期:2014-08-15 00:00:00

  • Deep grey matter iron accumulation in alcohol use disorder.

    abstract:PURPOSE:Evaluate brain iron accumulation in alcohol use disorder (AUD) patients compared to controls using quantitative susceptibility mapping (QSM). METHODS:QSM was performed retrospectively by using phase images from resting state functional magnetic resonance imaging (fMRI). 20 male AUD patients and 15 matched heal...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2017.01.007

    authors: Juhás M,Sun H,Brown MRG,MacKay MB,Mann KF,Sommer WH,Wilman AH,Dursun SM,Greenshaw AJ

    更新日期:2017-03-01 00:00:00

  • Problem solving, working memory, and motor correlates of association and commissural fiber bundles in normal aging: a quantitative fiber tracking study.

    abstract::Normal aging is accompanied by decline in selective cognitive and motor functions. A concurrent decline in regional white matter integrity, detectable with diffusion tensor imaging (DTI), potentially contributes to waning function. DTI analysis of white matter loci indicates an anterior-to-posterior gradient distribut...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2008.09.046

    authors: Zahr NM,Rohlfing T,Pfefferbaum A,Sullivan EV

    更新日期:2009-02-01 00:00:00

  • Biophysical modeling of high field diffusion MRI demonstrates micro-structural aberration in chronic mild stress rat brain.

    abstract::Depression is one of the leading causes of disability worldwide. Immense heterogeneity in symptoms of depression causes difficulty in diagnosis, and to date, there are no established biomarkers or imaging methods to examine depression. Unpredictable chronic mild stress (CMS) induced anhedonia is considered to be a rea...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2016.07.001

    authors: Khan AR,Chuhutin A,Wiborg O,Kroenke CD,Nyengaard JR,Hansen B,Jespersen SN

    更新日期:2016-11-15 00:00:00

  • Unbiased tensor-based morphometry: improved robustness and sample size estimates for Alzheimer's disease clinical trials.

    abstract::Various neuroimaging measures are being evaluated for tracking Alzheimer's disease (AD) progression in therapeutic trials, including measures of structural brain change based on repeated scanning of patients with magnetic resonance imaging (MRI). Methods to compute brain change must be robust to scan quality. Biases m...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2012.10.086

    authors: Hua X,Hibar DP,Ching CR,Boyle CP,Rajagopalan P,Gutman BA,Leow AD,Toga AW,Jack CR Jr,Harvey D,Weiner MW,Thompson PM,Alzheimer's Disease Neuroimaging Initiative.

    更新日期:2013-02-01 00:00:00

  • Right-hemispheric brain activation correlates to language performance.

    abstract::Language function in the right-hemispheric homologues of Broca's and Wernicke's areas does not only correlate with left-handedness or pathology, but occurs naturally in right-handed healthy subjects as well. In the current study, two non-invasive methods of assessing language lateralization are correlated with behavio...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2009.10.041

    authors: van Ettinger-Veenstra HM,Ragnehed M,Hällgren M,Karlsson T,Landtblom AM,Lundberg P,Engström M

    更新日期:2010-02-15 00:00:00

  • Increased sensitivity in neuroimaging analyses using robust regression.

    abstract::Robust regression techniques are a class of estimators that are relatively insensitive to the presence of one or more outliers in the data. They are especially well suited to data that require large numbers of statistical tests and may contain outliers due to factors not of experimental interest. Both these issues app...

    journal_title:NeuroImage

    pub_type: 临床试验,杂志文章

    doi:10.1016/j.neuroimage.2005.01.011

    authors: Wager TD,Keller MC,Lacey SC,Jonides J

    更新日期:2005-05-15 00:00:00

  • Targeting the functional properties of cortical neurons using fMR-adaptation.

    abstract::The introduction of functional brain imaging based on BOLD-fMRI, twenty years ago, has revolutionized the field of human brain research. However, right from its inception it became clear that the BOLD signal suffers from a serious limitation--it reflects the averaged activity of large neuronal populations and hence ca...

    journal_title:NeuroImage

    pub_type: 历史文章,杂志文章,评审

    doi:10.1016/j.neuroimage.2012.01.002

    authors: Malach R

    更新日期:2012-08-15 00:00:00

  • Connectivity-behavior analysis reveals that functional connectivity between left BA39 and Broca's area varies with reading ability.

    abstract::Correlations between temporal fluctuations in MRI signals may reveal functional connectivity between brain regions within individual subjects. Such correlations would be especially useful indices of functional connectivity if they covary with behavioral performance or other subject variables. This study investigated w...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2005.12.040

    authors: Hampson M,Tokoglu F,Sun Z,Schafer RJ,Skudlarski P,Gore JC,Constable RT

    更新日期:2006-06-01 00:00:00

  • Local volume fraction distributions of axons, astrocytes, and myelin in deep subcortical white matter.

    abstract::This study aims to statistically describe histologically stained white matter brain sections to subsequently inform and validate diffusion MRI techniques. For the first time, we characterise volume fraction distributions of three of the main structures in deep subcortical white matter (axons, astrocytes, and myelinate...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2018.06.040

    authors: Coelho S,Pozo JM,Costantini M,Highley JR,Mozumder M,Simpson JE,Ince PG,Frangi AF

    更新日期:2018-10-01 00:00:00

  • What do differences between multi-voxel and univariate analysis mean? How subject-, voxel-, and trial-level variance impact fMRI analysis.

    abstract::Multi-voxel pattern analysis (MVPA) has led to major changes in how fMRI data are analyzed and interpreted. Many studies now report both MVPA results and results from standard univariate voxel-wise analysis, often with the goal of drawing different conclusions from each. Because MVPA results can be sensitive to latent...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2014.04.037

    authors: Davis T,LaRocque KF,Mumford JA,Norman KA,Wagner AD,Poldrack RA

    更新日期:2014-08-15 00:00:00

  • Visual cortex responses reflect temporal structure of continuous quasi-rhythmic sensory stimulation.

    abstract::Neural processing of dynamic continuous visual input, and cognitive influences thereon, are frequently studied in paradigms employing strictly rhythmic stimulation. However, the temporal structure of natural stimuli is hardly ever fully rhythmic but possesses certain spectral bandwidths (e.g. lip movements in speech, ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2016.11.043

    authors: Keitel C,Thut G,Gross J

    更新日期:2017-02-01 00:00:00

  • Processing of unattended facial emotions: a visual mismatch negativity study.

    abstract::Facial emotions express our internal states and are fundamental in social interactions. Here we explore whether the repetition of unattended facial emotions builds up a predictive representation of frequently encountered emotions in the visual system. Participants (n=24) were presented peripherally with facial stimuli...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2011.10.041

    authors: Stefanics G,Csukly G,Komlósi S,Czobor P,Czigler I

    更新日期:2012-02-01 00:00:00

  • Reliability and validity of MRI-based automated volumetry software relative to auto-assisted manual measurement of subcortical structures in HIV-infected patients from a multisite study.

    abstract::The automated volumetric output of FreeSurfer and Individual Brain Atlases using Statistical Parametric Mapping (IBASPM), two widely used and well published software packages, was examined for accuracy and consistency relative to auto-assisted manual (AAM) tracings (i.e., manual correction of automated output) when me...

    journal_title:NeuroImage

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.neuroimage.2010.03.033

    authors: Dewey J,Hana G,Russell T,Price J,McCaffrey D,Harezlak J,Sem E,Anyanwu JC,Guttmann CR,Navia B,Cohen R,Tate DF,HIV Neuroimaging Consortium.

    更新日期:2010-07-15 00:00:00

  • Power estimation for non-standardized multisite studies.

    abstract::A concern for researchers planning multisite studies is that scanner and T1-weighted sequence-related biases on regional volumes could overshadow true effects, especially for studies with a heterogeneous set of scanners and sequences. Current approaches attempt to harmonize data by standardizing hardware, pulse sequen...

    journal_title:NeuroImage

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.neuroimage.2016.03.051

    authors: Keshavan A,Paul F,Beyer MK,Zhu AH,Papinutto N,Shinohara RT,Stern W,Amann M,Bakshi R,Bischof A,Carriero A,Comabella M,Crane JC,D'Alfonso S,Demaerel P,Dubois B,Filippi M,Fleischer V,Fontaine B,Gaetano L,Goris A,Gr

    更新日期:2016-07-01 00:00:00

  • Trajectories of brain aging in middle-aged and older adults: regional and individual differences.

    abstract::The human brain changes with age. However, the rate and the trajectories of change vary among the brain regions and among individuals, and the reasons for these differences are unclear. In a sample of healthy middle-aged and older adults, we examined mean volume change and individual differences in the rate of change ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2010.03.020

    authors: Raz N,Ghisletta P,Rodrigue KM,Kennedy KM,Lindenberger U

    更新日期:2010-06-01 00:00:00

  • The parietal role in the sense of self-ownership with temporal discrepancy between visual and proprioceptive feedbacks.

    abstract::One hypothesis on how we recognize an image of, for example, an arm as our own is through the co-occurrence of multiple sensory feedbacks, especially visual and proprioceptive feedbacks, in this process. It has been suggested that the parietal lobe is the region where proprioceptive and visual information of one's own...

    journal_title:NeuroImage

    pub_type: 临床试验,杂志文章

    doi:10.1016/j.neuroimage.2004.10.039

    authors: Shimada S,Hiraki K,Oda I

    更新日期:2005-02-15 00:00:00

  • Mouse BOLD fMRI at ultrahigh field detects somatosensory networks including thalamic nuclei.

    abstract::Forepaw somatosensory stimulation induces neural activities in relay thalamic nuclei, the primary somatosensory cortex of forelimb (S1FL), and the secondary somatosensory cortex (S2). However, rodent fMRI studies of somatosensory stimulation have commonly reported BOLD changes only in S1FL, which may be due to side ef...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2019.03.063

    authors: Jung WB,Shim HJ,Kim SG

    更新日期:2019-07-15 00:00:00

  • Spatial distribution and longitudinal development of deep cortical sulcal landmarks in infants.

    abstract::Sulcal pits, the locally deepest points in sulci of the highly convoluted and variable cerebral cortex, are found to be spatially consistent across human adult individuals. It is suggested that sulcal pits are genetically controlled and have close relationships with functional areas. To date, the existing imaging stud...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2014.06.004

    authors: Meng Y,Li G,Lin W,Gilmore JH,Shen D

    更新日期:2014-10-15 00:00:00

  • Effective auditory-verbal encoding activates the left prefrontal and the medial temporal lobes: A generalization to illiterate subjects.

    abstract::Recent event-related FMRI studies indicate that the prefrontal (PFC) and the medial temporal lobe (MTL) regions are more active during effective encoding than during ineffective encoding. The within-subject design and the use of well-educated young college students in these studies makes it important to replicate thes...

    journal_title:NeuroImage

    pub_type: 临床试验,杂志文章

    doi:10.1006/nimg.1999.0446

    authors: Petersson KM,Reis A,Castro-Caldas A,Ingvar M

    更新日期:1999-07-01 00:00:00