Reading visually embodied meaning from the brain: Visually grounded computational models decode visual-object mental imagery induced by written text.

Abstract:

:Embodiment theory predicts that mental imagery of object words recruits neural circuits involved in object perception. The degree of visual imagery present in routine thought and how it is encoded in the brain is largely unknown. We test whether fMRI activity patterns elicited by participants reading objects' names include embodied visual-object representations, and whether we can decode the representations using novel computational image-based semantic models. We first apply the image models in conjunction with text-based semantic models to test predictions of visual-specificity of semantic representations in different brain regions. Representational similarity analysis confirms that fMRI structure within ventral-temporal and lateral-occipital regions correlates most strongly with the image models and conversely text models correlate better with posterior-parietal/lateral-temporal/inferior-frontal regions. We use an unsupervised decoding algorithm that exploits commonalities in representational similarity structure found within both image model and brain data sets to classify embodied visual representations with high accuracy (8/10) and then extend it to exploit model combinations to robustly decode different brain regions in parallel. By capturing latent visual-semantic structure our models provide a route into analyzing neural representations derived from past perceptual experience rather than stimulus-driven brain activity. Our results also verify the benefit of combining multimodal data to model human-like semantic representations.

journal_name

Neuroimage

journal_title

NeuroImage

authors

Anderson AJ,Bruni E,Lopopolo A,Poesio M,Baroni M

doi

10.1016/j.neuroimage.2015.06.093

subject

Has Abstract

pub_date

2015-10-15 00:00:00

pages

309-22

eissn

1053-8119

issn

1095-9572

pii

S1053-8119(15)00634-5

journal_volume

120

pub_type

杂志文章
  • Head motion during MRI acquisition reduces gray matter volume and thickness estimates.

    abstract::Imaging biomarkers derived from magnetic resonance imaging (MRI) data are used to quantify normal development, disease, and the effects of disease-modifying therapies. However, motion during image acquisition introduces image artifacts that, in turn, affect derived markers. A systematic effect can be problematic since...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2014.12.006

    authors: Reuter M,Tisdall MD,Qureshi A,Buckner RL,van der Kouwe AJW,Fischl B

    更新日期:2015-02-15 00:00:00

  • Does function fit structure? A ground truth for non-invasive neuroimaging.

    abstract::There are now a number of non-invasive methods to image human brain function in-vivo. However, the accuracy of these images remains unknown and can currently only be estimated through the use of invasive recordings to generate a functional ground truth. Neuronal activity follows grey matter structure and accurate esti...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2014.02.033

    authors: Stevenson C,Brookes M,López JD,Troebinger L,Mattout J,Penny W,Morris P,Hillebrand A,Henson R,Barnes G

    更新日期:2014-07-01 00:00:00

  • Neural recruitment for the production of native and novel speech sounds.

    abstract::Two primary areas of damage have been implicated in apraxia of speech (AOS) based on the time post-stroke: (1) the left inferior frontal gyrus (IFG) in acute patients, and (2) the left anterior insula (aIns) in chronic patients. While AOS is widely characterized as a disorder in motor speech planning, little is known ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2009.01.015

    authors: Moser D,Fridriksson J,Bonilha L,Healy EW,Baylis G,Baker JM,Rorden C

    更新日期:2009-06-01 00:00:00

  • Neuroimaging of the periaqueductal gray: state of the field.

    abstract::This review and meta-analysis aims at summarizing and integrating the human neuroimaging studies that report periaqueductal gray (PAG) involvement; 250 original manuscripts on human neuroimaging of the PAG were identified. A narrative review and meta-analysis using activation likelihood estimates is included. Behavior...

    journal_title:NeuroImage

    pub_type: 杂志文章,meta分析,评审

    doi:10.1016/j.neuroimage.2011.11.095

    authors: Linnman C,Moulton EA,Barmettler G,Becerra L,Borsook D

    更新日期:2012-03-01 00:00:00

  • Diffusion time dependency along the human corpus callosum and exploration of age and sex differences as assessed by oscillating gradient spin-echo diffusion tensor imaging.

    abstract::Conventional diffusion imaging uses pulsed gradient spin echo (PGSE) waveforms with diffusion times of tens of milliseconds (ms) to infer differences of white matter microstructure. The combined use of these long diffusion times with short diffusion times (<10 ​ms) enabled by oscillating gradient spin echo (OGSE) wave...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2020.116533

    authors: Tétreault P,Harkins KD,Baron CA,Stobbe R,Does MD,Beaulieu C

    更新日期:2020-04-15 00:00:00

  • Error processing network dynamics in schizophrenia.

    abstract::Current theories of cognitive dysfunction in schizophrenia emphasize an impairment in the ability of individuals suffering from this disorder to monitor their own performance, and adjust their behavior to changing demands. Detecting an error in performance is a critical component of evaluative functions that allow the...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2010.09.046

    authors: Becerril KE,Repovs G,Barch DM

    更新日期:2011-01-15 00:00:00

  • Frontoparietal theta-gamma interactions track working memory enhancement with training and tDCS.

    abstract::Despite considerable interest in enhancing, preserving, and rehabilitating working memory (WM), efforts to elicit sustained behavioral improvements have been met with limited success. Here, we paired WM training with transcranial direct current stimulation (tDCS) to the frontoparietal network over four days. Active tD...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2020.116615

    authors: Jones KT,Johnson EL,Berryhill ME

    更新日期:2020-05-01 00:00:00

  • Comparison of noise-normalized minimum norm estimates for MEG analysis using multiple resolution metrics.

    abstract::Noise-normalization has been shown to partly compensate for the localization bias towards superficial sources in minimum norm estimation. However, it has been argued that in order to make inferences for the case of multiple sources, localization properties alone are insufficient. Instead, multiple measures of resoluti...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2010.09.053

    authors: Hauk O,Wakeman DG,Henson R

    更新日期:2011-02-01 00:00:00

  • Intrinsic, dynamic and effective connectivity among large-scale brain networks modulated by oxytocin.

    abstract::The neuropeptide oxytocin is a key modulator of social-emotional behavior and its intranasal administration can influence the functional connectivity of brain networks involved in the control of attention, emotion and reward reported in humans. However, no studies have systematically investigated the effects of oxytoc...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2020.117668

    authors: Jiang X,Ma X,Geng Y,Zhao Z,Zhou F,Zhao W,Yao S,Yang S,Zhao Z,Becker B,Kendrick KM

    更新日期:2020-12-24 00:00:00

  • Simulation study of brain blood flow regulation by intra-cortical arterioles in an anatomically accurate large human vascular network: Part I: methodology and baseline flow.

    abstract::Hemodynamically based functional neuroimaging techniques, such as BOLD fMRI and PET, provide indirect measures of neuronal activity. The quantitative relationship between neuronal activity and the measured signals is not yet precisely known, with uncertainties remaining about the relative contribution by their metabol...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2010.09.032

    authors: Lorthois S,Cassot F,Lauwers F

    更新日期:2011-01-15 00:00:00

  • Increased sensitivity to age-related differences in brain functional connectivity during continuous multiple object tracking compared to resting-state.

    abstract::Age-related differences in cognitive agility vary greatly between individuals and cognitive functions. This heterogeneity is partly mirrored in individual differences in brain network connectivity as revealed using resting-state functional magnetic resonance imaging (fMRI), suggesting potential imaging biomarkers for ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2017.01.048

    authors: Dørum ES,Kaufmann T,Alnæs D,Andreassen OA,Richard G,Kolskår KK,Nordvik JE,Westlye LT

    更新日期:2017-03-01 00:00:00

  • Functional neuroanatomy of emotion: a meta-analysis of emotion activation studies in PET and fMRI.

    abstract::Neuroimagingstudies with positron emission tomography (PET) and functional magnetic resonance imaging (fMRI) have begun to describe the functional neuroanatomy of emotion. Taken separately, specific studies vary in task dimensions and in type(s) of emotion studied and are limited by statistical power and sensitivity. ...

    journal_title:NeuroImage

    pub_type: 杂志文章,meta分析

    doi:10.1006/nimg.2002.1087

    authors: Phan KL,Wager T,Taylor SF,Liberzon I

    更新日期:2002-06-01 00:00:00

  • Attention to speed of motion, speed discrimination, and task difficulty: an fMRI study.

    abstract::We studied the functional neuroanatomy of attention to speed of motion using functional magnetic resonance imaging in eight healthy subjects, who performed a speed discrimination (SID) task using a random textured pattern moving at a reference speed of 6 deg/s. During the control condition (DIM), with retinal stimulat...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1006/nimg.2000.0587

    authors: Sunaert S,Van Hecke P,Marchal G,Orban GA

    更新日期:2000-06-01 00:00:00

  • Task-induced deactivation and the "resting" state.

    abstract::Task-induced decreases in blood flow and the widespread use of "resting" baselines produced unexpected and discrepant results in early cognitive imaging studies, especially in language comprehension experiments. Here I describe from a personal perspective some of the events and thought processes leading to the first h...

    journal_title:NeuroImage

    pub_type: 历史文章,杂志文章,评审

    doi:10.1016/j.neuroimage.2011.09.026

    authors: Binder JR

    更新日期:2012-08-15 00:00:00

  • Brain activation modulated by the comprehension of normal and pseudo-word sentences of different processing demands: a functional magnetic resonance imaging study.

    abstract::Recent data from lesion and brain imaging studies have questioned the well-established assumption of a close functional-anatomic link between syntax and Broca's area and semantics and Wernicke's area. In the present study we used functional magnetic resonance imaging (fMRI) to investigate the neuroanatomical correlate...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1006/nimg.2001.1026

    authors: Röder B,Stock O,Neville H,Bien S,Rösler F

    更新日期:2002-04-01 00:00:00

  • A probabilistic MR atlas of the human cerebellum.

    abstract::The functional organization of the cerebellum is reflected in large part by the unique afferent and efferent connectivity of the individual cerebellar lobules. This functional diversity on a relatively small spatial scale makes accurate localization methods for human functional imaging and anatomical patient-based res...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2009.01.045

    authors: Diedrichsen J,Balsters JH,Flavell J,Cussans E,Ramnani N

    更新日期:2009-05-15 00:00:00

  • Effect of paroxetine and bupropion on human resting brain perfusion: an arterial spin labeling study.

    abstract::Due to the physiological coupling with metabolism, brain perfusion is a potential aid in understanding the effects of medication on neural activity in vivo. In this study, arterial spin labeling (ASL) was used to quantify the effect of seven days of administration of paroxetine (20 mg), bupropion (150 mg) on cerebral ...

    journal_title:NeuroImage

    pub_type: 杂志文章,随机对照试验

    doi:10.1016/j.neuroimage.2012.03.014

    authors: Viviani R,Abler B,Seeringer A,Stingl JC

    更新日期:2012-07-16 00:00:00

  • Functional connectivity of the human insular cortex during noxious and innocuous thermal stimulation.

    abstract::The insula plays a key role in brain processing of noxious and innocuous thermal stimuli. The anterior and the posterior portions of the insular cortex are involved in different ways in nociceptive and thermoceptive processing. Therefore, their stimulus-specific functional connectivity may also differ. Here we used fu...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2010.09.012

    authors: Peltz E,Seifert F,DeCol R,Dörfler A,Schwab S,Maihöfner C

    更新日期:2011-01-15 00:00:00

  • The effect of linear mixing in the EEG on Hurst exponent estimation.

    abstract::Although the long-range temporal correlation (LRTC) of the amplitude fluctuations of neuronal EEG/MEG oscillations is widely acknowledged, the majority of studies to date have been performed in sensor space, disregarding the mixing effects implied by volume conduction and confounding noise. While the effect of mixing ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2014.05.041

    authors: Blythe DA,Haufe S,Müller KR,Nikulin VV

    更新日期:2014-10-01 00:00:00

  • Stochastic synchronization of dynamics on the human connectome.

    abstract::Synchronization is a collective mechanism by which oscillatory networks achieve their functions. Factors driving synchronization include the network's topological and dynamical properties. However, how these factors drive the emergence of synchronization in the presence of potentially disruptive external inputs like s...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2021.117738

    authors: Pang JC,Gollo LL,Roberts JA

    更新日期:2021-01-14 00:00:00

  • Removing inter-subject technical variability in magnetic resonance imaging studies.

    abstract::Magnetic resonance imaging (MRI) intensities are acquired in arbitrary units, making scans non-comparable across sites and between subjects. Intensity normalization is a first step for the improvement of comparability of the images across subjects. However, we show that unwanted inter-scan variability associated with ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2016.02.036

    authors: Fortin JP,Sweeney EM,Muschelli J,Crainiceanu CM,Shinohara RT,Alzheimer's Disease Neuroimaging Initiative.

    更新日期:2016-05-15 00:00:00

  • Combining MRI and CSF measures for classification of Alzheimer's disease and prediction of mild cognitive impairment conversion.

    abstract::The suggested revision of the NINCDS-ADRDA criterion for the diagnosis of Alzheimer's disease (AD) includes at least one abnormal biomarker among magnetic resonance imaging (MRI), positron emission tomography (PET) and cerebrospinal fluid (CSF). We aimed to investigate if the combination of baseline MRI and CSF could ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2012.04.056

    authors: Westman E,Muehlboeck JS,Simmons A

    更新日期:2012-08-01 00:00:00

  • Association between GAB2 haplotype and higher glucose metabolism in Alzheimer's disease-affected brain regions in cognitively normal APOEε4 carriers.

    abstract::In a genome-wide association study (GWAS) of late-onset Alzheimer's disease (AD), we found an association between common haplotypes of the GAB2 gene and AD risk in carriers of the apolipoprotein E (APOE) ε4 allele, the major late-onset AD susceptibility gene. We previously proposed the use of fluorodeoxyglucose positr...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2010.09.066

    authors: Liang WS,Chen K,Lee W,Sidhar K,Corneveaux JJ,Allen AN,Myers A,Villa S,Meechoovet B,Pruzin J,Bandy D,Fleisher AS,Langbaum JB,Huentelman MJ,Jensen K,Dunckley T,Caselli RJ,Kaib S,Reiman EM

    更新日期:2011-02-01 00:00:00

  • Investigating resting-state functional connectivity in the cervical spinal cord at 3T.

    abstract::The study of spontaneous fluctuations in the blood-oxygen-level-dependent (BOLD) signal has recently been extended from the brain to the spinal cord. Two ultra-high field functional magnetic resonance imaging (fMRI) studies in humans have provided evidence for reproducible resting-state connectivity between the dorsal...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2016.12.072

    authors: Eippert F,Kong Y,Winkler AM,Andersson JL,Finsterbusch J,Büchel C,Brooks JCW,Tracey I

    更新日期:2017-02-15 00:00:00

  • Computational modelling of movement-related beta-oscillatory dynamics in human motor cortex.

    abstract::Oscillatory activity in the beta range, in human primary motor cortex (M1), shows interesting dynamics that are tied to behaviour and change systematically in disease. To investigate the pathophysiology underlying these changes, we must first understand how changes in beta activity are caused in healthy subjects. We t...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2016.02.078

    authors: Bhatt MB,Bowen S,Rossiter HE,Dupont-Hadwen J,Moran RJ,Friston KJ,Ward NS

    更新日期:2016-06-01 00:00:00

  • Sensory stimulation activates both motor and sensory components of the swallowing system.

    abstract::Volitional swallowing in humans involves the coordination of both brainstem and cerebral swallowing control regions. Peripheral sensory inputs are necessary for safe and efficient swallowing, and their importance to the patterned components of swallowing has been demonstrated. However, the role of sensory inputs to th...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2008.04.234

    authors: Lowell SY,Poletto CJ,Knorr-Chung BR,Reynolds RC,Simonyan K,Ludlow CL

    更新日期:2008-08-01 00:00:00

  • It takes two to tango: Suppression of task-irrelevant features requires (spatial) competition.

    abstract::In a recent electrophysiological study, we reported on global facilitation but local suppression of color stimuli in feature-based attention in human early visual cortex. Subjects attended to one of two centrally located superimposed red/blue random dot kinematograms (RDKs). Task-irrelevant single RDKs in the same col...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2018.05.073

    authors: Müller MM,Gundlach C,Forschack N,Brummerloh B

    更新日期:2018-09-01 00:00:00

  • Longitudinal stability of MRI for mapping brain change using tensor-based morphometry.

    abstract::Measures of brain change can be computed from sequential MRI scans, providing valuable information on disease progression, e.g., for patient monitoring and drug trials. Tensor-based morphometry (TBM) creates maps of these brain changes, visualizing the 3D profile and rates of tissue growth or atrophy, but its sensitiv...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2005.12.013

    authors: Leow AD,Klunder AD,Jack CR Jr,Toga AW,Dale AM,Bernstein MA,Britson PJ,Gunter JL,Ward CP,Whitwell JL,Borowski BJ,Fleisher AS,Fox NC,Harvey D,Kornak J,Schuff N,Studholme C,Alexander GE,Weiner MW,Thompson PM,ADNI Pre

    更新日期:2006-06-01 00:00:00

  • Exploring experimental autoimmune optic neuritis using multimodal imaging.

    abstract:BACKGROUND:Neuro-axonal injury is a key contributor to non-reversible long-term disability in multiple sclerosis (MS). However, the underlying mechanisms are not yet fully understood. Visual impairment is common among MS patients, in which episodes of optic neuritis (ON) are often followed by structural retinal damage ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2018.04.004

    authors: Manogaran P,Walker-Egger C,Samardzija M,Waschkies C,Grimm C,Rudin M,Schippling S

    更新日期:2018-07-15 00:00:00

  • Individual alpha peak frequency is related to latent factors of general cognitive abilities.

    abstract::Some eighty years after the discovery of the human electroencephalogram (EEG) and its dominant rhythm, alpha (~10Hz), the neurophysiological functions and behavioral correlates of alpha oscillations are still under debate. Similarly, the biological mechanisms contributing to the general factor of intelligence, or g, h...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2013.04.059

    authors: Grandy TH,Werkle-Bergner M,Chicherio C,Lövdén M,Schmiedek F,Lindenberger U

    更新日期:2013-10-01 00:00:00