Integration of iconic gestures and speech in left superior temporal areas boosts speech comprehension under adverse listening conditions.

Abstract:

:Iconic gestures are spontaneous hand movements that illustrate certain contents of speech and, as such, are an important part of face-to-face communication. This experiment targets the brain bases of how iconic gestures and speech are integrated during comprehension. Areas of integration were identified on the basis of two classic properties of multimodal integration, bimodal enhancement and inverse effectiveness (i.e., greater enhancement for unimodally least effective stimuli). Participants underwent fMRI while being presented with videos of gesture-supported sentences as well as their unimodal components, which allowed us to identify areas showing bimodal enhancement. Additionally, we manipulated the signal-to-noise ratio of speech (either moderate or good) to probe for integration areas exhibiting the inverse effectiveness property. Bimodal enhancement was found at the posterior end of the superior temporal sulcus and adjacent superior temporal gyrus (pSTS/STG) in both hemispheres, indicating that the integration of iconic gestures and speech takes place in these areas. Furthermore, we found that the left pSTS/STG specifically showed a pattern of inverse effectiveness, i.e., the neural enhancement for bimodal stimulation was greater under adverse listening conditions. This indicates that activity in this area is boosted when an iconic gesture accompanies an utterance that is otherwise difficult to comprehend. The neural response paralleled the behavioral data observed. The present data extends results from previous gesture-speech integration studies in showing that pSTS/STG plays a key role in the facilitation of speech comprehension through simultaneous gestural input.

journal_name

Neuroimage

journal_title

NeuroImage

authors

Holle H,Obleser J,Rueschemeyer SA,Gunter TC

doi

10.1016/j.neuroimage.2009.08.058

subject

Has Abstract

pub_date

2010-01-01 00:00:00

pages

875-84

issue

1

eissn

1053-8119

issn

1095-9572

pii

S1053-8119(09)00970-7

journal_volume

49

pub_type

杂志文章
  • An evaluation of four automatic methods of segmenting the subcortical structures in the brain.

    abstract::The automation of segmentation of subcortical structures in the brain is an active research area. We have comprehensively evaluated four novel methods of fully automated segmentation of subcortical structures using volumetric, spatial overlap and distance-based measures. Two methods are atlas-based - classifier fusion...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2009.05.029

    authors: Babalola KO,Patenaude B,Aljabar P,Schnabel J,Kennedy D,Crum W,Smith S,Cootes T,Jenkinson M,Rueckert D

    更新日期:2009-10-01 00:00:00

  • Insight into the fundamental trade-offs of diffusion MRI from polarization-sensitive optical coherence tomography in ex vivo human brain.

    abstract::In the first study comparing high angular resolution diffusion MRI (dMRI) in the human brain to axonal orientation measurements from polarization-sensitive optical coherence tomography (PSOCT), we compare the accuracy of orientation estimates from various dMRI sampling schemes and reconstruction methods. We find that,...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2020.116704

    authors: Jones R,Grisot G,Augustinack J,Magnain C,Boas DA,Fischl B,Wang H,Yendiki A

    更新日期:2020-07-01 00:00:00

  • Association between waist circumference and gray matter volume in 2344 individuals from two adult community-based samples.

    abstract::We analyzed the putative association between abdominal obesity (measured in waist circumference) and gray matter volume (Study of Health in Pomerania: SHIP-2, N=758) adjusted for age and gender by applying volumetric analysis and voxel-based morphometry (VBM) with VBM8 to brain magnetic resonance (MR) imaging. We soug...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2015.07.086

    authors: Janowitz D,Wittfeld K,Terock J,Freyberger HJ,Hegenscheid K,Völzke H,Habes M,Hosten N,Friedrich N,Nauck M,Domanska G,Grabe HJ

    更新日期:2015-11-15 00:00:00

  • A vascular anatomical network model of the spatio-temporal response to brain activation.

    abstract::Neuronal activity-induced changes in vascular tone and oxygen consumption result in a dynamic evolution of blood flow, volume, and oxygenation. Functional neuroimaging techniques, such as functional magnetic resonance imaging, optical imaging, and PET, provide indirect measures of the neural-induced vascular dynamics ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2007.12.061

    authors: Boas DA,Jones SR,Devor A,Huppert TJ,Dale AM

    更新日期:2008-04-15 00:00:00

  • Visual cortex responses reflect temporal structure of continuous quasi-rhythmic sensory stimulation.

    abstract::Neural processing of dynamic continuous visual input, and cognitive influences thereon, are frequently studied in paradigms employing strictly rhythmic stimulation. However, the temporal structure of natural stimuli is hardly ever fully rhythmic but possesses certain spectral bandwidths (e.g. lip movements in speech, ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2016.11.043

    authors: Keitel C,Thut G,Gross J

    更新日期:2017-02-01 00:00:00

  • Verbal episodic memory impairment in Alzheimer's disease: a combined structural and functional MRI study.

    abstract::Anatomical and functional MRI images were acquired in a group of healthy elderly subjects (n = 11) and a group of patients diagnosed with probable Alzheimer's disease, from mild to moderate severity (n = 8). During functional sessions, verbal episodic Encoding and Recognition tasks were presented to subjects. Both gro...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2004.10.045

    authors: Rémy F,Mirrashed F,Campbell B,Richter W

    更新日期:2005-03-01 00:00:00

  • Language mapping with navigated repetitive TMS: proof of technique and validation.

    abstract:OBJECTIVE:Lesion-based mapping of speech pathways has been possible only during invasive neurosurgical procedures using direct cortical stimulation (DCS). However, navigated transcranial magnetic stimulation (nTMS) may allow for lesion-based interrogation of language pathways noninvasively. Although not lesion-based, m...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2013.05.018

    authors: Tarapore PE,Findlay AM,Honma SM,Mizuiri D,Houde JF,Berger MS,Nagarajan SS

    更新日期:2013-11-15 00:00:00

  • Dynamic functional connectivity impairments in early schizophrenia and clinical high-risk for psychosis.

    abstract::Individuals at clinical high-risk (CHR) for psychosis are characterized by attenuated psychotic symptoms. Only a minority of CHR individuals convert to full-blown psychosis. Therefore, there is a strong interest in identifying neurobiological abnormalities underlying the psychosis risk syndrome. Dynamic functional con...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2017.10.022

    authors: Du Y,Fryer SL,Fu Z,Lin D,Sui J,Chen J,Damaraju E,Mennigen E,Stuart B,Loewy RL,Mathalon DH,Calhoun VD

    更新日期:2018-10-15 00:00:00

  • Internalization of MWCNTs by microglia: possible application in immunotherapy of brain tumors.

    abstract::There is a pressing need for new therapeutic, diagnostic, and drug delivery approaches for treating brain cancers. Nanotechnology offers a new method for targeted brain cancer therapy and could play a major role in gene and drug delivery. The goals of our study were to visualize in vitro ingestion, cytotoxicity, and l...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2007.03.078

    authors: Kateb B,Van Handel M,Zhang L,Bronikowski MJ,Manohara H,Badie B

    更新日期:2007-01-01 00:00:00

  • In vivo multi-slice mapping of myelin water content using T2* decay.

    abstract::Quantitative assessment of the myelin water content in the brain can substantially improve our understanding of white matter diseases such as multiple sclerosis. In this study, in vivo myelin water content was estimated using T(2)* relaxation with multi-slice acquisitions in magnetic resonance imaging (MRI). The main ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2010.04.023

    authors: Hwang D,Kim DH,Du YP

    更新日期:2010-08-01 00:00:00

  • Reconstructing spatially extended brain sources via enforcing multiple transform sparseness.

    abstract::Accurate estimation of location and extent of neuronal sources from EEG/MEG remain challenging. In the present study, a new source imaging method, i.e. variation and wavelet based sparse source imaging (VW-SSI), is proposed to better estimate cortical source locations and extents. VW-SSI utilizes the L1-norm regulariz...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2013.09.070

    authors: Zhu M,Zhang W,Dickens DL,Ding L

    更新日期:2014-02-01 00:00:00

  • Joint neuronal tuning for object form and position in the human lateral occipital complex.

    abstract::A long-standing heuristic in visual neuroscience holds that extrastriate visual cortex is parceled into a dorsal "where" pathway concerned with stimulus position and motion and a ventral "what" pathway concerned with stimulus form. Several recent studies using functional magnetic resonance imaging (fMRI), however, hav...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2012.08.043

    authors: MacEvoy SP,Yang Z

    更新日期:2012-12-01 00:00:00

  • Neighborhood deprivation, prefrontal morphology and neurocognition in late childhood to early adolescence.

    abstract:BACKGROUND:Neighborhood deprivation adversely effects neurodevelopment and cognitive function; however, mechanisms remain unexplored. Neighborhood deprivation could be particularly impactful in late childhood/early adolescence, in neural regions with protracted developmental trajectories, e.g., prefrontal cortex (PFC)....

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2020.117086

    authors: Vargas T,Damme KSF,Mittal VA

    更新日期:2020-10-15 00:00:00

  • Neuro-current response functions: A unified approach to MEG source analysis under the continuous stimuli paradigm.

    abstract::Characterizing the neural dynamics underlying sensory processing is one of the central areas of investigation in systems and cognitive neuroscience. Neuroimaging techniques such as magnetoencephalography (MEG) and Electroencephalography (EEG) have provided significant insights into the neural processing of continuous ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2020.116528

    authors: Das P,Brodbeck C,Simon JZ,Babadi B

    更新日期:2020-05-01 00:00:00

  • Audio-visual synchrony modulates the ventriloquist illusion and its neural/spatial representation in the auditory cortex.

    abstract::An essential task of our perceptual systems is to bind together the distinctive features of single objects and events into unitary percepts, even when those features are registered in different sensory modalities. In cases where auditory and visual inputs are spatially incongruent, they may still be perceived as belon...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2014.04.077

    authors: Bonath B,Noesselt T,Krauel K,Tyll S,Tempelmann C,Hillyard SA

    更新日期:2014-09-01 00:00:00

  • A random effects modelling approach to the crossing-fibre problem in tractography.

    abstract::This paper examines a Bayesian random effects modelling approach to the analysis of multiple-directions diffusion-weighted MR data, with a focus on the crossing-fibre problem. Various models were investigated including a spatial (Markov random field) model, an exchangeable model and the Besag-York-Mollie model, which ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2008.09.058

    authors: King MD,Gadian DG,Clark CA

    更新日期:2009-02-01 00:00:00

  • Dynamical causal modelling for M/EEG: spatial and temporal symmetry constraints.

    abstract::We describe the use of spatial and temporal constraints in dynamic causal modelling (DCM) of magneto- and electroencephalography (M/EEG) data. DCM for M/EEG is based on a spatiotemporal, generative model of electromagnetic brain activity. The temporal dynamics are described by neural-mass models of equivalent current ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2008.07.041

    authors: Fastenrath M,Friston KJ,Kiebel SJ

    更新日期:2009-01-01 00:00:00

  • Synchronization facilitates removal of MRI artefacts from concurrent EEG recordings and increases usable bandwidth.

    abstract::Investigating human brain function non-invasively by simultaneous EEG and fMRI measurements is gaining in popularity as more and better solutions to the inherent technical challenges emerge. We demonstrate the use of a commercially available frequency divider and phase-locking device for the purpose of synchronizing a...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2006.04.231

    authors: Mandelkow H,Halder P,Boesiger P,Brandeis D

    更新日期:2006-09-01 00:00:00

  • In vivo animation of auditory-language-induced gamma-oscillations in children with intractable focal epilepsy.

    abstract::We determined if high-frequency gamma-oscillations (50- to 150-Hz) were induced by simple auditory communication over the language network areas in children with focal epilepsy. Four children (aged 7, 9, 10 and 16 years) with intractable left-hemispheric focal epilepsy underwent extraoperative electrocorticography (EC...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2008.03.011

    authors: Brown EC,Rothermel R,Nishida M,Juhász C,Muzik O,Hoechstetter K,Sood S,Chugani HT,Asano E

    更新日期:2008-07-01 00:00:00

  • Pyramidal tract lesions and movement-associated cortical recruitment in patients with MS.

    abstract::Cortical functional changes, with the potential to limit the functional consequences of tissue injury, have been shown in patients with multiple sclerosis (MS). In this study, we assessed the influence of MS-related tissue damage of the brain portion of the left pyramidal tract on the corresponding movement-associated...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2004.05.005

    authors: Rocca MA,Gallo A,Colombo B,Falini A,Scotti G,Comi G,Filippi M

    更新日期:2004-09-01 00:00:00

  • Specialization within the ventral stream: the case for the visual word form area.

    abstract::Is there specialization for visual word recognition within the visual ventral stream of literate human adults? We review the evidence for a specialized "visual word form area" and critically examine some of the arguments recently placed against this hypothesis. Three distinct forms of specialization must be distinguis...

    journal_title:NeuroImage

    pub_type: 信件

    doi:10.1016/j.neuroimage.2003.12.049

    authors: Cohen L,Dehaene S

    更新日期:2004-05-01 00:00:00

  • RE: Warnings and caveats in brain controllability.

    abstract::The use of network control theory to analyze the organization of white matter fibers in the human brain has the potential to enable mechanistic theories of cognition, and to inform the development of novel diagnostics and treatments for neurological disease and psychiatric disorders (Gu et al., 2015). The recent artic...

    journal_title:NeuroImage

    pub_type: 评论,杂志文章

    doi:10.1016/j.neuroimage.2019.05.001

    authors: Pasqualetti F,Gu S,Bassett DS

    更新日期:2019-08-15 00:00:00

  • The neural system underlying Chinese logograph reading.

    abstract::Written Chinese as logographic script differs notably from alphabets such as English in visual form, orthography, phonology, and semantics. Thus, research on the Chinese language is important to advance our understanding of the universality and particularity of the organization of language systems in the brain. In thi...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1006/nimg.2001.0749

    authors: Tan LH,Liu HL,Perfetti CA,Spinks JA,Fox PT,Gao JH

    更新日期:2001-05-01 00:00:00

  • Mirtazapine antagonises the subjective, hormonal and neuronal effects of m-chlorophenylpiperazine (mCPP) infusion: a pharmacological-challenge fMRI (phMRI) study.

    abstract::Aberrant signalling through central 5-HT(2C) receptor pathways has been implicated in various psychiatric disorders but this has not been amenable to experimental investigation in the absence of a valid in-vivo biomarker of functional 5-HT(2C) neurotransmission. One approach is drug-challenge pharmaco-magnetic resonan...

    journal_title:NeuroImage

    pub_type: 杂志文章,随机对照试验

    doi:10.1016/j.neuroimage.2011.06.049

    authors: McKie S,Richardson P,Elliott R,Völlm BA,Dolan MC,Williams SR,Anderson IM,Deakin JF

    更新日期:2011-09-15 00:00:00

  • Sex differences in the neural correlates of child facial resemblance: an event-related fMRI study.

    abstract::Detection of genetic relatedness (i.e. kinship) impacts the social, parental, and sexual behavior of many species. In humans, self-referent phenotype matching based on facial resemblance may indicate kinship. For example, faces that resemble ours are perceived as more trustworthy and attractive. Sex differences in beh...

    journal_title:NeuroImage

    pub_type: 临床试验,杂志文章

    doi:10.1016/j.neuroimage.2004.12.037

    authors: Platek SM,Keenan JP,Mohamed FB

    更新日期:2005-05-01 00:00:00

  • A comparison of morphometric techniques for studying the shape of the corpus callosum in adolescent idiopathic scoliosis.

    abstract::The purpose of this paper is to compare volume- and boundary-based morphometry methods by applying them in the statistical analysis of 2-D shapes. The methods discussed in the first category include voxel-, deformation-, and tensor-based morphometry. The active shape model is demonstrated as an example of the second c...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2008.12.068

    authors: Wang D,Shi L,Chu WC,Paus T,Cheng JC,Heng PA

    更新日期:2009-04-15 00:00:00

  • Atypical development of white matter microstructure in adolescents with autism spectrum disorders.

    abstract::Diffusion tensor imaging (DTI) studies in adolescents with autism spectrum disorders (ASD) indicate aberrant neurodevelopment of frontal white matter (WM), potentially underlying abnormal social cognition and communication in ASD. Here, we further use tract-based spatial statistics (TBSS) to examine the developmental ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2010.01.011

    authors: Cheng Y,Chou KH,Chen IY,Fan YT,Decety J,Lin CP

    更新日期:2010-04-15 00:00:00

  • Normative database of the serotonergic system in healthy subjects using multi-tracer PET.

    abstract::The highly diverse serotonergic system with at least 16 different receptor subtypes is implicated in the pathophysiology of most neuropsychiatric disorders including affective and anxiety disorders, obsessive compulsive disorder, post-traumatic stress disorder, eating disorders, sleep disturbance, attention deficit/hy...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2012.07.001

    authors: Savli M,Bauer A,Mitterhauser M,Ding YS,Hahn A,Kroll T,Neumeister A,Haeusler D,Ungersboeck J,Henry S,Isfahani SA,Rattay F,Wadsak W,Kasper S,Lanzenberger R

    更新日期:2012-10-15 00:00:00

  • Blink- and Saccade-related suppression effects in early visual areas of the human brain: Intracranial EEG investigations during natural viewing conditions.

    abstract::Blinks and saccades, both ubiquitous in natural viewing conditions, cause rapid changes of visual inputs that are hardly consciously perceived. The neural dynamics in early visual areas of the human brain underlying this remarkable visual stability are still incompletely understood. We used electrocorticography (ECoG)...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2021.117788

    authors: Kern M,Schulze-Bonhage A,Ball T

    更新日期:2021-01-24 00:00:00

  • Multi-spectral oscillatory dynamics serving directed and divided attention.

    abstract::Attention-related amplification of neural representations of external stimuli has been well documented in the visual domain, however, research concerning the oscillatory dynamics of such directed attention is relatively sparse in humans. Specifically, it is unknown which spectrally-specific neural responses are mainly...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2020.116927

    authors: McCusker MC,Wiesman AI,Schantell MD,Eastman JA,Wilson TW

    更新日期:2020-08-15 00:00:00