The EEG and fMRI signatures of neural integration: An investigation of meaningful gestures and corresponding speech.

Abstract:

:One of the key features of human interpersonal communication is our ability to integrate information communicated by speech and accompanying gestures. However, it is still not fully understood how this essential combinatory process is represented in the human brain. Functional magnetic resonance imaging (fMRI) studies have unanimously attested the relevance of activation in the posterior superior temporal sulcus/middle temporal gyrus (pSTS/MTG), while electroencephalography (EEG) studies have shown oscillatory activity in specific frequency bands to be associated with multisensory integration. In the current study, we used fMRI and EEG to separately investigate the anatomical and oscillatory neural signature of integrating intrinsically meaningful gestures (IMG; e.g. "Thumbs-up gesture") and corresponding speech (e.g., "The actor did a good job"). In both the fMRI (n =2 0) and EEG (n = 20) study, participants were presented with videos of an actor either: performing IMG in the context of a German sentence (GG), IMG in the context of a Russian (as a foreign language) sentence (GR), or speaking an isolated German sentence without gesture (SG). The results of the fMRI experiment confirmed that gesture-speech processing of IMG activates the posterior MTG (GG>GR∩GG>SG). In the EEG experiment we found that the identical integration process (GG>GR∩GG>SG) is related to a centrally-distributed alpha (7-13 Hz) power decrease within 700-1400 ms post-onset of the critical word. These new findings suggest that BOLD response increase in the pMTG and alpha power decrease represent the neural correlates of integrating intrinsically meaningful gestures with their corresponding speech.

journal_name

Neuropsychologia

journal_title

Neuropsychologia

authors

He Y,Gebhardt H,Steines M,Sammer G,Kircher T,Nagels A,Straube B

doi

10.1016/j.neuropsychologia.2015.04.018

subject

Has Abstract

pub_date

2015-06-01 00:00:00

pages

27-42

eissn

0028-3932

issn

1873-3514

pii

S0028-3932(15)30001-4

journal_volume

72

pub_type

杂志文章
  • The effect of future time perspective on delay discounting is mediated by the gray matter volume of vmPFC.

    abstract::Although several previous studies have shown that individuals' attitude towards time could affect their intertemporal preference, little is known about the neural basis of the relation between time perspective (TP) and delay discounting. In the present study, we quantified the gray matter (GM) cortical volume using vo...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2017.06.021

    authors: Guo Y,Chen Z,Feng T

    更新日期:2017-07-28 00:00:00

  • Is handwriting posture associated with differences in motor control? An analysis of asymmetries in the readiness potential.

    abstract::Levy and Reid's [1] hypothesis that persons who write using the inverted posture have ipsilateral control of distal limb movements, particularly those involved in handwriting, was tested in three experiments in which asymmetries in the readiness potential (RP) were measured. In the first experiment, each subject execu...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/0028-3932(82)90107-5

    authors: Bashore TR,McCarthy G,Heffley EF 3rd,Clapman RM,Donchin E

    更新日期:1982-01-01 00:00:00

  • Musical expertise affects neural bases of letter recognition.

    abstract::It is known that early music learning (playing of an instrument) modifies functional brain structure (both white and gray matter) and connectivity, especially callosal transfer, motor control/coordination and auditory processing. We compared visual processing of notes and words in 15 professional musicians and 15 cont...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2012.12.001

    authors: Proverbio AM,Manfredi M,Zani A,Adorni R

    更新日期:2013-02-01 00:00:00

  • Induced letter migrations between words and what they reveal about the orthographic-visual analyzer.

    abstract::When skilled readers read briefly-presented word pairs, they produce between-word errors, in which letters migrate between neighboring words (e.g., mild wind can be misread as wild mind). Such errors are also produced by individuals with attentional dyslexia, even without time limitation. In this study, we tested seve...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2010.11.026

    authors: Shetreet E,Friedmann N

    更新日期:2011-02-01 00:00:00

  • The neural mechanism underlying the effects of preceding contexts on current categorization decisions.

    abstract::Preceding contexts strongly influence current decision-making. To elucidate the neural mechanism that underlies this phenomenon, magnetoencephalographic signals were recorded while participants performed a binary categorization task on a sequence of facial expressions. The behavioral data indicated that the categoriza...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2014.11.003

    authors: Hsu SM

    更新日期:2015-01-01 00:00:00

  • Modeling neural and self-reported factors of affective distress in the relationship between pain and working memory in healthy individuals.

    abstract::The relationship between pain and cognition has primarily been investigated in patients with chronic pain and healthy participants undergoing experimental pain. Recently, there has been interest in understanding the disruptive effects of non-experimental pain in otherwise healthy individuals. Recent studies suggest th...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2021.107766

    authors: Anderson SR,Witkin JE,Bolt T,Llabre MM,Ashton-James CE,Reynolds Losin EA

    更新日期:2021-01-24 00:00:00

  • Attentional and sensory effects of lowered levels of intrinsic alertness.

    abstract::Low levels of intrinsic alertness are associated with lateralized performance in visual tasks, similar to neglect of the left (ipsilesional) visual hemi-field. However, it is unclear whether reduced alertness produces a specific lateralization of spatial-attentional processes in terms of the prioritization of right- o...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2009.08.004

    authors: Matthias E,Bublak P,Costa A,Müller HJ,Schneider WX,Finke K

    更新日期:2009-12-01 00:00:00

  • Semantic P600-but not N400-effects index crosslinguistic variability in speakers' expectancies for expression of motion.

    abstract::The expression of motion shows strong crosslinguistic variability; however, less is known about speakers' expectancies for lexicalizations of motion at the neural level. We examined event-related brain potentials (ERPs) in native English or Spanish speakers while they read grammatical sentences describing animations i...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2020.107638

    authors: Emerson SN,Conway CM,Özçalışkan Ş

    更新日期:2020-12-01 00:00:00

  • Lexical access versus lexical decision processes for auditory, visual, and audiovisual items: Insights from behavioral and neural measures.

    abstract::In two experiments, we investigated the relationship between lexical access processes, and processes that are specifically related to making lexical decisions. In Experiment 1, participants performed a standard lexical decision task in which they had to respond as quickly and as accurately as possible to visual (writt...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2019.107305

    authors: López Zunini RA,Baart M,Samuel AG,Armstrong BC

    更新日期:2020-02-03 00:00:00

  • Current issues in the neuropsychology of image generation.

    abstract::Image generation is the process by which long-term memory knowledge of the visual appearance of objects or scenes is used to create a short-term, percept-like image. In this article I will review recent neuropsychological evidence relevant to two questions about image generation: First, is there a distinct component o...

    journal_title:Neuropsychologia

    pub_type: 杂志文章,评审

    doi:10.1016/0028-3932(95)00075-e

    authors: Farah MJ

    更新日期:1995-11-01 00:00:00

  • Phonological working memory and FOXP2.

    abstract::The discovery and description of the affected members of the KE family (aKE) initiated research on how genes enable the unique human trait of speech and language. Many aspects of this genetic influence on speech-related cognitive mechanisms are still elusive, e.g. if and how cognitive processes not directly involved i...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2017.11.027

    authors: Schulze K,Vargha-Khadem F,Mishkin M

    更新日期:2018-01-08 00:00:00

  • Interference between number processing and line bisection: a methodology.

    abstract::It is usually thought that numbers are represented on a "mental number line" on which the progression of smaller to larger numbers occurs from left to right suggesting a spatial representation of numbers. Reciprocally, number processing has been shown to influence space representation [Fischer, M. (2001). Number proce...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2004.06.027

    authors: Calabria M,Rossetti Y

    更新日期:2005-01-01 00:00:00

  • Effects of symbol type and numerical distance on the human event-related potential.

    abstract::This study investigated the influence of the symbol type and numerical distance of numbers on the amplitudes and peak latencies of event-related potentials (ERPs). Our aim was to (1) determine the point in time of magnitude information access in visual number processing; and (2) identify at what stage the advantage of...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2009.09.005

    authors: Jiang T,Qiao S,Li J,Cao Z,Gao X,Song Y,Xue G,Dong Q,Chen C

    更新日期:2010-01-01 00:00:00

  • The effects of TMS over dorsolateral prefrontal cortex on trans-saccadic memory of multiple objects.

    abstract::Humans typically make several rapid eye movements (saccades) per second. It is thought that visual working memory can retain and spatially integrate three to four objects or features across each saccade but little is known about this neural mechanism. Previously we showed that transcranial magnetic stimulation (TMS) t...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2014.08.025

    authors: Tanaka LL,Dessing JC,Malik P,Prime SL,Crawford JD

    更新日期:2014-10-01 00:00:00

  • Detecting subtle facial emotion recognition deficits in high-functioning Autism using dynamic stimuli of varying intensities.

    abstract::Autism Spectrum Disorders (ASD) are characterised by social and communication impairment, yet evidence for deficits in the ability to recognise facial expressions of basic emotions is conflicting. Many studies reporting no deficits have used stimuli that may be too simple (with associated ceiling effects), for example...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2010.03.008

    authors: Law Smith MJ,Montagne B,Perrett DI,Gill M,Gallagher L

    更新日期:2010-07-01 00:00:00

  • Action verbs and the primary motor cortex: a comparative TMS study of silent reading, frequency judgments, and motor imagery.

    abstract::Single pulse transcranial magnetic stimulation (TMS) was applied to the hand area of the left primary motor cortex or, as a control, to the vertex (STIMULATION: TMS(M1) vs. TMS(vertex)) while right-handed volunteers silently read verbs related to hand actions. We examined three different tasks and time points for stim...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2008.01.015

    authors: Tomasino B,Fink GR,Sparing R,Dafotakis M,Weiss PH

    更新日期:2008-01-01 00:00:00

  • Grip-dependent cortico-spinal excitability during grasping imagination and execution.

    abstract::Studies converge in indicating a substantial similarity of the rules and mechanisms underlying execution, observation and imagery of actions, along with a large overlapping of their neural substrates. Recent transcranial magnetic stimulation (TMS) studies have demonstrated a muscle-specific facilitation of the observe...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2011.04.019

    authors: Cesari P,Pizzolato F,Fiorio M

    更新日期:2011-06-01 00:00:00

  • A matched filter hypothesis for cognitive control.

    abstract::The prefrontal cortex exerts top-down influences on several aspects of higher-order cognition by functioning as a filtering mechanism that biases bottom-up sensory information toward a response that is optimal in context. However, research also indicates that not all aspects of complex cognition benefit from prefronta...

    journal_title:Neuropsychologia

    pub_type: 杂志文章,评审

    doi:10.1016/j.neuropsychologia.2013.10.021

    authors: Chrysikou EG,Weber MJ,Thompson-Schill SL

    更新日期:2014-09-01 00:00:00

  • Isolated numerical skills in posterior cortical atrophy--an fMRI study.

    abstract::Posterior cortical atrophy (PCA) is characterized by bilateral parieto-occipito-temporal atrophy and hypometabolism. Neuropsychological impairments include complex visual disturbances, alexia, agraphia, finger agnosia, right-left disorientation and dyscalculia. A recent case study reported severe numerical deficits wi...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2006.02.007

    authors: Delazer M,Benke T,Trieb T,Schocke M,Ischebeck A

    更新日期:2006-01-01 00:00:00

  • It does not look odd to me: perceptual impairments and eye movements in amnesic patients with medial temporal lobe damage.

    abstract::Studies of people with memory impairments have shown that a specific set of brain structures in the medial temporal lobe (MTL) is vital for memory function. However, whether these structures have a role outside of memory remains contentious. Recent studies of amnesic patients with damage to two structures within the M...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2012.11.003

    authors: Erez J,Lee AC,Barense MD

    更新日期:2013-01-01 00:00:00

  • Theta-band functional connectivity in the dorsal fronto-parietal network predicts goal-directed attention.

    abstract::Functional imaging studies have identified a dorsal fronto-parietal network whose activity reflects shifts of attention in space and is sensitive to the behavioural relevance of stimuli. In patients with severe deficits of spatial attention this network is often structurally preserved. Here, we show that resting-state...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2016.07.012

    authors: Fellrath J,Mottaz A,Schnider A,Guggisberg AG,Ptak R

    更新日期:2016-11-01 00:00:00

  • Facial expression recognition in people with medicated and unmedicated Parkinson's disease.

    abstract::Recognition of facial expressions of emotion was investigated in people with medicated and unmedicated Parkinson's disease (PD) and matched controls (unmedicated PD, n=16; medicated PD, n=20; controls, n=40). Participants in the medicated group showed some visual impairment (impaired contrast sensitivity) and performe...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/s0028-3932(02)00295-6

    authors: Sprengelmeyer R,Young AW,Mahn K,Schroeder U,Woitalla D,Büttner T,Kuhn W,Przuntek H

    更新日期:2003-01-01 00:00:00

  • A developmental window into trade-offs in executive function: the case of task switching versus response inhibition in 6-year-olds.

    abstract::Good executive function has been linked to many positive outcomes in academic performance, health, and social competence. However, some aspects of executive function may interfere with other cognitive processes. Childhood provides a unique test case for investigating such cognitive trade-offs, given the dramatic failu...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2014.04.016

    authors: Blackwell KA,Chatham CH,Wiseheart M,Munakata Y

    更新日期:2014-09-01 00:00:00

  • Recognition of unfamiliar faces in prosopagnosia.

    abstract::Prosopagnosia is clinically defined as a specific and extreme inability to recognize familiar faces. However, doubts have been expressed concerning prosopagnosics' preserved ability to recognize unfamiliar faces and to make other within category discriminations. The present study pursues these doubts. If recognition o...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/0028-3932(90)90051-o

    authors: Davidoff J,Landis T

    更新日期:1990-01-01 00:00:00

  • Emotion and cognition and the amygdala: from "what is it?" to "what's to be done?".

    abstract::The amygdala is a fascinating, complex structure that lies at the center of much of our current thinking about emotion. Here, I will review data that suggest that the amygdala is involved in several processes linked to determining what a stimulus is and what the organism should therefore do - the two questions that ar...

    journal_title:Neuropsychologia

    pub_type: 杂志文章,评审

    doi:10.1016/j.neuropsychologia.2010.06.038

    authors: Pessoa L

    更新日期:2010-10-01 00:00:00

  • Visual motion serves but is not under the purview of the dorsal pathway.

    abstract::Visual motion processing is often attributed to the dorsal visual pathway despite visual motion's involvement in almost all visual functions. Furthermore, some visual motion tasks critically depend on the structural integrity of regions outside the dorsal pathway. Here, based on numerous studies, I propose that visual...

    journal_title:Neuropsychologia

    pub_type: 杂志文章,评审

    doi:10.1016/j.neuropsychologia.2016.07.018

    authors: Gilaie-Dotan S

    更新日期:2016-08-01 00:00:00

  • Effects of transcranial direct current stimulation (tDCS) on behaviour and electrophysiology of language production.

    abstract::Excitatory anodal transcranial direct current stimulation (A-tDCS) over the left dorsal prefrontal cortex (DPFC) has been shown to improve language production. The present study examined neurophysiological underpinnings of this effect. In a single-blinded within-subject design, we traced effects of A-tDCS compared to ...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2011.10.015

    authors: Wirth M,Rahman RA,Kuenecke J,Koenig T,Horn H,Sommer W,Dierks T

    更新日期:2011-12-01 00:00:00

  • Memory signals from the thalamus: early thalamocortical phase synchronization entrains gamma oscillations during long-term memory retrieval.

    abstract::The thalamus is believed to be a key node in human memory networks, however, very little is known about its real-time functional role. Here we examined the dynamics of thalamocortical communication during long-term episodic memory retrieval in two experiments. In experiment 1, intrathalamic and surface EEG was recorde...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2012.08.023

    authors: Staudigl T,Zaehle T,Voges J,Hanslmayr S,Esslinger C,Hinrichs H,Schmitt FC,Heinze HJ,Richardson-Klavehn A

    更新日期:2012-12-01 00:00:00

  • Abnormal global processing along the dorsal visual pathway in autism: a possible mechanism for weak visuospatial coherence?

    abstract::Frith and Happe (Frith, U., & Happe, F. (1994). Autism: Beyond theory of mind. Cognition, 50, 115-132) argue that individuals with autism exhibit 'weak central coherence': an inability to integrate elements of information into coherent wholes. Some authors have speculated that a high-level impairment might be present ...

    journal_title:Neuropsychologia

    pub_type: 临床试验,杂志文章

    doi:10.1016/j.neuropsychologia.2004.10.003

    authors: Pellicano E,Gibson L,Maybery M,Durkin K,Badcock DR

    更新日期:2005-01-01 00:00:00

  • Retinotopic effects during spatial audio-visual integration.

    abstract::The successful integration of visual and auditory stimuli requires information about whether visual and auditory signals originate from corresponding places in the external world. Here we report crossmodal effects of spatially congruent and incongruent audio-visual (AV) stimulation. Visual and auditory stimuli were pr...

    journal_title:Neuropsychologia

    pub_type: 杂志文章

    doi:10.1016/j.neuropsychologia.2006.05.018

    authors: Meienbrock A,Naumer MJ,Doehrmann O,Singer W,Muckli L

    更新日期:2007-02-01 00:00:00