The Principle of Inverse Effectiveness in Audiovisual Speech Perception.

Abstract:

:We assessed how synchronous speech listening and lipreading affects speech recognition in acoustic noise. In simple audiovisual perceptual tasks, inverse effectiveness is often observed, which holds that the weaker the unimodal stimuli, or the poorer their signal-to-noise ratio, the stronger the audiovisual benefit. So far, however, inverse effectiveness has not been demonstrated for complex audiovisual speech stimuli. Here we assess whether this multisensory integration effect can also be observed for the recognizability of spoken words. To that end, we presented audiovisual sentences to 18 native-Dutch normal-hearing participants, who had to identify the spoken words from a finite list. Speech-recognition performance was determined for auditory-only, visual-only (lipreading), and auditory-visual conditions. To modulate acoustic task difficulty, we systematically varied the auditory signal-to-noise ratio. In line with a commonly observed multisensory enhancement on speech recognition, audiovisual words were more easily recognized than auditory-only words (recognition thresholds of -15 and -12 dB, respectively). We here show that the difficulty of recognizing a particular word, either acoustically or visually, determines the occurrence of inverse effectiveness in audiovisual word integration. Thus, words that are better heard or recognized through lipreading, benefit less from bimodal presentation. Audiovisual performance at the lowest acoustic signal-to-noise ratios (45%) fell below the visual recognition rates (60%), reflecting an actual deterioration of lipreading in the presence of excessive acoustic noise. This suggests that the brain may adopt a strategy in which attention has to be divided between listening and lipreading.

journal_name

Front Hum Neurosci

authors

van de Rijt LPH,Roye A,Mylanus EAM,van Opstal AJ,van Wanrooij MM

doi

10.3389/fnhum.2019.00335

subject

Has Abstract

pub_date

2019-09-26 00:00:00

pages

335

issn

1662-5161

journal_volume

13

pub_type

杂志文章
  • Sex Differences in Categorical Adaptation for Faces and Chinese Characters during Early Perceptual Processing.

    abstract::Previous event-related potential studies support sex differences in the N170 response during face and word processing; however, it remains unclear whether N170 categorical adaptation for faces and words is different between women and men. Using an adaptation paradigm, in which an adaptor and subsequent test stimulus a...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2017.00656

    authors: Zhu C,Ma X,Ji L,Chen S,Cao X

    更新日期:2018-01-12 00:00:00

  • Situating emotional experience.

    abstract::Psychological construction approaches to emotion suggest that emotional experience is situated and dynamic. Fear, for example, is typically studied in a physical danger context (e.g., threatening snake), but in the real world, it often occurs in social contexts, especially those involving social evaluation (e.g., publ...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2013.00764

    authors: Wilson-Mendenhall CD,Barrett LF,Barsalou LW

    更新日期:2013-11-26 00:00:00

  • Corrigendum: Who Deserves My Trust? Cue-Elicited Feedback Negativity Tracks Reputation Learning in Repeated Social Interactions.

    abstract::[This corrects the article DOI: 10.3389/fnhum.2017.00307.]. ...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章,已发布勘误

    doi:10.3389/fnhum.2019.00083

    authors: Li D,Meng L,Ma Q

    更新日期:2019-03-06 00:00:00

  • Affective Processing in Non-invasive Brain Stimulation Over Prefrontal Cortex.

    abstract::The prefrontal cortex (PFC) is the most frequently targeted brain region by non-invasive brain stimulation (NBS) studies. Non-invasively stimulating the PFC has been shown to both modulate affective processing and improve the clinical symptoms of several psychiatric disorders, such as depression and schizophrenia. The...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章,评审

    doi:10.3389/fnhum.2017.00439

    authors: Liu W,Leng YS,Zou XH,Cheng ZQ,Yang W,Li BJ

    更新日期:2017-09-04 00:00:00

  • The neural basis of implicit perceptual sequence learning.

    abstract::The present fMRI study investigated the neural areas involved in implicit perceptual sequence learning. To obtain more insight in the functional contributions of the brain areas, we tracked both the behavioral and neural time course of the learning process, using a perceptual serial color matching task. Next, to inves...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2011.00137

    authors: Gheysen F,Van Opstal F,Roggeman C,Van Waelvelde H,Fias W

    更新日期:2011-11-11 00:00:00

  • Change in Brainstem Gray Matter Concentration Following a Mindfulness-Based Intervention is Correlated with Improvement in Psychological Well-Being.

    abstract::Individuals can improve their levels of psychological well-being (PWB) through utilization of psychological interventions, including the practice of mindfulness meditation, which is defined as the non-judgmental awareness of experiences in the present moment. We recently reported that an 8-week-mindfulness-based stres...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2014.00033

    authors: Singleton O,Hölzel BK,Vangel M,Brach N,Carmody J,Lazar SW

    更新日期:2014-02-18 00:00:00

  • Examining the role of the temporo-parietal network in memory, imagery, and viewpoint transformations.

    abstract::The traditional view of the medial temporal lobe (MTL) focuses on its role in episodic memory. However, some of the underlying functions of the MTL can be ascertained from its wider role in supporting spatial cognition in concert with parietal and prefrontal regions. The MTL is strongly implicated in the formation of ...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2014.00709

    authors: Dhindsa K,Drobinin V,King J,Hall GB,Burgess N,Becker S

    更新日期:2014-09-16 00:00:00

  • Magnitude Processing in the Brain: An fMRI Study of Time, Space, and Numerosity as a Shared Cortical System.

    abstract::Continuous dimensions, such as time, space, and numerosity, have been suggested to be subserved by common neurocognitive mechanisms. Neuroimaging studies that have investigated either one or two dimensions simultaneously have consistently identified neural correlates in the parietal cortex of the brain. However, studi...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2016.00500

    authors: Skagerlund K,Karlsson T,Träff U

    更新日期:2016-10-05 00:00:00

  • Flat vs. Expressive Storytelling: Young Children's Learning and Retention of a Social Robot's Narrative.

    abstract::Prior research with preschool children has established that dialogic or active book reading is an effective method for expanding young children's vocabulary. In this exploratory study, we asked whether similar benefits are observed when a robot engages in dialogic reading with preschoolers. Given the established effec...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2017.00295

    authors: Kory Westlund JM,Jeong S,Park HW,Ronfard S,Adhikari A,Harris PL,DeSteno D,Breazeal CL

    更新日期:2017-06-07 00:00:00

  • A predictive nature for tactile awareness? Insights from damaged and intact central-nervous-system functioning.

    abstract::In the present paper, we will attempt to gain hints regarding the nature of tactile awareness in humans. At first, we will review some recent literature showing that an actual tactile experience can emerge in absence of any tactile stimulus (e.g., tactile hallucinations, tactile illusions). According to the current mo...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2015.00287

    authors: Pia L,Garbarini F,Burin D,Fossataro C,Berti A

    更新日期:2015-05-19 00:00:00

  • Behind the Scenes of Developmental Language Disorder: Time to Call Neuropsychology Back on Stage.

    abstract::Although the Developmental Language Disorder (DLD), also known as Specific Language Impairment in children has been the focus of unceasing scientific attention for decades, the nature and mechanisms of this disorder remain unclear. Most importantly, we still cannot reliably identify children requiring urgent intervent...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章,评审

    doi:10.3389/fnhum.2018.00517

    authors: Tomas E,Vissers C

    更新日期:2019-01-09 00:00:00

  • Examining neural correlates of skill acquisition in a complex videogame training program.

    abstract::Acquisition of complex skills is a universal feature of human behavior that has been conceptualized as a process that starts with intense resource dependency, requires effortful cognitive control, and ends in relative automaticity on the multi-faceted task. The present study examined the effects of different theoretic...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2012.00115

    authors: Prakash RS,De Leon AA,Mourany L,Lee H,Voss MW,Boot WR,Basak C,Fabiani M,Gratton G,Kramer AF

    更新日期:2012-05-15 00:00:00

  • Systems, Subjects, Sessions: To What Extent Do These Factors Influence EEG Data?

    abstract::Lab-based electroencephalography (EEG) techniques have matured over decades of research and can produce high-quality scientific data. It is often assumed that the specific choice of EEG system has limited impact on the data and does not add variance to the results. However, many low cost and mobile EEG systems are now...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2017.00150

    authors: Melnik A,Legkov P,Izdebski K,Kärcher SM,Hairston WD,Ferris DP,König P

    更新日期:2017-03-30 00:00:00

  • The Relationship between Virtual Self Similarity and Social Anxiety.

    abstract::In virtual reality (VR), it is possible to embody avatars that are dissimilar to the physical self. We examined whether embodying a dissimilar self in VR would decrease anxiety in a public speaking situation. We report the results of an observational pilot study and two laboratory experiments. In the pilot study (N = ...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2014.00944

    authors: Aymerich-Franch L,Kizilcec RF,Bailenson JN

    更新日期:2014-11-19 00:00:00

  • The influence of the amplitude of low-frequency fluctuations on resting-state functional connectivity.

    abstract::Studies of brain functional connectivity have provided a better understanding of organization and integration of large-scale brain networks. Functional connectivity using resting-state functional magnetic resonance imaging (fMRI) is typically based upon the correlations of the low-frequency fluctuation of fMRI signals...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2013.00118

    authors: Di X,Kim EH,Huang CC,Tsai SJ,Lin CP,Biswal BB

    更新日期:2013-04-02 00:00:00

  • Habits: bridging the gap between personhood and personal identity.

    abstract::In philosophy, the criteria for personhood (PH) at a specific point in time (synchronic), and the necessary and sufficient conditions of personal identity (PI) over time (diachronic) are traditionally separated. Hence, the transition between both timescales of a person's life remains largely unclear. Personal habits r...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2014.00330

    authors: Wagner NF,Northoff G

    更新日期:2014-05-21 00:00:00

  • More creative through positive mood? Not everyone!

    abstract::It is commonly assumed that positive mood improves human creativity and that the neurotransmitter dopamine might mediate this association. However, given the non-linear relation between dopamine and flexibility in divergent thinking (Akbari Chermahini and Hommel, 2010), the impact of mood on divergent kinds of creativ...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2012.00319

    authors: Akbari Chermahini S,Hommel B

    更新日期:2012-11-26 00:00:00

  • Effects of arousal on cognitive control: empirical tests of the conflict-modulated Hebbian-learning hypothesis.

    abstract::An increasing number of empirical phenomena that were previously interpreted as a result of cognitive control, turn out to reflect (in part) simple associative-learning effects. A prime example is the proportion congruency effect, the finding that interference effects (such as the Stroop effect) decrease as the propor...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2014.00023

    authors: Brown SB,van Steenbergen H,Kedar T,Nieuwenhuis S

    更新日期:2014-01-30 00:00:00

  • A Neurophysiological and Neuropsychological Consideration of Mindful Movement: Clinical and Research Implications.

    abstract::In this article, we present ideas related to three key aspects of mindfulness training: the regulation of attention via noradrenaline, the importance of working memory and its various components (particularly the central executive and episodic buffer), and the relationship of both of these to mind-wandering. These sam...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章,评审

    doi:10.3389/fnhum.2015.00282

    authors: Russell TA,Arcuri SM

    更新日期:2015-05-26 00:00:00

  • The steady-state response of the cerebral cortex to the beat of music reflects both the comprehension of music and attention.

    abstract::The brain's analyses of speech and music share a range of neural resources and mechanisms. Music displays a temporal structure of complexity similar to that of speech, unfolds over comparable timescales, and elicits cognitive demands in tasks involving comprehension and attention. During speech processing, synchronize...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2015.00436

    authors: Meltzer B,Reichenbach CS,Braiman C,Schiff ND,Hudspeth AJ,Reichenbach T

    更新日期:2015-08-06 00:00:00

  • Contributions of Sensory Coding and Attentional Control to Individual Differences in Performance in Spatial Auditory Selective Attention Tasks.

    abstract::Listeners with normal hearing thresholds (NHTs) differ in their ability to steer attention to whatever sound source is important. This ability depends on top-down executive control, which modulates the sensory representation of sound in the cortex. Yet, this sensory representation also depends on the coding fidelity o...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2016.00530

    authors: Dai L,Shinn-Cunningham BG

    更新日期:2016-10-20 00:00:00

  • The Neuro-Mechanical Processes That Underlie Goal-Directed Medio-Lateral APA during Gait Initiation.

    abstract::Gait initiation (GI) involves passing from bipedal to unipedal stance. It requires a rapid movement of the center of foot pressure (CoP) towards the future swing foot and of the center of mass (CoM) in the direction of the stance foot prior to the incoming step. This anticipatory postural adjustment (APA) allows disen...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2016.00445

    authors: Honeine JL,Schieppati M,Crisafulli O,Do MC

    更新日期:2016-08-31 00:00:00

  • Eye Movements During Everyday Behavior Predict Personality Traits.

    abstract::Besides allowing us to perceive our surroundings, eye movements are also a window into our mind and a rich source of information on who we are, how we feel, and what we do. Here we show that eye movements during an everyday task predict aspects of our personality. We tracked eye movements of 42 participants while they...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2018.00105

    authors: Hoppe S,Loetscher T,Morey SA,Bulling A

    更新日期:2018-04-13 00:00:00

  • Imaging the neural mechanisms of TMS neglect-like bias in healthy volunteers with the interleaved TMS/fMRI technique: preliminary evidence.

    abstract::Applying a precisely timed pulse of transcranial magnetic stimulation (TMS) over the right posterior parietal cortex (PPC) can produce temporary visuo-spatial neglect-like effects. Although the TMS is applied over PPC, it is not clear what other brain regions are involved. We applied TMS within a functional magnetic r...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2012.00326

    authors: Ricci R,Salatino A,Li X,Funk AP,Logan SL,Mu Q,Johnson KA,Bohning DE,George MS

    更新日期:2012-12-17 00:00:00

  • Functional Brain Network Characterization and Adaptivity during Task Practice in Healthy Volunteers and People with Schizophrenia.

    abstract::Cognitive remediation involves task practice and may improve deficits in people suffering from schizophrenia, but little is known about underlying neurophysiological mechanisms. In people with schizophrenia and controls, we used magnetoencephalography (MEG) to examine accuracy and practice-related changes in parameter...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2011.00081

    authors: Weiss SA,Bassett DS,Rubinstein D,Holroyd T,Apud J,Dickinson D,Coppola R

    更新日期:2011-08-17 00:00:00

  • Corrigendum "fNIRS-based brain-computer interfaces: a review".

    abstract::[This corrects the article on p. 3 in vol. 9, PMID: 25674060.]. ...

    journal_title:Frontiers in human neuroscience

    pub_type: 已发布勘误

    doi:10.3389/fnhum.2015.00172

    authors: Naseer N,Hong KS

    更新日期:2015-03-26 00:00:00

  • Cognitive Function Impairments Linked to Alcohol and Cannabis Use During Adolescence: A Study of Gender Differences.

    abstract::Major neurocognitive changes occur during adolescence, making this phase one of the most critical developmental periods of life. Furthermore, this phase in life is also the time in which youth substance use begins. Several studies have demonstrated the differential associations of alcohol and cannabis use concerning t...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2020.00095

    authors: Noorbakhsh S,Afzali MH,Boers E,Conrod PJ

    更新日期:2020-04-07 00:00:00

  • Contagious itch: what we know and what we would like to know.

    abstract::All humans experience itch in the course of their life. Even a discussion on the topic of itch or seeing people scratch can evoke the desire to scratch. These events are coined "contagious itch" and are very common. We and others have shown that videos showing people scratching and pictures of affected skin or insects...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章,评审

    doi:10.3389/fnhum.2015.00057

    authors: Schut C,Grossman S,Gieler U,Kupfer J,Yosipovitch G

    更新日期:2015-02-11 00:00:00

  • The neural basis of monitoring goal progress.

    abstract::The neural basis of progress monitoring has received relatively little attention compared to other sub-processes that are involved in goal directed behavior such as motor control and response inhibition. Studies of error-monitoring have identified the dorsal anterior cingulate cortex (dACC) as a structure that is sens...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2014.00688

    authors: Benn Y,Webb TL,Chang BP,Sun YH,Wilkinson ID,Farrow TF

    更新日期:2014-09-10 00:00:00

  • Somatosensory evoked field in response to visuotactile stimulation in 3- to 4-year-old children.

    abstract::A child-customized magnetoencephalography system was used to investigate somatosensory evoked field (SEF) in 3- to 4-year-old children. Three stimulus conditions were used in which the children received tactile-only stimulation to their left index finger or visuotactile stimulation. In the two visuotactile conditions,...

    journal_title:Frontiers in human neuroscience

    pub_type: 杂志文章

    doi:10.3389/fnhum.2014.00170

    authors: Remijn GB,Kikuchi M,Shitamichi K,Ueno S,Yoshimura Y,Nagao K,Tsubokawa T,Kojima H,Higashida H,Minabe Y

    更新日期:2014-03-24 00:00:00