Adaptive behaviour and feedback processing integrate experience and instruction in reinforcement learning.

Abstract:

:In any non-deterministic environment, unexpected events can indicate true changes in the world (and require behavioural adaptation) or reflect chance occurrence (and must be discounted). Adaptive behaviour requires distinguishing these possibilities. We investigated how humans achieve this by integrating high-level information from instruction and experience. In a series of EEG experiments, instructions modulated the perceived informativeness of feedback: Participants performed a novel probabilistic reinforcement learning task, receiving instructions about reliability of feedback or volatility of the environment. Importantly, our designs de-confound informativeness from surprise, which typically co-vary. Behavioural results indicate that participants used instructions to adapt their behaviour faster to changes in the environment when instructions indicated that negative feedback was more informative, even if it was simultaneously less surprising. This study is the first to show that neural markers of feedback anticipation (stimulus-preceding negativity) and of feedback processing (feedback-related negativity; FRN) reflect informativeness of unexpected feedback. Meanwhile, changes in P3 amplitude indicated imminent adjustments in behaviour. Collectively, our findings provide new evidence that high-level information interacts with experience-driven learning in a flexible manner, enabling human learners to make informed decisions about whether to persevere or explore new options, a pivotal ability in our complex environment.

journal_name

Neuroimage

journal_title

NeuroImage

authors

Schiffer AM,Siletti K,Waszak F,Yeung N

doi

10.1016/j.neuroimage.2016.08.057

subject

Has Abstract

pub_date

2017-02-01 00:00:00

pages

626-641

eissn

1053-8119

issn

1095-9572

pii

S1053-8119(16)30449-9

journal_volume

146

pub_type

杂志文章
  • Children's intellectual ability is associated with structural network integrity.

    abstract::Recent structural and functional neuroimaging studies of adults suggest that efficient patterns of brain connectivity are fundamental to human intelligence. Specifically, whole brain networks with an efficient small-world organization, along with specific brain regions (i.e., Parieto-Frontal Integration Theory, P-FIT)...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2015.09.012

    authors: Kim DJ,Davis EP,Sandman CA,Sporns O,O'Donnell BF,Buss C,Hetrick WP

    更新日期:2016-01-01 00:00:00

  • Bilateral spatial filtering: refining methods for localizing brain activation in the presence of parenchymal abnormalities.

    abstract::Functional MRI (fMRI) is an important tool for pre-surgical localization of eloquent cortex prior to resection of brain lesions. To increase the inherently low activation signal to noise ratio, fMRI pre-processing steps often include spatial smoothing. However, the effects of smoothing in the presence of brain lesions...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2006.06.051

    authors: Walker SA,Miller D,Tanabe J

    更新日期:2006-11-01 00:00:00

  • Optimal spline smoothing of fMRI time series by generalized cross-validation.

    abstract::Linear parametric regression models of fMRI time series have correlated residuals. One approach to address this problem is to condition the autocorrelation structure by temporal smoothing. Smoothing splines with the degree of smoothing selected by generalized cross-validation (GCV-spline) provide a method to find an o...

    journal_title:NeuroImage

    pub_type: 临床试验,杂志文章

    doi:10.1016/s1053-8119(03)00013-2

    authors: Carew JD,Wahba G,Xie X,Nordheim EV,Meyerand ME

    更新日期:2003-04-01 00:00:00

  • Novel artefact removal algorithms for co-registered EEG/fMRI based on selective averaging and subtraction.

    abstract::Co-registered EEG and functional MRI (EEG/fMRI) is a potential clinical tool for planning invasive EEG in patients with epilepsy. In addition, the analysis of EEG/fMRI data provides a fundamental insight into the precise physiological meaning of both fMRI and EEG data. Routine application of EEG/fMRI for localization ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2012.09.022

    authors: de Munck JC,van Houdt PJ,Gonçalves SI,van Wegen E,Ossenblok PP

    更新日期:2013-01-01 00:00:00

  • Automatic search for fMRI connectivity mapping: an alternative to Granger causality testing using formal equivalences among SEM path modeling, VAR, and unified SEM.

    abstract::Modeling the relationships among brain regions of interest (ROIs) carries unique potential to explicate how the brain orchestrates information processing. However, hurdles arise when using functional MRI data. Variation in ROI activity contains sequential dependencies and shared influences on synchronized activation. ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2009.12.117

    authors: Gates KM,Molenaar PC,Hillary FG,Ram N,Rovine MJ

    更新日期:2010-04-15 00:00:00

  • Motion artifact in magnetic resonance imaging: implications for automated analysis.

    abstract::Automated measures of cerebral magnetic resonance images (MRI) often provide greater speed and reliability compared to manual techniques but can be particularly sensitive to motion artifact. This study employed an automatic MRI analysis program that quantified regional gray matter volume and created images for verific...

    journal_title:NeuroImage

    pub_type: 临床试验,杂志文章

    doi:10.1006/nimg.2002.1076

    authors: Blumenthal JD,Zijdenbos A,Molloy E,Giedd JN

    更新日期:2002-05-01 00:00:00

  • Biophysical linkage between MRI and EEG coherence in closed head injury.

    abstract::Using conventional MRI procedures, nuclear magnetic resonance (NMR) of brain water proton (1H) T2 relaxation times and EEG coherence were obtained from two independent groups of closed head injured (CHI) patients and a group of normal control subjects. Statistically significant intercorrelations were observed between ...

    journal_title:NeuroImage

    pub_type: 临床试验,杂志文章

    doi:10.1006/nimg.1998.0365

    authors: Thatcher RW,Biver C,McAlaster R,Salazar A

    更新日期:1998-11-01 00:00:00

  • Identification of the human sympathetic connectome involved in blood pressure regulation.

    abstract::We review our recent data obtained on the cortical and subcortical components of the human sympathetic connectome - the network of regions involved in the sympathetic control of blood pressure. Specifically, we functionally identified the human homologue of the rostral ventrolateral medulla (RVLM), the primary premoto...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2019.116119

    authors: Macefield VG,Henderson LA

    更新日期:2019-11-15 00:00:00

  • Imaging genetics of mood disorders.

    abstract::Mood disorders are highly heritable and have been linked to brain regions of emotion processing. Over the past few years, an enormous amount of imaging genetics studies has demonstrated the impact of risk genes on brain regions and systems of emotion processing in vivo in healthy subjects as well as in mood disorder p...

    journal_title:NeuroImage

    pub_type: 杂志文章,评审

    doi:10.1016/j.neuroimage.2010.02.019

    authors: Scharinger C,Rabl U,Sitte HH,Pezawas L

    更新日期:2010-11-15 00:00:00

  • Involvement of right piriform cortex in olfactory familiarity judgments.

    abstract::Previous studies have shown activation of right orbitofrontal cortex during judgments of odor familiarity. In the present study, we sought to extend our knowledge about the neural circuits involved in such a task by exploring the involvement of the right prefrontal areas and limbic/primary olfactory structures. Fourte...

    journal_title:NeuroImage

    pub_type: 临床试验,杂志文章

    doi:10.1016/j.neuroimage.2004.10.028

    authors: Plailly J,Bensafi M,Pachot-Clouard M,Delon-Martin C,Kareken DA,Rouby C,Segebarth C,Royet JP

    更新日期:2005-02-15 00:00:00

  • Brain correlates of risky decision-making.

    abstract::Understanding the neurocognitive basis of risk-taking behavior is an important issue, especially in economic decision-making. Classical behavioral studies have shown that risk-attitude changes across different contexts, but little is so far known about the brain correlates of processing of outcomes across such context...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2009.08.068

    authors: Polezzi D,Sartori G,Rumiati R,Vidotto G,Daum I

    更新日期:2010-01-15 00:00:00

  • In vivo multi-slice mapping of myelin water content using T2* decay.

    abstract::Quantitative assessment of the myelin water content in the brain can substantially improve our understanding of white matter diseases such as multiple sclerosis. In this study, in vivo myelin water content was estimated using T(2)* relaxation with multi-slice acquisitions in magnetic resonance imaging (MRI). The main ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2010.04.023

    authors: Hwang D,Kim DH,Du YP

    更新日期:2010-08-01 00:00:00

  • Altered directional connectivity in Parkinson's disease during performance of a visually guided task.

    abstract::Recent animal studies have suggested that cortical areas may play a greater role in the modulation of abnormal oscillatory activity in Parkinson's disease (PD) than previously recognized. We investigated task and medication-dependent, EEG-based directional cortical connectivity in the θ (4-7Hz), α (8-12Hz), β (13-30Hz...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2011.03.018

    authors: Tropini G,Chiang J,Wang ZJ,Ty E,McKeown MJ

    更新日期:2011-06-15 00:00:00

  • Robust automated constellation-based landmark detection in human brain imaging.

    abstract::A robust fully automated algorithm for identifying an arbitrary number of landmark points in the human brain is described and validated. The proposed method combines statistical shape models with trained brain morphometric measures to estimate midbrain landmark positions reliably and accurately. Gross morphometric con...

    journal_title:NeuroImage

    pub_type: 杂志文章,评审

    doi:10.1016/j.neuroimage.2017.04.012

    authors: Ghayoor A,Vaidya JG,Johnson HJ

    更新日期:2018-04-15 00:00:00

  • Why voxel-based morphometry should be used.

    abstract::This article has been written in response to Dr. Fred L. Bookstein's article entitled '"Voxel-Based Morphometry" Should Not Be Used with Imperfectly Registered Images' in this issue of NeuroImage. We will address three main issues: (i) Dr. Bookstein appears to have misunderstood the objective of voxel-based morphometr...

    journal_title:NeuroImage

    pub_type: 评论,杂志文章

    doi:10.1006/nimg.2001.0961

    authors: Ashburner J,Friston KJ

    更新日期:2001-12-01 00:00:00

  • Material-specific memory processing is related to language dominance.

    abstract::Neuropsychological and neuroimaging data have shown a functional lateralization of the medial temporal lobes for verbal and non-verbal memory material respectively. We hypothesized that this lateralization is related to language lateralization. Using fMRI we conducted three memory tasks with different memory material ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2007.05.022

    authors: Weber B,Fliessbach K,Lange N,Kügler F,Elger CE

    更新日期:2007-08-15 00:00:00

  • The posterior cingulate and medial prefrontal cortex mediate the anticipatory allocation of spatial attention.

    abstract::The purpose of this study was to identify brain regions underlying internally generated anticipatory biases toward locations where significant events are expected to occur. Subjects fixated centrally and responded to peripheral targets preceded by a spatially valid (predictive), invalid (misleading), or neutral centra...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/s1053-8119(02)00012-5

    authors: Small DM,Gitelman DR,Gregory MD,Nobre AC,Parrish TB,Mesulam MM

    更新日期:2003-03-01 00:00:00

  • Mapping the functional connectome in traumatic brain injury: What can graph metrics tell us?

    abstract:OBJECTIVE:Traumatic brain injury (TBI) is associated with cognitive and motor deficits, and poses a significant personal, societal, and economic burden. One mechanism by which TBI is thought to affect cognition and behavior is through changes in functional connectivity. Graph theory is a powerful framework for quantify...

    journal_title:NeuroImage

    pub_type: 杂志文章,评审

    doi:10.1016/j.neuroimage.2016.12.003

    authors: Caeyenberghs K,Verhelst H,Clemente A,Wilson PH

    更新日期:2017-10-15 00:00:00

  • Age-related changes in reappraisal of appetitive cravings during adolescence.

    abstract::The ability to regulate temptation and manage appetitive cravings is an important aspect of healthy adolescent development, but the neural systems underlying this process are understudied. In the present study, 60 healthy females evenly distributed from 10 to 23years of age used reappraisal to regulate the desire to c...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2014.12.037

    authors: Giuliani NR,Pfeifer JH

    更新日期:2015-03-01 00:00:00

  • Differential activation of right superior parietal cortex and intraparietal sulcus by spatial and nonspatial attention.

    abstract::Neuropsychological and functional neuroimaging studies have implicated the right posterior parietal cortex (PPC) in human spatial attention. We tested the hypothesis that this area is also involved in nonspatial aspects of attention and working memory using positron emission tomography in healthy volunteers. In an ini...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1006/nimg.1998.0354

    authors: Coull JT,Frith CD

    更新日期:1998-08-01 00:00:00

  • Genetic influences on prefrontal activation during a verbal fluency task in adults: a twin study based on multichannel near-infrared spectroscopy.

    abstract::Near-infrared spectroscopy (NIRS) studies have reported that prefrontal hemodynamic dysfunction during executive function tasks may be a promising biomarker of psychiatric disorders, because its portability and noninvasiveness allow easy measurements in clinical settings. Here, we investigated the degree to which pref...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2013.03.052

    authors: Sakakibara E,Takizawa R,Nishimura Y,Kawasaki S,Satomura Y,Kinoshita A,Koike S,Marumo K,Kinou M,Tochigi M,Nishida N,Tokunaga K,Eguchi S,Yamasaki S,Natsubori T,Iwashiro N,Inoue H,Takano Y,Takei K,Suga M,Yamasue H,

    更新日期:2014-01-15 00:00:00

  • Estimation of false discovery rates for wavelet-denoised statistical parametric maps.

    abstract::Correction for multiple comparisons in neuroimaging data is an important area of research. Recently, wavelet-based methods have gained popularity and have been reported to achieve better sensitivity compared to spatial domain methods. However, these techniques produce smoothed statistical maps which are difficult to i...

    journal_title:NeuroImage

    pub_type: 临床试验,杂志文章

    doi:10.1016/j.neuroimage.2006.06.033

    authors: Srikanth R,Casanova R,Laurienti PJ,Peiffer AM,Maldjian JA

    更新日期:2006-10-15 00:00:00

  • Anatomic constraints on cognitive theories of category specificity.

    abstract::Many cognitive theories of semantic organization stem from reports of patients with selective, category-specific deficits for particular classes of objects (e.g., fruit). The anatomical assumptions underlying the competing claims can be evaluated with functional neuroimaging but the findings to date have been inconsis...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1006/nimg.2001.1002

    authors: Devlin JT,Moore CJ,Mummery CJ,Gorno-Tempini ML,Phillips JA,Noppeney U,Frackowiak RS,Friston KJ,Price CJ

    更新日期:2002-03-01 00:00:00

  • Atrophy is associated with posterior cingulate white matter disruption in dementia with Lewy bodies and Alzheimer's disease.

    abstract::Hippocampal atrophy and posterior cingulate hypometabolism are common features of both Alzheimer's disease (AD) and dementia with Lewy bodies (DLB). These regions show correlated activity at rest as part of the 'default network', and they are connected by the cingulum, a white matter (WM) tract. We hypothesised that h...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2007.02.027

    authors: Firbank MJ,Blamire AM,Krishnan MS,Teodorczuk A,English P,Gholkar A,Harrison R,O'Brien JT

    更新日期:2007-05-15 00:00:00

  • Structural consequences of diffuse traumatic brain injury: a large deformation tensor-based morphometry study.

    abstract::Traumatic brain injury (TBI) is one of the most common causes of long-term disability. Despite the importance of identifying neuropathology in individuals with chronic TBI, methodological challenges posed at the stage of inter-subject image registration have hampered previous voxel-based MRI studies from providing a c...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2007.10.005

    authors: Kim J,Avants B,Patel S,Whyte J,Coslett BH,Pluta J,Detre JA,Gee JC

    更新日期:2008-02-01 00:00:00

  • Voxel-based cortical thickness measurements in MRI.

    abstract::The thickness of the cerebral cortex can provide valuable information about normal and abnormal neuroanatomy. High resolution MRI together with powerful image processing techniques has made it possible to perform these measurements automatically over the whole brain. Here we present a method for automatically generati...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2008.01.027

    authors: Hutton C,De Vita E,Ashburner J,Deichmann R,Turner R

    更新日期:2008-05-01 00:00:00

  • A cross-laboratory study of event-related gamma activity in a standard object recognition paradigm.

    abstract::This study proposes a standard paradigm for the investigation of visual information processing by means of gamma activity and presents a novel set of stimuli with a broad range of complex, coloured familiar real world and unfamiliar nonsense objects which are well matched with respect to physical stimulus properties. ...

    journal_title:NeuroImage

    pub_type: 杂志文章,多中心研究

    doi:10.1016/j.neuroimage.2006.07.034

    authors: Busch NA,Herrmann CS,Müller MM,Lenz D,Gruber T

    更新日期:2006-12-01 00:00:00

  • Premature brain aging in humans exposed to maternal nutrient restriction during early gestation.

    abstract:BACKGROUND:Prenatal exposure to undernutrition is widespread in both developing and industrialized countries, causing irreversible damage to the developing brain, resulting in altered brain structure and decreased cognitive function during adulthood. The Dutch famine in 1944/45 was a humanitarian disaster, now enabling...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2017.10.047

    authors: Franke K,Gaser C,Roseboom TJ,Schwab M,de Rooij SR

    更新日期:2018-06-01 00:00:00

  • In vivo functional localization of the temporal monocular crescent representation in human primary visual cortex.

    abstract::The temporal monocular crescent (TMC) is the most peripheral portion of the visual field whose perception relies solely on input from the ipsilateral eye. According to a handful of post-mortem histological studies in humans and non-human primates, the TMC is represented visuotopically within the most anterior portion ...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2020.116516

    authors: Nasr S,LaPierre C,Vaughn CE,Witzel T,Stockmann JP,Polimeni JR

    更新日期:2020-04-01 00:00:00

  • Focal brain matter differences associated with lifetime alcohol intake and visual attention in male but not in female non-alcohol-dependent drinkers.

    abstract::The purpose of this study was to investigate whether current or lifetime alcohol intake is related to focal gray and white matter in healthy non-alcohol-dependent drinkers, and, if so, whether these densities are related to functional brain activity associated with visual attention. Voxel-based morphometric analyses o...

    journal_title:NeuroImage

    pub_type: 杂志文章

    doi:10.1016/j.neuroimage.2005.01.036

    authors: de Bruin EA,Hulshoff Pol HE,Schnack HG,Janssen J,Bijl S,Evans AC,Kenemans JL,Kahn RS,Verbaten MN

    更新日期:2005-06-01 00:00:00