Integrating Models of Interval Timing and Reinforcement Learning.

Abstract:

:We present an integrated view of interval timing and reinforcement learning (RL) in the brain. The computational goal of RL is to maximize future rewards, and this depends crucially on a representation of time. Different RL systems in the brain process time in distinct ways. A model-based system learns 'what happens when', employing this internal model to generate action plans, while a model-free system learns to predict reward directly from a set of temporal basis functions. We describe how these systems are subserved by a computational division of labor between several brain regions, with a focus on the basal ganglia and the hippocampus, as well as how these regions are influenced by the neuromodulator dopamine.

journal_name

Trends Cogn Sci

authors

Petter EA,Gershman SJ,Meck WH

doi

10.1016/j.tics.2018.08.004

subject

Has Abstract

pub_date

2018-10-01 00:00:00

pages

911-922

issue

10

eissn

1364-6613

issn

1879-307X

pii

S1364-6613(18)30193-1

journal_volume

22

pub_type

杂志文章,评审
  • Neural mechanisms and temporal dynamics of performance monitoring.

    abstract::Successful goal-directed behavior critically depends on performance monitoring, a set of cognitive and affective functions determining whether adaptive control is needed and, if so, which type and magnitude is required. Knowledge of the brain structures involved in such a process has grown enormously, although the tim...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2014.02.009

    authors: Ullsperger M,Fischer AG,Nigbur R,Endrass T

    更新日期:2014-05-01 00:00:00

  • Understanding face perception by means of human electrophysiology.

    abstract::Electrophysiological recordings on the human scalp provide a wealth of information about the temporal dynamics and nature of face perception at a global level of brain organization. The time window between 100 and 200 ms witnesses the transition between low-level and high-level vision, an N170 component correlating wi...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2014.02.013

    authors: Rossion B

    更新日期:2014-06-01 00:00:00

  • CSI (Crime Scene Induction): Creating False Memories of Committing Crime.

    abstract::We describe two merging lines of empirical inquiry: entire false memories for autobiographical events and false confessions. A recent study showed that people can be led to remember, and confess to, perpetrating serious crimes that never occurred when confronted with suggestive interview tactics commonly used in polic...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/j.tics.2015.08.014

    authors: Porter SB,Baker AT

    更新日期:2015-12-01 00:00:00

  • Decision Activity in Parietal Cortex - Leader or Follower?

    abstract::Katz and colleagues demonstrate that inactivating the primate lateral intraparietal area (LIP) spares visual motion decisions, even though these same decisions strongly modulate LIP neurons. This work is the latest addition to an intense effort spanning sensory modalities, animals, and techniques to understand which s...

    journal_title:Trends in cognitive sciences

    pub_type: 评论,杂志文章

    doi:10.1016/j.tics.2016.09.005

    authors: Pisupati S,Chartarifsky L,Churchland AK

    更新日期:2016-11-01 00:00:00

  • Disruptions in autobiographical memory processing in depression and the emergence of memory therapeutics.

    abstract::Depression is characterized by distinct profiles of disturbance in ways autobiographical memories are represented, recalled, and maintained. We review four core domains of difficulty: systematic biases in favor of negative material; impoverished access and responses to positive memories; reduced access to the specific...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2014.06.010

    authors: Dalgleish T,Werner-Seidler A

    更新日期:2014-11-01 00:00:00

  • Neural mechanisms of motivated forgetting.

    abstract::Not all memories are equally welcome in awareness. People limit the time they spend thinking about unpleasant experiences, a process that begins during encoding, but that continues when cues later remind someone of the memory. Here, we review the emerging behavioural and neuroimaging evidence that suppressing awarenes...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2014.03.002

    authors: Anderson MC,Hanslmayr S

    更新日期:2014-06-01 00:00:00

  • Animal Learning and Cognition: A neural network approach.

    abstract::by Nestor A. Schmajuk, Cambridge University Press, 1997. £29.95 (xii+340 pages) ISBN 0 521 45086 1. ...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/s1364-6613(98)01192-9

    authors: McLaren IP

    更新日期:1998-06-01 00:00:00

  • Comparative Connectomics.

    abstract::We introduce comparative connectomics, the quantitative study of cross-species commonalities and variations in brain network topology that aims to discover general principles of network architecture of nervous systems and the identification of species-specific features of brain connectivity. By comparing connectomes d...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2016.03.001

    authors: van den Heuvel MP,Bullmore ET,Sporns O

    更新日期:2016-05-01 00:00:00

  • Can cognitive processes be inferred from neuroimaging data?

    abstract::There is much interest currently in using functional neuroimaging techniques to understand better the nature of cognition. One particular practice that has become common is 'reverse inference', by which the engagement of a particular cognitive process is inferred from the activation of a particular brain region. Such ...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/j.tics.2005.12.004

    authors: Poldrack RA

    更新日期:2006-02-01 00:00:00

  • The origins of religious disbelief.

    abstract::Although most people are religious, there are hundreds of millions of religious disbelievers in the world. What is religious disbelief and how does it arise? Recent developments in the scientific study of religious beliefs and behaviors point to the conclusion that religious disbelief arises from multiple interacting ...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2012.11.006

    authors: Norenzayan A,Gervais WM

    更新日期:2013-01-01 00:00:00

  • Brain-to-brain coupling: a mechanism for creating and sharing a social world.

    abstract::Cognition materializes in an interpersonal space. The emergence of complex behaviors requires the coordination of actions among individuals according to a shared set of rules. Despite the central role of other individuals in shaping one's mind, most cognitive studies focus on processes that occur within a single indiv...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/j.tics.2011.12.007

    authors: Hasson U,Ghazanfar AA,Galantucci B,Garrod S,Keysers C

    更新日期:2012-02-01 00:00:00

  • Core systems of number.

    abstract::What representations underlie the ability to think and reason about number? Whereas certain numerical concepts, such as the real numbers, are only ever represented by a subset of human adults, other numerical abilities are widespread and can be observed in adults, infants and other animal species. We review recent beh...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2004.05.002

    authors: Feigenson L,Dehaene S,Spelke E

    更新日期:2004-07-01 00:00:00

  • Dynamic Network Connectivity: A new form of neuroplasticity.

    abstract::Prefrontal cortical (PFC) working memory functions depend on pyramidal cell networks that interconnect on dendritic spines. Recent research has revealed that the strength of PFC network connections can be rapidly and reversibly increased or decreased by molecular signaling events within slender, elongated spines: a pr...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/j.tics.2010.05.003

    authors: Arnsten AF,Paspalas CD,Gamo NJ,Yang Y,Wang M

    更新日期:2010-08-01 00:00:00

  • Genetics of human episodic memory: dealing with complexity.

    abstract::Episodic memory is a polygenic behavioral trait with substantial heritability estimates. Despite its complexity, recent empirical evidence supports the notion that behavioral genetic studies of episodic memory might successfully identify trait-associated molecules and pathways. The development of high-throughput genot...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2011.07.005

    authors: Papassotiropoulos A,de Quervain DJ

    更新日期:2011-09-01 00:00:00

  • The compulsive habit of cars.

    abstract::The car dependence of people living in contemporary cities is a major concern for policy makers, who often find it difficult to persuade people into more sustainable transport modes. By contrast, recent insights from neuroscience have shown that a broad spectrum of behaviors can become habitual and, thus, resistant to...

    journal_title:Trends in cognitive sciences

    pub_type: 评论,杂志文章

    doi:10.1016/j.tics.2014.01.008

    authors: Yalachkov Y,Naumer MJ,Plyushteva A

    更新日期:2014-05-01 00:00:00

  • The Dark Room Problem.

    abstract::Predictive Processing theories hold that the mind's core aim is to minimize prediction-error about its experiences. But prediction-error minimization can be 'hacked', by placing oneself in highly predictable environments where nothing happens. Recent philosophical work suggests that this is a surprisingly serious chal...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/j.tics.2020.02.006

    authors: Sun Z,Firestone C

    更新日期:2020-05-01 00:00:00

  • The Neural Representations Underlying Human Episodic Memory.

    abstract::A fundamental question of human episodic memory concerns the cognitive and neural representations and processes that give rise to the neural signals of memory. By integrating behavioral tests, formal computational models, and neural measures of brain activity patterns, recent studies suggest that memory signals not on...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2018.03.004

    authors: Xue G

    更新日期:2018-06-01 00:00:00

  • The evolution of speech: a comparative review.

    abstract::The evolution of speech can be studied independently of the evolution of language, with the advantage that most aspects of speech acoustics, physiology and neural control are shared with animals, and thus open to empirical investigation. At least two changes were necessary prerequisites for modern human speech abiliti...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/s1364-6613(00)01494-7

    authors: Fitch WT

    更新日期:2000-07-01 00:00:00

  • Dynamic Neural Representations: An Inferential Challenge for fMRI.

    abstract::Measures of brain activity with high temporal resolution have shown that the information represented in a single brain region undergoes dynamic changes on the scale of milliseconds. This dynamic process presents a unique inferential challenge to low temporal resolution neural measures, such as BOLD fMRI. Potential sol...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2019.04.004

    authors: Ghuman AS,Martin A

    更新日期:2019-07-01 00:00:00

  • How our hands help us learn.

    abstract::When people talk they gesture, and those gestures often reflect thoughts not expressed in their words. In this sense, gesture and the speech it accompanies can mismatch. Gesture-speech 'mismatches' are found when learners are on the verge of making progress on a task - when they are ready to learn. Moreover, mismatche...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2005.03.006

    authors: Goldin-Meadow S,Wagner SM

    更新日期:2005-05-01 00:00:00

  • What's next? New evidence for prediction in human vision.

    abstract::Everyday visual experience involves making implicit predictions, as revealed by our surprise when something disturbs our expectations. Many theories of vision have been premised on the central role played by prediction. Yet, implicit prediction in human vision has been difficult to assess in the laboratory, and many r...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/j.tics.2008.06.001

    authors: Enns JT,Lleras A

    更新日期:2008-09-01 00:00:00

  • Early signs of brain asymmetry.

    abstract::A new study shows a leftward asymmetry of the choroid plexus in two-thirds of first-trimester human fetuses. This is the earliest brain asymmetry so far identified and may be a precursor to other asymmetries, including that of the temporal planum, which is evident from the 31st week of gestation. ...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/j.tics.2013.09.008

    authors: Corballis MC

    更新日期:2013-11-01 00:00:00

  • A connectionist perspective on the development of reading skills in children.

    abstract::The development of decoding skills has traditionally been viewed as a stage-like process during which children's reading strategies change as a consequence of the acquisition of phonological awareness. More explicit accounts of the mechanisms involved in learning to read are provided by recent connectionist models in ...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/S1364-6613(97)89053-5

    authors: Snowling M,Hulme C,Nation K

    更新日期:1997-06-01 00:00:00

  • Integrated Intelligence from Distributed Brain Activity.

    abstract::How does organized cognition arise from distributed brain activity? Recent analyses of fluid intelligence suggest a core process of cognitive focus and integration, organizing the components of a cognitive operation into the required computational structure. A cortical 'multiple-demand' (MD) system is closely linked t...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2020.06.012

    authors: Duncan J,Assem M,Shashidhara S

    更新日期:2020-10-01 00:00:00

  • The Hippocampal Cognitive Map: One Space or Many?

    abstract::New evidence reported by Solomon et al. that hippocampal activity tracks distance in semantic space during recall supports the growing consensus of a domain-general cognitive map. Nevertheless, are all inputs equally processed into a 'universal map', or are there input constraints (e.g., space, semantics) that lead to...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/j.tics.2019.12.013

    authors: Spiers HJ

    更新日期:2020-03-01 00:00:00

  • Is Birdsong More Like Speech or Music?

    abstract::Music and speech share many acoustic cues but not all are equally important. For example, harmonic pitch is essential for music but not for speech. When birds communicate is their song more like speech or music? A new study contrasting pitch and spectral patterns shows that birds perceive their song more like humans p...

    journal_title:Trends in cognitive sciences

    pub_type: 评论,杂志文章

    doi:10.1016/j.tics.2016.02.004

    authors: Shannon RV

    更新日期:2016-04-01 00:00:00

  • Vision and vigilance on the go.

    abstract::Moving purposefully through the world requires the seamless coordination of a wide variety of sensory, motor, and motivation systems. Recent experiments suggest that mouse visual cortex may offer a particularly well suited forum for experimental investigation of this coordination. ...

    journal_title:Trends in cognitive sciences

    pub_type: 评论,杂志文章

    doi:10.1016/j.tics.2015.02.001

    authors: Ghose GM

    更新日期:2015-03-01 00:00:00

  • All in Good Time: Long-Lasting Postdictive Effects Reveal Discrete Perception.

    abstract::Is consciousness a continuous stream of percepts or is it discrete, occurring only at certain moments in time? This question has puzzled philosophers, psychologists, and neuroscientists for centuries. Both hypotheses have fallen repeatedly in and out of favor. Here, we review recent studies exploring long-lasting post...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2020.07.001

    authors: Herzog MH,Drissi-Daoudi L,Doerig A

    更新日期:2020-10-01 00:00:00

  • Shared Mechanisms May Support Mnemonic Benefits from Self-Referencing and Emotion.

    abstract::The literatures on episodic memory for self-referential and emotional information have proceeded relatively independently, and most studies examining the effects of age on these memory processes have been interpreted within domain-specific frameworks. However, there is increasing evidence for shared mechanisms that co...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章,评审

    doi:10.1016/j.tics.2018.05.001

    authors: Gutchess A,Kensinger EA

    更新日期:2018-08-01 00:00:00

  • Circadian clocks: genes, sleep, and cognition.

    abstract::The endogenous circadian clock modulates cognitive performance over the daily 24-h cycle. Environmental disturbance of the clock, such as shift work or jet lag schedules, compromises sleep, alertness and problem solving. What is not generally appreciated, however, is that the circadian clock also modulates cognitive a...

    journal_title:Trends in cognitive sciences

    pub_type: 杂志文章

    doi:10.1016/j.tics.2010.03.007

    authors: Kyriacou CP,Hastings MH

    更新日期:2010-06-01 00:00:00