Abstract:
:Reinforcement learning (RL) is a powerful concept underlying forms of associative learning governed by the use of a scalar reward signal, with learning taking place if expectations are violated. RL may be assessed using model-based and model-free approaches. Model-based reinforcement learning involves the amygdala, the hippocampus, and the orbitofrontal cortex (OFC). The model-free system involves the pedunculopontine-tegmental nucleus (PPTgN), the ventral tegmental area (VTA) and the ventral striatum (VS). Based on the functional connectivity of VS, model-free and model based RL systems center on the VS that by integrating model-free signals (received as reward prediction error) and model-based reward related input computes value. Using the concept of reinforcement learning agent we propose that the VS serves as the value function component of the RL agent. Regarding the model utilized for model-based computations we turned to the proactive brain concept, which offers an ubiquitous function for the default network based on its great functional overlap with contextual associative areas. Hence, by means of the default network the brain continuously organizes its environment into context frames enabling the formulation of analogy-based association that are turned into predictions of what to expect. The OFC integrates reward-related information into context frames upon computing reward expectation by compiling stimulus-reward and context-reward information offered by the amygdala and hippocampus, respectively. Furthermore we suggest that the integration of model-based expectations regarding reward into the value signal is further supported by the efferent of the OFC that reach structures canonical for model-free learning (e.g., the PPTgN, VTA, and VS).
journal_name
Behav Neuroscijournal_title
Behavioral neuroscienceauthors
Zsuga J,Biro K,Papp C,Tajti G,Gesztelyi Rdoi
10.1037/bne0000116subject
Has Abstractpub_date
2016-02-01 00:00:00pages
6-18issue
1eissn
0735-7044issn
1939-0084pii
2016-02685-002journal_volume
130pub_type
杂志文章abstract::Food reinforcement, or the motivation to eat, has been associated with increased energy intake, greater body weight, and prospective weight gain. Much of the previous research on the reinforcing value of food has focused on the role of dopamine, but it may be worthwhile to examine genetic polymorphisms in the serotoni...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/a0032026
更新日期:2013-06-01 00:00:00
abstract::Galanin is a 29-amino-acid neuropeptide that is overexpressed in Alzheimer's disease (AD) and impairs performance on rodent learning and memory tasks. M40, a peptidergic galanin receptor ligand, blocks galanin-induced impairments on delayed non-matching-to-position (DNMTP). The present experiments used the 192IgG-sapo...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.111.3.552
更新日期:1997-06-01 00:00:00
abstract::The effects of lesions of the medial dorsal thalamic nucleus (MD) on blocking and latent inhibition (LI) of the rat eyeblink response were examined in the present study. Previous work has demonstrated that the cingulate cortex and related thalamic areas are involved in processing conditioning stimuli throughout traini...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.116.2.276
更新日期:2002-04-01 00:00:00
abstract::Dopamine (DA)-containing neurons in the ventral tegmental area (VTA) are thought to play an important role in fear motivation. The primary objective of the present study was to determine the connection between DA D2, gamma aminobutyric acid (GABA)A, and benzodiazepine receptors in the VTA and footshock-associated emot...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:
更新日期:2002-10-01 00:00:00
abstract::This study was performed with male rats categorized as sexually naive (SN), sexually active (SA), or sexually inactive (SI). In a first experiment the effects of dopamine (DA) D2 agonist SND 919 (0.05, 1, and 10 mg/kg) on the copulatory behavior of SN, SA and SI rats were assessed. In a second experiment the DA D2 ago...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.110.4.802
更新日期:1996-08-01 00:00:00
abstract::Rats with unilateral dopamine (DA) depletion (hemiParkinson analogue rats) are impaired in using the contralateral (bad) limbs for skilled movements and for postural adjustments and compensate by using their good limbs in novel ways. The present study consisted of a reaching task in which compensatory adjustments usin...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.110.1.117
更新日期:1996-02-01 00:00:00
abstract::To determine whether dopamine signaling in the dorsal striatum is sufficient for performance of goal-directed behaviors, local dopamine production was restored in the dorsal striatum of dopamine-deficient (DD) mice through viral-mediated gene therapy. Virally rescued DD (vrDD) mice were tested for learning of an appet...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/0735-7044.120.1.000
更新日期:2006-02-01 00:00:00
abstract::The addition of a dual runway configuration did not disrupt the successful performance of normal animals, nor did it improve the deficit of septal rats on the Maier three-table spatial integration task. Both groups of animals displayed a preference for the outside runway configuration during exploration. During testin...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.99.4.631
更新日期:1985-08-01 00:00:00
abstract::In rhesus monkeys, the magnitude of the cochlear response to auditory stimuli (click-evoked otoacoustic emissions, [CEOAEs]) is correlated with seasonal changes in circulating testosterone levels. The present study investigated the association between circulating testosterone and CEOAE production in men. CEOAEs were m...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/a0027193
更新日期:2012-04-01 00:00:00
abstract::An acoustic stimulus previously paired with footshock elicits stereotyped increases in arterial pressure and heart rate and induces freezing behavior in freely behaving rats. Although the arterial pressure and freezing responses differ between groups given paired and random presentations of the tone and shock, the inc...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.102.1.66
更新日期:1988-02-01 00:00:00
abstract::Based on work done in animal models showing that autism-like symptoms are ameliorated following exposure to an enriched sensorimotor environment, we attempted to develop a comparable therapy for children with autism. In an initial randomized controlled trial, children with autism who received sensorimotor enrichment a...
journal_title:Behavioral neuroscience
pub_type: 杂志文章,随机对照试验
doi:10.1037/bne0000068
更新日期:2015-08-01 00:00:00
abstract::Spatial learning and memory were studied in dogs of varying ages and sources. Compared to young dogs, a significantly higher proportion of aged dogs could not acquire a spatial delayed nonmatching-to-sample task. A regression analysis revealed a significant age effect during acquisition. Spatial memory was studied by ...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.109.5.851
更新日期:1995-10-01 00:00:00
abstract::The influence of 4-hydroxyamphetamine (4-OHAM) on food and water intake and in vivo brown adipose thermogenesis was examined in two experiments. In Experiment 1, female rats were treated with 0.00, 0.25, 0.50, 1.00, or 2.00 mg/kg 4-OHAM (ip) prior to assessment of interscapular brown adipose tissue (IBAT) thermogenesi...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.98.6.1060
更新日期:1984-12-01 00:00:00
abstract::The gustatory rejection response of the cabbage looper, Trichoplusia ni (Lepidoptera: Noctuidae), habituates to antifeedant compounds, allowing for the consumption of deterrent yet nontoxic plant materials. In the present study, we demonstrate that habituation to an antifeedant compound (quinine) persists through the ...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/a0020741
更新日期:2010-10-01 00:00:00
abstract::Previous research emphasized the impact of traumatic brain injury on cholinergic systems and associated cognitive functions. Here we addressed the converse question: Because of the available evidence indicating cognitive and neuronal vulnerabilities in humans expressing low-capacity cholinergic systems or with declini...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/bne0000310
更新日期:2019-08-01 00:00:00
abstract::The ability of the rat brain to acquire or to retain specific learning tasks was tested under conditions of multiple lesions and widely different amounts of practice. Lesion targets were (a) the medial prefrontal and cingulate cortex, (b) the anterior and mediodorsal thalamus, and (c) the dorsal and ventral hippocampu...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:
更新日期:1985-04-01 00:00:00
abstract::The rat's willingness to ingest glucose after an initial intraoral intake test was probed by beginning a 2nd intraoral intake test at variable durations (1-120 min). In Experiment 1, after an initial meal of 12.5% glucose solution averaging 26.9 +/- 1.7 ml (SEM, n = 10), the size of the 2nd (probe) meal of the same st...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:
更新日期:1994-04-01 00:00:00
abstract::Microstructural analysis of licking behavior in the rat was conducted (a) to describe in detail the characteristics of benzodiazepine-induced changes in ingestion and (b) to determine if the changes are consistent with an alteration in palatability. The effects of the benzodiazepine receptor (BZR) agonist midazolam (0...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.112.2.447
更新日期:1998-04-01 00:00:00
abstract::Many studies have pointed to vulnerability to stress and stress-related pathologies at different timepoints during an individual's life span. These sensitive windows are usually during periods of neural development, such as embryogenesis, infancy, and adolescence. It is critical to understand how neural circuitry may ...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/a0033602
更新日期:2013-10-01 00:00:00
abstract::Rats with bilateral, electrolytic lesions of perirhinal cortex (PRC), lateral entorhinal cortex (LEC), or combined lesions (PRLE) were impaired relative to controls (sham) during initial acquisition in the Morris water maze, although all groups were eventually able to learn to locate the platform. A further deficit in...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.112.2.304
更新日期:1998-04-01 00:00:00
abstract::Parametric manipulations of the task demand were used to examine the role of the hippocampus and amygdala in nonspatial and spatial working memory in rats. Hippocampal lesions produced an immediate and long-lasting impairment of nonspatial working memory in an operant task. The memory deficits increased as the delay i...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.108.5.866
更新日期:1994-10-01 00:00:00
abstract::The authors report an effort to advance animal models that mimic the cognitive decline of Alzheimer's disease. Rats were trained and repeatedly tested in a spatial delayed matching-to-position paradigm in the water maze, with the location of the submerged platform changing between, but not within, days. After Trial 1 ...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.116.2.351
更新日期:2002-04-01 00:00:00
abstract::Effects of the ingestion of protein and carbohydrate conditioned a preference for one size of chow particle over another, which was triggered by need for a specific nutrient. This learned elicitation of nutrient-specific dietary selection was not changed by injection of dl-fenfluramine HCl (2.5 mg/kg). This indicates ...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:
更新日期:1990-02-01 00:00:00
abstract::The glutamate analog, L-2-amino-4-phosphonobutyric acid (L-AP4) is a selective agonist for several members of the metabotropic glutamate receptor (mGluR) family. Activation of presynaptic mGluRs by L-AP4 causes a suppression of synaptic transmission in the central nervous system. In this study, the role of 1 subtype o...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.112.3.525
更新日期:1998-06-01 00:00:00
abstract::Binge eating and substance dependence are disorders characterized by a loss of control over consummatory behaviors. Given the common characteristics of these two types of disorders, it is not surprising that the comorbidity between eating disorders and substance abuse disorders is high (20-40%; Conason et al., 2006). ...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/a0025759
更新日期:2011-12-01 00:00:00
abstract::Guinea pig pups that were separated from their mothers and placed into a novel environment for 90 min showed an increase in dopamine (DA) turnover (ratio of metabolites to DA) in the septum compared with undisturbed baseline controls. Pups placed into the novel environment with their mothers exhibited an intermediate ...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037//0735-7044.104.4.607
更新日期:1990-08-01 00:00:00
abstract::The spontaneously hypertensive rat (SHR) has been suggested as a possible animal model of attention-deficit/hyperactivity disorder (ADHD). Reductions in the volume of the cerebellum and impairments in cerebellar-dependent eyeblink conditioning have been observed in ADHD, prompting investigation into whether SHRs also ...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/0735-7044.122.1.63
更新日期:2008-02-01 00:00:00
abstract::Cognitive functions, such as working memory, are disrupted in most psychiatric disorders. Many of these processes are believed to depend on the medial prefrontal cortex (mPFC). Traditionally, maze-based behavioral tasks, which have a strong exploratory component, have been used to study the role of the mPFC in working...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/bne0000357
更新日期:2020-06-01 00:00:00
abstract::A good deal is known about psychological factors that contribute to reward value, but very little is known about how reward is computed in the brain. Integration in the circuit for brain stimulation reward may provide a simple model system. Parametric studies have favored the idea that the integration is linear, altho...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/a0026180
更新日期:2011-12-01 00:00:00
abstract::Attention-deficit hyperactivity disorder (ADHD) may be caused by genetic or environmental factors. Among environmental factors, perinatal complications are related, such as neonatal hypoxia-ischemia (HI). Thus, the aim of this study was to investigate whether HI contributes to the development of characteristics relate...
journal_title:Behavioral neuroscience
pub_type: 杂志文章
doi:10.1037/bne0000063
更新日期:2015-06-01 00:00:00