One-shot learning and behavioral eligibility traces in sequential decision making.

Abstract:

:In many daily tasks, we make multiple decisions before reaching a goal. In order to learn such sequences of decisions, a mechanism to link earlier actions to later reward is necessary. Reinforcement learning (RL) theory suggests two classes of algorithms solving this credit assignment problem: In classic temporal-difference learning, earlier actions receive reward information only after multiple repetitions of the task, whereas models with eligibility traces reinforce entire sequences of actions from a single experience (one-shot). Here, we show one-shot learning of sequences. We developed a novel paradigm to directly observe which actions and states along a multi-step sequence are reinforced after a single reward. By focusing our analysis on those states for which RL with and without eligibility trace make qualitatively distinct predictions, we find direct behavioral (choice probability) and physiological (pupil dilation) signatures of reinforcement learning with eligibility trace across multiple sensory modalities.

journal_name

Elife

journal_title

eLife

authors

Lehmann MP,Xu HA,Liakoni V,Herzog MH,Gerstner W,Preuschoff K

doi

10.7554/eLife.47463

subject

Has Abstract

pub_date

2019-11-11 00:00:00

issn

2050-084X

pii

47463

journal_volume

8

pub_type

杂志文章

相关文献

eLife文献大全
  • A Phytophthora effector recruits a host cytoplasmic transacetylase into nuclear speckles to enhance plant susceptibility.

    abstract::Oomycete pathogens secrete host cell-entering effector proteins to manipulate host immunity during infection. We previously showed that PsAvh52, an early-induced RxLR effector secreted from the soybean root rot pathogen, Phytophthora sojae, could suppress plant immunity. Here, we found that PsAvh52 is required for ful...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.40039

    authors: Li H,Wang H,Jing M,Zhu J,Guo B,Wang Y,Lin Y,Chen H,Kong L,Ma Z,Wang Y,Ye W,Dong S,Tyler B,Wang Y

    更新日期:2018-10-22 00:00:00

  • Symmetry broken and rebroken during the ATP hydrolysis cycle of the mitochondrial Hsp90 TRAP1.

    abstract::Hsp90 is a homodimeric ATP-dependent molecular chaperone that remodels its substrate 'client' proteins, facilitating their folding and activating them for biological function. Despite decades of research, the mechanism connecting ATP hydrolysis and chaperone function remains elusive. Particularly puzzling has been the...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.25235

    authors: Elnatan D,Betegon M,Liu Y,Ramelot T,Kennedy MA,Agard DA

    更新日期:2017-07-25 00:00:00

  • MELK expression correlates with tumor mitotic activity but is not required for cancer growth.

    abstract::The Maternal Embryonic Leucine Zipper Kinase (MELK) has been identified as a promising therapeutic target in multiple cancer types. MELK over-expression is associated with aggressive disease, and MELK has been implicated in numerous cancer-related processes, including chemotherapy resistance, stem cell renewal, and tu...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.32838

    authors: Giuliano CJ,Lin A,Smith JC,Palladino AC,Sheltzer JM

    更新日期:2018-02-08 00:00:00

  • Embryonic hematopoiesis modulates the inflammatory response and larval hematopoiesis in Drosophila.

    abstract::Recent lineage tracing analyses have significantly improved our understanding of immune system development and highlighted the importance of the different hematopoietic waves. The current challenge is to understand whether these waves interact and whether this affects the function of the immune system. Here we report ...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.34890

    authors: Bazzi W,Cattenoz PB,Delaporte C,Dasari V,Sakr R,Yuasa Y,Giangrande A

    更新日期:2018-07-11 00:00:00

  • Integration of genomics and transcriptomics predicts diabetic retinopathy susceptibility genes.

    abstract::We determined differential gene expression in response to high glucose in lymphoblastoid cell lines derived from matched individuals with type 1 diabetes with and without retinopathy. Those genes exhibiting the largest difference in glucose response were assessed for association with diabetic retinopathy in a genome-w...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.59980

    authors: Skol AD,Jung SC,Sokovic AM,Chen S,Fazal S,Sosina O,Borkar PP,Lin A,Sverdlov M,Cao D,Swaroop A,Bebu I,DCCT\/EDIC Study group.,Stranger BE,Grassi MA

    更新日期:2020-11-09 00:00:00

  • Kinesin-4 KIF21B limits microtubule growth to allow rapid centrosome polarization in T cells.

    abstract::When a T cell and an antigen-presenting cell form an immunological synapse, rapid dynein-driven translocation of the centrosome toward the contact site leads to reorganization of microtubules and associated organelles. Currently, little is known about how the regulation of microtubule dynamics contributes to this proc...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.62876

    authors: Hooikaas PJ,Damstra HG,Gros OJ,van Riel WE,Martin M,Smits YT,van Loosdregt J,Kapitein LC,Berger F,Akhmanova A

    更新日期:2020-12-21 00:00:00

  • An unexpected INAD PDZ tandem-mediated plcβ binding in Drosophila photo receptors.

    abstract::INAD assembles key enzymes of the Drosophila compound eye photo-transduction pathway into a supramolecular complex, supporting efficient and fast light signaling. However, the molecular mechanism that governs the interaction between INAD and NORPA (phospholipase Cβ, PLCβ), a key step for the fast kinetics of the light...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.41848

    authors: Ye F,Huang Y,Li J,Ma Y,Xie C,Liu Z,Deng X,Wan J,Xue T,Liu W,Zhang M

    更新日期:2018-12-10 00:00:00

  • Dynamics of venom composition across a complex life cycle.

    abstract::Little is known about venom in young developmental stages of animals. The appearance of toxins and stinging cells during early embryonic stages in the sea anemone Nematostella vectensis suggests that venom is already expressed in eggs and larvae of this species. Here, we harness transcriptomic, biochemical and transge...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.35014

    authors: Columbus-Shenkar YY,Sachkova MY,Macrander J,Fridrich A,Modepalli V,Reitzel AM,Sunagar K,Moran Y

    更新日期:2018-02-09 00:00:00

  • Behavioural diversity of bonobo prey preference as a potential cultural trait.

    abstract::The importance of cultural processes to behavioural diversity in our closest living relatives is central to revealing the evolutionary origins of human culture. However, the bonobo is often overlooked as a candidate model. Further, a prominent critique to many examples of proposed animal cultures is premature exclusio...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.59191

    authors: Samuni L,Wegdell F,Surbeck M

    更新日期:2020-09-01 00:00:00

  • Alterations of specific cortical GABAergic circuits underlie abnormal network activity in a mouse model of Down syndrome.

    abstract::Down syndrome (DS) results in various degrees of cognitive deficits. In DS mouse models, recovery of behavioral and neurophysiological deficits using GABAAR antagonists led to hypothesize an excessive activity of inhibitory circuits in this condition. Nonetheless, whether over-inhibition is present in DS and whether t...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.58731

    authors: Zorrilla de San Martin J,Donato C,Peixoto J,Aguirre A,Choudhary V,De Stasi AM,Lourenço J,Potier MC,Bacci A

    更新日期:2020-08-12 00:00:00

  • Rat anterior cingulate cortex recalls features of remote reward locations after disfavoured reinforcements.

    abstract::The anterior cingulate cortex (ACC) encodes information supporting mnemonic and cognitive processes. We show here that a rat's position can be decoded with high spatiotemporal resolution from ACC activity. ACC neurons encoded the current state of the animal and task, except for brief excursions that sometimes occurred...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.29793

    authors: Mashhoori A,Hashemnia S,McNaughton BL,Euston DR,Gruber AJ

    更新日期:2018-04-17 00:00:00

  • Perceptually relevant remapping of human somatotopy in 24 hours.

    abstract::Experience-dependent reorganisation of functional maps in the cerebral cortex is well described in the primary sensory cortices. However, there is relatively little evidence for such cortical reorganisation over the short-term. Using human somatosensory cortex as a model, we investigated the effects of a 24 hr gluing ...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.17280

    authors: Kolasinski J,Makin TR,Logan JP,Jbabdi S,Clare S,Stagg CJ,Johansen-Berg H

    更新日期:2016-12-30 00:00:00

  • HIV-1 integrase tetramers are the antiviral target of pyridine-based allosteric integrase inhibitors.

    abstract::Allosteric HIV-1 integrase (IN) inhibitors (ALLINIs) are a promising new class of antiretroviral agents that disrupt proper viral maturation by inducing hyper-multimerization of IN. Here we show that lead pyridine-based ALLINI KF116 exhibits striking selectivity for IN tetramers versus lower order protein oligomers. I...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.46344

    authors: Koneru PC,Francis AC,Deng N,Rebensburg SV,Hoyte AC,Lindenberger J,Adu-Ampratwum D,Larue RC,Wempe MF,Engelman AN,Lyumkis D,Fuchs JR,Levy RM,Melikyan GB,Kvaratskhelia M

    更新日期:2019-05-23 00:00:00

  • In vivo single-cell lineage tracing in zebrafish using high-resolution infrared laser-mediated gene induction microscopy.

    abstract::Heterogeneity broadly exists in various cell types both during development and at homeostasis. Investigating heterogeneity is crucial for comprehensively understanding the complexity of ontogeny, dynamics, and function of specific cell types. Traditional bulk-labeling techniques are incompetent to dissect heterogeneit...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.52024

    authors: He S,Tian Y,Feng S,Wu Y,Shen X,Chen K,He Y,Sun Q,Li X,Xu J,Wen Z,Qu JY

    更新日期:2020-01-06 00:00:00

  • Akt regulation of glycolysis mediates bioenergetic stability in epithelial cells.

    abstract::Cells use multiple feedback controls to regulate metabolism in response to nutrient and signaling inputs. However, feedback creates the potential for unstable network responses. We examined how concentrations of key metabolites and signaling pathways interact to maintain homeostasis in proliferating human cells, using...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.27293

    authors: Hung YP,Teragawa C,Kosaisawe N,Gillies TE,Pargett M,Minguet M,Distor K,Rocha-Gregg BL,Coloff JL,Keibler MA,Stephanopoulos G,Yellen G,Brugge JS,Albeck JG

    更新日期:2017-12-14 00:00:00

  • Shade-induced nuclear localization of PIF7 is regulated by phosphorylation and 14-3-3 proteins in Arabidopsis.

    abstract::Shade avoidance syndrome enables shaded plants to grow and compete effectively against their neighbors. In Arabidopsis, the shade-induced de-phosphorylation of the transcription factor PIF7 (PHYTOCHROME-INTERACTING FACTOR 7) is the key event linking light perception to stem elongation. However, the mechanism through w...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.31636

    authors: Huang X,Zhang Q,Jiang Y,Yang C,Wang Q,Li L

    更新日期:2018-06-21 00:00:00

  • The double-edged sword of evolution.

    abstract::Two gene variants provide different levels of protection against sleeping sickness, but this comes with an increased risk of developing chronic kidney disease. ...

    journal_title:eLife

    pub_type: 评论,杂志文章

    doi:10.7554/eLife.29056

    authors: Kruzel-Davila E,Skorecki K

    更新日期:2017-07-03 00:00:00

  • Freshwater monitoring by nanopore sequencing.

    abstract::While traditional microbiological freshwater tests focus on the detection of specific bacterial indicator species, including pathogens, direct tracing of all aquatic DNA through metagenomics poses a profound alternative. Yet, in situ metagenomic water surveys face substantial challenges in cost and logistics. Here, we...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.61504

    authors: Urban L,Holzer A,Baronas JJ,Hall MB,Braeuninger-Weimer P,Scherm MJ,Kunz DJ,Perera SN,Martin-Herranz DE,Tipper ET,Salter SJ,Stammnitz MR

    更新日期:2021-01-19 00:00:00

  • Identification of a molecular basis for the juvenile sleep state.

    abstract::Across species, sleep in young animals is critical for normal brain maturation. The molecular determinants of early life sleep remain unknown. Through an RNAi-based screen, we identified a gene, pdm3, required for sleep maturation in Drosophila. Pdm3, a transcription factor, coordinates an early developmental program ...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.52676

    authors: Chakravarti Dilley L,Szuperak M,Gong NN,Williams CE,Saldana RL,Garbe DS,Syed MH,Jain R,Kayser MS

    更新日期:2020-03-23 00:00:00

  • Sex-based differences in clearance of chronic Plasmodium falciparum infection.

    abstract::Multiple studies have reported a male bias in incidence and/or prevalence of malaria infection in males compared to females. To test the hypothesis that sex-based differences in host-parasite interactions affect the epidemiology of malaria, we intensively followed Plasmodium falciparum infections in a cohort in a mala...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.59872

    authors: Briggs J,Teyssier N,Nankabirwa JI,Rek J,Jagannathan P,Arinaitwe E,Bousema T,Drakeley C,Murray M,Crawford E,Hathaway N,Staedke SG,Smith D,Rosenthal PJ,Kamya M,Dorsey G,Rodriguez-Barraquer I,Greenhouse B

    更新日期:2020-10-27 00:00:00

  • Odd-paired is a pioneer-like factor that coordinates with Zelda to control gene expression in embryos.

    abstract::Pioneer factors such as Zelda (Zld) help initiate zygotic transcription in Drosophila early embryos, but whether other factors support this dynamic process is unclear. Odd-paired (Opa), a zinc-finger transcription factor expressed at cellularization, controls the transition of genes from pair-rule to segmental pattern...

    journal_title:eLife

    pub_type: 评论,杂志文章

    doi:10.7554/eLife.59610

    authors: Koromila T,Gao F,Iwasaki Y,He P,Pachter L,Gergen JP,Stathopoulos A

    更新日期:2020-07-23 00:00:00

  • A counter gradient of Activin A and follistatin instructs the timing of hair cell differentiation in the murine cochlea.

    abstract::The mammalian auditory sensory epithelium has one of the most stereotyped cellular patterns known in vertebrates. Mechano-sensory hair cells are arranged in precise rows, with one row of inner and three rows of outer hair cells spanning the length of the spiral-shaped sensory epithelium. Aiding such precise cellular p...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.47613

    authors: Prajapati-DiNubila M,Benito-Gonzalez A,Golden EJ,Zhang S,Doetzlhofer A

    更新日期:2019-06-12 00:00:00

  • Importance of miRNA stability and alternative primary miRNA isoforms in gene regulation during Drosophila development.

    abstract::Mature microRNAs (miRNAs) are processed from primary transcripts (pri-miRNAs), and their expression is controlled at transcriptional and post-transcriptional levels. However, how regulation at multiple levels achieves precise control remains elusive. Using published and new datasets, we profile a time course of mature...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.38389

    authors: Zhou L,Lim MYT,Kaur P,Saj A,Bortolamiol-Becet D,Gopal V,Tolwinski N,Tucker-Kellogg G,Okamura K

    更新日期:2018-07-19 00:00:00

  • Area 2 of primary somatosensory cortex encodes kinematics of the whole arm.

    abstract::Proprioception, the sense of body position, movement, and associated forces, remains poorly understood, despite its critical role in movement. Most studies of area 2, a proprioceptive area of somatosensory cortex, have simply compared neurons' activities to the movement of the hand through space. Using motion tracking...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.48198

    authors: Chowdhury RH,Glaser JI,Miller LE

    更新日期:2020-01-23 00:00:00

  • The Caenorhabditis elegans microtubule minus-end binding homolog PTRN-1 stabilizes synapses and neurites.

    abstract::Microtubule dynamics facilitate neurite growth and establish morphology, but the role of minus-end binding proteins in these processes is largely unexplored. CAMSAP homologs associate with microtubule minus-ends, and are important for the stability of epithelial cell adhesions. In this study, we report morphological d...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.01637

    authors: Marcette JD,Chen JJ,Nonet ML

    更新日期:2014-02-25 00:00:00

  • Ragulator and GATOR1 complexes promote fission yeast growth by attenuating TOR complex 1 through Rag GTPases.

    abstract::TOR complex 1 (TORC1) is an evolutionarily conserved protein kinase complex that promotes cellular macromolecular synthesis and suppresses autophagy. Amino-acid-induced activation of mammalian TORC1 is initiated by its recruitment to the RagA/B-RagC/D GTPase heterodimer, which is anchored to lysosomal membranes throug...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.30880

    authors: Chia KH,Fukuda T,Sofyantoro F,Matsuda T,Amai T,Shiozaki K

    更新日期:2017-12-04 00:00:00

  • Optogenetic investigation of BMP target gene expression diversity.

    abstract::Signaling molecules activate distinct patterns of gene expression to coordinate embryogenesis, but how spatiotemporal expression diversity is generated is an open question. In zebrafish, a BMP signaling gradient patterns the dorsal-ventral axis. We systematically identified target genes responding to BMP and found tha...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.58641

    authors: Rogers KW,ElGamacy M,Jordan BM,Müller P

    更新日期:2020-11-11 00:00:00

  • Tissue-autonomous immune response regulates stress signalling during hypertrophy.

    abstract::Postmitotic tissues are incapable of replacing damaged cells through proliferation, but need to rely on buffering mechanisms to prevent tissue disintegration. By constitutively activating the Ras/MAPK-pathway via RasV12-overexpression in the postmitotic salivary glands of Drosophila larvae, we overrode the glands adap...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.64919

    authors: Krautz R,Khalili D,Theopold U

    更新日期:2020-12-30 00:00:00

  • Chemical structure-guided design of dynapyrazoles, cell-permeable dynein inhibitors with a unique mode of action.

    abstract::Cytoplasmic dyneins are motor proteins in the AAA+ superfamily that transport cellular cargos toward microtubule minus-ends. Recently, ciliobrevins were reported as selective cell-permeable inhibitors of cytoplasmic dyneins. As is often true for first-in-class inhibitors, the use of ciliobrevins has in part been limit...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.25174

    authors: Steinman JB,Santarossa CC,Miller RM,Yu LS,Serpinskaya AS,Furukawa H,Morimoto S,Tanaka Y,Nishitani M,Asano M,Zalyte R,Ondrus AE,Johnson AG,Ye F,Nachury MV,Fukase Y,Aso K,Foley MA,Gelfand VI,Chen JK,Carter AP,Kapo

    更新日期:2017-05-19 00:00:00

  • Time-resolved studies define the nature of toxic IAPP intermediates, providing insight for anti-amyloidosis therapeutics.

    abstract::Islet amyloidosis by IAPP contributes to pancreatic β-cell death in diabetes, but the nature of toxic IAPP species remains elusive. Using concurrent time-resolved biophysical and biological measurements, we define the toxic species produced during IAPP amyloid formation and link their properties to induction of rat IN...

    journal_title:eLife

    pub_type: 杂志文章

    doi:10.7554/eLife.12977

    authors: Abedini A,Plesner A,Cao P,Ridgway Z,Zhang J,Tu LH,Middleton CT,Chao B,Sartori DJ,Meng F,Wang H,Wong AG,Zanni MT,Verchere CB,Raleigh DP,Schmidt AM

    更新日期:2016-05-23 00:00:00