Learning Generative State Space Models for Active Inference.

Abstract:

:In this paper we investigate the active inference framework as a means to enable autonomous behavior in artificial agents. Active inference is a theoretical framework underpinning the way organisms act and observe in the real world. In active inference, agents act in order to minimize their so called free energy, or prediction error. Besides being biologically plausible, active inference has been shown to solve hard exploration problems in various simulated environments. However, these simulations typically require handcrafting a generative model for the agent. Therefore we propose to use recent advances in deep artificial neural networks to learn generative state space models from scratch, using only observation-action sequences. This way we are able to scale active inference to new and challenging problem domains, whilst still building on the theoretical backing of the free energy principle. We validate our approach on the mountain car problem to illustrate that our learnt models can indeed trade-off instrumental value and ambiguity. Furthermore, we show that generative models can also be learnt using high-dimensional pixel observations, both in the OpenAI Gym car racing environment and a real-world robotic navigation task. Finally we show that active inference based policies are an order of magnitude more sample efficient than Deep Q Networks on RL tasks.

journal_name

Front Comput Neurosci

authors

Çatal O,Wauthier S,De Boom C,Verbelen T,Dhoedt B

doi

10.3389/fncom.2020.574372

subject

Has Abstract

pub_date

2020-11-16 00:00:00

pages

574372

issn

1662-5188

journal_volume

14

pub_type

杂志文章
  • Multimodal Neural Network for Rapid Serial Visual Presentation Brain Computer Interface.

    abstract::Brain computer interfaces allow users to preform various tasks using only the electrical activity of the brain. BCI applications often present the user a set of stimuli and record the corresponding electrical response. The BCI algorithm will then have to decode the acquired brain response and perform the desired task....

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00130

    authors: Manor R,Mishali L,Geva AB

    更新日期:2016-12-20 00:00:00

  • A Phenomenological Model of the Electrically Stimulated Auditory Nerve Fiber: Temporal and Biphasic Response Properties.

    abstract::We present a phenomenological model of electrically stimulated auditory nerve fibers (ANFs). The model reproduces the probabilistic and temporal properties of the ANF response to both monophasic and biphasic stimuli, in isolation. The main contribution of the model lies in its ability to reproduce statistics of the AN...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00008

    authors: Horne CD,Sumner CJ,Seeber BU

    更新日期:2016-02-08 00:00:00

  • Visual motion integration is mediated by directional ambiguities in local motion signals.

    abstract::The output of primary visual cortex (V1) is a piecemeal representation of the visual scene and the response of any one cell cannot unambiguously guide sensorimotor behavior. It remains unsolved how subsequent stages of cortical processing combine ("pool") these early visual signals into a coherent representation. We (...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00167

    authors: Rocchi F,Ledgeway T,Webb BS

    更新日期:2013-11-18 00:00:00

  • Model-Based Comparison of Deep Brain Stimulation Array Functionality with Varying Number of Radial Electrodes and Machine Learning Feature Sets.

    abstract::Deep brain stimulation (DBS) leads with radially distributed electrodes have potential to improve clinical outcomes through more selective targeting of pathways and networks within the brain. However, increasing the number of electrodes on clinical DBS leads by replacing conventional cylindrical shell electrodes with ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00058

    authors: Teplitzky BA,Zitella LM,Xiao Y,Johnson MD

    更新日期:2016-06-10 00:00:00

  • A network that performs brute-force conversion of a temporal sequence to a spatial pattern: relevance to odor recognition.

    abstract::A classic problem in neuroscience is how temporal sequences (TSs) can be recognized. This problem is exemplified in the olfactory system, where an odor is defined by the TS of olfactory bulb (OB) output that occurs during a sniff. This sequence is discrete because the output is subdivided by gamma frequency oscillatio...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2014.00108

    authors: Sanders H,Kolterman BE,Shusterman R,Rinberg D,Koulakov A,Lisman J

    更新日期:2014-09-17 00:00:00

  • Modeling spontaneous activity across an excitable epithelium: Support for a coordination scenario of early neural evolution.

    abstract::Internal coordination models hold that early nervous systems evolved in the first place to coordinate internal activity at a multicellular level, most notably the use of multicellular contractility as an effector for motility. A recent example of such a model, the skin brain thesis, suggests that excitable epithelia u...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2015.00110

    authors: de Wiljes OO,van Elburg RA,Biehl M,Keijzer FA

    更新日期:2015-09-15 00:00:00

  • Stochastic Resonance Based Visual Perception Using Spiking Neural Networks.

    abstract::Our aim is to propose an efficient algorithm for enhancing the contrast of dark images based on the principle of stochastic resonance in a global feedback spiking network of integrate-and-fire neurons. By linear approximation and direct simulation, we disclose the dependence of the peak signal-to-noise ratio on the sp...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2020.00024

    authors: Fu Y,Kang Y,Chen G

    更新日期:2020-05-15 00:00:00

  • Learning view invariant recognition with partially occluded objects.

    abstract::This paper investigates how a neural network model of the ventral visual pathway, VisNet, can form separate view invariant representations of a number of objects seen rotating together. In particular, in the current work one of the rotating objects is always partially occluded by the other objects present during train...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2012.00048

    authors: Tromans JM,Higgins I,Stringer SM

    更新日期:2012-07-25 00:00:00

  • Input-output relation and energy efficiency in the neuron with different spike threshold dynamics.

    abstract::Neuron encodes and transmits information through generating sequences of output spikes, which is a high energy-consuming process. The spike is initiated when membrane depolarization reaches a threshold voltage. In many neurons, threshold is dynamic and depends on the rate of membrane depolarization (dV/dt) preceding a...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2015.00062

    authors: Yi GS,Wang J,Tsang KM,Wei XL,Deng B

    更新日期:2015-05-27 00:00:00

  • Spatiotemporal imaging of complexity.

    abstract::What are the functional neuroimaging measurements required for more fully characterizing the events and locations of neocortical activity? A prime assumption has been that modulation of cortical activity will inevitably be reflected in changes in energy utilization (for the most part) changes of glucose and oxygen con...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2012.00101

    authors: Robinson SE,Mandell AJ,Coppola R

    更新日期:2013-01-24 00:00:00

  • On the role of spatial phase and phase correlation in vision, illusion, and cognition.

    abstract::Numerous findings indicate that spatial phase bears an important cognitive information. Distortion of phase affects topology of edge structures and makes images unrecognizable. In turn, appropriately phase-structured patterns give rise to various illusions of virtual image content and apparent motion. Despite a large ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2015.00045

    authors: Gladilin E,Eils R

    更新日期:2015-04-21 00:00:00

  • Classification of EEG Signals Based on Pattern Recognition Approach.

    abstract::Feature extraction is an important step in the process of electroencephalogram (EEG) signal classification. The authors propose a "pattern recognition" approach that discriminates EEG signals recorded during different cognitive conditions. Wavelet based feature extraction such as, multi-resolution decompositions into ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2017.00103

    authors: Amin HU,Mumtaz W,Subhani AR,Saad MNM,Malik AS

    更新日期:2017-11-21 00:00:00

  • Computational Neural Modeling of Auditory Cortical Receptive Fields.

    abstract::Previous studies have shown that the auditory cortex can enhance the perception of behaviorally important sounds in the presence of background noise, but the mechanisms by which it does this are not yet elucidated. Rapid plasticity of spectrotemporal receptive fields (STRFs) in the primary (A1) cortical neurons is obs...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2019.00028

    authors: Chambers JD,Elgueda D,Fritz JB,Shamma SA,Burkitt AN,Grayden DB

    更新日期:2019-05-24 00:00:00

  • A Spiking Neural Model of HT3D for Corner Detection.

    abstract::Obtaining good quality image features is of remarkable importance for most computer vision tasks. It has been demonstrated that the first layers of the human visual cortex are devoted to feature detection. The need for these features has made line, segment, and corner detection one of the most studied topics in comput...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2018.00037

    authors: Bachiller-Burgos P,Manso LJ,Bustos P

    更新日期:2018-06-01 00:00:00

  • Nine criteria for a measure of scientific output.

    abstract::Scientific research produces new knowledge, technologies, and clinical treatments that can lead to enormous returns. Often, the path from basic research to new paradigms and direct impact on society takes time. Precise quantification of scientific output in the short-term is not an easy task but is critical for evalua...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2011.00048

    authors: Kreiman G,Maunsell JH

    更新日期:2011-11-10 00:00:00

  • Generation of Granule Cell Dendritic Morphologies by Estimating the Spatial Heterogeneity of Dendritic Branching.

    abstract::Biological realism of dendritic morphologies is important for simulating electrical stimulation of brain tissue. By adding point process modeling and conditional sampling to existing generation strategies, we provide a novel means of reproducing the nuanced branching behavior that occurs in different layers of granule...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2020.00023

    authors: Chou ZZ,Yu GJ,Berger TW

    更新日期:2020-04-09 00:00:00

  • Cooperation and Competition with Hyperscanning Methods: Review and Future Application to Emotion Domain.

    abstract::Cooperation and competition, as two common and opposite examples of interpersonal dynamics, are thought to be reflected by different cognitive, neural, and behavioral patterns. According to the conventional approach, they have been explored by measuring subjects' reactions during individual performance or turn-based i...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章,评审

    doi:10.3389/fncom.2017.00086

    authors: Balconi M,Vanutelli ME

    更新日期:2017-09-29 00:00:00

  • Impact of Physical Obstacles on the Structural and Effective Connectivity of in silico Neuronal Circuits.

    abstract::Scaffolds and patterned substrates are among the most successful strategies to dictate the connectivity between neurons in culture. Here, we used numerical simulations to investigate the capacity of physical obstacles placed on a flat substrate to shape structural connectivity, and in turn collective dynamics and effe...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2020.00077

    authors: Ludl AA,Soriano J

    更新日期:2020-08-31 00:00:00

  • A Neuronal Network Model for Pitch Selectivity and Representation.

    abstract::Pitch is a perceptual correlate of periodicity. Sounds with distinct spectra can elicit the same pitch. Despite the importance of pitch perception, understanding the cellular mechanism of pitch perception is still a major challenge and a mechanistic model of pitch is lacking. A multi-stage neuronal network model is de...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00057

    authors: Huang C,Rinzel J

    更新日期:2016-06-16 00:00:00

  • Neural Coding With Bursts-Current State and Future Perspectives.

    abstract::Neuronal action potentials or spikes provide a long-range, noise-resistant means of communication between neurons. As point processes single spikes contain little information in themselves, i.e., outside the context of spikes from other neurons. Moreover, they may fail to cross a synapse. A burst, which consists of a ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章,评审

    doi:10.3389/fncom.2018.00048

    authors: Zeldenrust F,Wadman WJ,Englitz B

    更新日期:2018-07-06 00:00:00

  • Model selection for the extraction of movement primitives.

    abstract::A wide range of blind source separation methods have been used in motor control research for the extraction of movement primitives from EMG and kinematic data. Popular examples are principal component analysis (PCA), independent component analysis (ICA), anechoic demixing, and the time-varying synergy model (d'Avella ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00185

    authors: Endres DM,Chiovetto E,Giese MA

    更新日期:2013-12-20 00:00:00

  • Simultaneous learning and filtering without delusions: a Bayes-optimal combination of Predictive Inference and Adaptive Filtering.

    abstract::Predictive coding appears to be one of the fundamental working principles of brain processing. Amongst other aspects, brains often predict the sensory consequences of their own actions. Predictive coding resembles Kalman filtering, where incoming sensory information is filtered to produce prediction errors for subsequ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2015.00047

    authors: Kneissler J,Drugowitsch J,Friston K,Butz MV

    更新日期:2015-04-30 00:00:00

  • Visual Cortex Inspired CNN Model for Feature Construction in Text Analysis.

    abstract::Recently, biologically inspired models are gradually proposed to solve the problem in text analysis. Convolutional neural networks (CNN) are hierarchical artificial neural networks, which include a various of multilayer perceptrons. According to biological research, CNN can be improved by bringing in the attention mod...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00064

    authors: Fu H,Niu Z,Zhang C,Ma J,Chen J

    更新日期:2016-07-14 00:00:00

  • Neural variability, or lack thereof.

    abstract::We do not claim that the brain is completely deterministic, and we agree that noise may be beneficial in some cases. But we suggest that neuronal variability may be often overestimated, due to uncontrolled internal variables, and/or the use of inappropriate reference times. These ideas are not new, but should be re-ex...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00007

    authors: Masquelier T

    更新日期:2013-02-25 00:00:00

  • Computational models of neuron-astrocyte interaction in epilepsy.

    abstract::Astrocytes actively shape the dynamics of neurons and neuronal ensembles by affecting several aspects critical to neuronal function, such as regulating synaptic plasticity, modulating neuronal excitability, and maintaining extracellular ion balance. These pathways for astrocyte-neuron interaction can also enhance the ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2012.00058

    authors: Volman V,Bazhenov M,Sejnowski TJ

    更新日期:2012-08-13 00:00:00

  • Analog Signaling With the "Digital" Molecular Switch CaMKII.

    abstract::Molecular switches, such as the protein kinase CaMKII, play a fundamental role in cell signaling by decoding inputs into either high or low states of activity; because the high activation state can be turned on and persist after the input ceases, these switches have earned a reputation as "digital." Although this on/o...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2018.00092

    authors: Clarke SE

    更新日期:2018-11-22 00:00:00

  • Effects of Calcium Spikes in the Layer 5 Pyramidal Neuron on Coincidence Detection and Activity Propagation.

    abstract::The role of dendritic spiking mechanisms in neural processing is so far poorly understood. To investigate the role of calcium spikes in the functional properties of the single neuron and recurrent networks, we investigated a three compartment neuron model of the layer 5 pyramidal neuron with calcium dynamics in the di...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00076

    authors: Chua Y,Morrison A

    更新日期:2016-07-22 00:00:00

  • Estimating neuronal connectivity from axonal and dendritic density fields.

    abstract::Neurons innervate space by extending axonal and dendritic arborizations. When axons and dendrites come in close proximity of each other, synapses between neurons can be formed. Neurons vary greatly in their morphologies and synaptic connections with other neurons. The size and shape of the arborizations determine the ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00160

    authors: van Pelt J,van Ooyen A

    更新日期:2013-11-25 00:00:00

  • Striatal Network Models of Huntington's Disease Dysfunction Phenotypes.

    abstract::We present a network model of striatum, which generates "winnerless" dynamics typical for a network of sparse, unidirectionally connected inhibitory units. We observe that these dynamics, while interesting and a good match to normal striatal electrophysiological recordings, are fragile. Specifically, we find that rand...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2017.00070

    authors: Zheng P,Kozloski J

    更新日期:2017-07-27 00:00:00

  • Direct connections assist neurons to detect correlation in small amplitude noises.

    abstract::We address a question on the effect of common stochastic inputs on the correlation of the spike trains of two neurons when they are coupled through direct connections. We show that the change in the correlation of small amplitude stochastic inputs can be better detected when the neurons are connected by direct excitat...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00108

    authors: Bolhasani E,Azizi Y,Valizadeh A

    更新日期:2013-08-14 00:00:00