Learning modular policies for robotics.

Abstract:

:A promising idea for scaling robot learning to more complex tasks is to use elemental behaviors as building blocks to compose more complex behavior. Ideally, such building blocks are used in combination with a learning algorithm that is able to learn to select, adapt, sequence and co-activate the building blocks. While there has been a lot of work on approaches that support one of these requirements, no learning algorithm exists that unifies all these properties in one framework. In this paper we present our work on a unified approach for learning such a modular control architecture. We introduce new policy search algorithms that are based on information-theoretic principles and are able to learn to select, adapt and sequence the building blocks. Furthermore, we developed a new representation for the individual building block that supports co-activation and principled ways for adapting the movement. Finally, we summarize our experiments for learning modular control architectures in simulation and with real robots.

journal_name

Front Comput Neurosci

authors

Neumann G,Daniel C,Paraschos A,Kupcsik A,Peters J

doi

10.3389/fncom.2014.00062

subject

Has Abstract

pub_date

2014-06-11 00:00:00

pages

62

issn

1662-5188

journal_volume

8

pub_type

杂志文章,评审
  • An Approximation to the Adaptive Exponential Integrate-and-Fire Neuron Model Allows Fast and Predictive Fitting to Physiological Data.

    abstract::For large-scale network simulations, it is often desirable to have computationally tractable, yet in a defined sense still physiologically valid neuron models. In particular, these models should be able to reproduce physiological measurements, ideally in a predictive sense, and under different input regimes in which n...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2012.00062

    authors: Hertäg L,Hass J,Golovko T,Durstewitz D

    更新日期:2012-09-06 00:00:00

  • Tonality Tunes the Statistical Characteristics in Music: Computational Approaches on Statistical Learning.

    abstract::Statistical learning is a learning mechanism based on transition probability in sequences such as music and language. Recent computational and neurophysiological studies suggest that the statistical learning contributes to production, action, and musical creativity as well as prediction and perception. The present stu...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2019.00070

    authors: Daikoku T

    更新日期:2019-10-02 00:00:00

  • Optimizing Clinical Assessments in Parkinson's Disease Through the Use of Wearable Sensors and Data Driven Modeling.

    abstract::The emergence of motion sensors as a tool that provides objective motor performance data on individuals afflicted with Parkinson's disease offers an opportunity to expand the horizon of clinical care for this neurodegenerative condition. Subjective clinical scales and patient based motor diaries have limited clinometr...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章,评审

    doi:10.3389/fncom.2018.00072

    authors: Ramdhani RA,Khojandi A,Shylo O,Kopell BH

    更新日期:2018-09-11 00:00:00

  • Multimodal Neural Network for Rapid Serial Visual Presentation Brain Computer Interface.

    abstract::Brain computer interfaces allow users to preform various tasks using only the electrical activity of the brain. BCI applications often present the user a set of stimuli and record the corresponding electrical response. The BCI algorithm will then have to decode the acquired brain response and perform the desired task....

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00130

    authors: Manor R,Mishali L,Geva AB

    更新日期:2016-12-20 00:00:00

  • Invariant object recognition based on extended fragments.

    abstract::Visual appearance of natural objects is profoundly affected by viewing conditions such as viewpoint and illumination. Human subjects can nevertheless compensate well for variations in these viewing conditions. The strategies that the visual system uses to accomplish this are largely unclear. Previous computational stu...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2012.00056

    authors: Bart E,Hegdé J

    更新日期:2012-08-24 00:00:00

  • Spatiotemporal imaging of complexity.

    abstract::What are the functional neuroimaging measurements required for more fully characterizing the events and locations of neocortical activity? A prime assumption has been that modulation of cortical activity will inevitably be reflected in changes in energy utilization (for the most part) changes of glucose and oxygen con...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2012.00101

    authors: Robinson SE,Mandell AJ,Coppola R

    更新日期:2013-01-24 00:00:00

  • Alterations of Muscle Synergies During Voluntary Arm Reaching Movement in Subacute Stroke Survivors at Different Levels of Impairment.

    abstract::Motor system uses muscle synergies as a modular organization to simplify the control of movements. Motor cortical impairments, such as stroke and spinal cord injuries, disrupt the orchestration of the muscle synergies and result in abnormal movements. In this paper, the alterations of muscle synergies in subacute stro...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2018.00069

    authors: Pan B,Sun Y,Xie B,Huang Z,Wu J,Hou J,Liu Y,Huang Z,Zhang Z

    更新日期:2018-08-21 00:00:00

  • Short-Term Facilitation may Stabilize Parametric Working Memory Trace.

    abstract::Networks with continuous set of attractors are considered to be a paradigmatic model for parametric working memory (WM), but require fine tuning of connections and are thus structurally unstable. Here we analyzed the network with ring attractor, where connections are not perfectly tuned and the activity state therefor...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2011.00040

    authors: Itskov V,Hansel D,Tsodyks M

    更新日期:2011-10-24 00:00:00

  • Causal Role of Thalamic Interneurons in Brain State Transitions: A Study Using a Neural Mass Model Implementing Synaptic Kinetics.

    abstract::Experimental studies on the Lateral Geniculate Nucleus (LGN) of mammals and rodents show that the inhibitory interneurons (IN) receive around 47.1% of their afferents from the retinal spiking neurons, and constitute around 20-25% of the LGN cell population. However, there is a definite gap in knowledge about the role ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00115

    authors: Bhattacharya BS,Bond TP,O'Hare L,Turner D,Durrant SJ

    更新日期:2016-11-16 00:00:00

  • Unsupervised Domain Adaptation With Optimal Transport in Multi-Site Segmentation of Multiple Sclerosis Lesions From MRI Data.

    abstract::Automatic segmentation of Multiple Sclerosis (MS) lesions from Magnetic Resonance Imaging (MRI) images is essential for clinical assessment and treatment planning of MS. Recent years have seen an increasing use of Convolutional Neural Networks (CNNs) for this task. Although these methods provide accurate segmentation,...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2020.00019

    authors: Ackaouy A,Courty N,Vallée E,Commowick O,Barillot C,Galassi F

    更新日期:2020-03-09 00:00:00

  • Deep networks for motor control functions.

    abstract::The motor system generates time-varying commands to move our limbs and body. Conventional descriptions of motor control and learning rely on dynamical representations of our body's state (forward and inverse models), and control policies that must be integrated forward to generate feedforward time-varying commands; th...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2015.00032

    authors: Berniker M,Kording KP

    更新日期:2015-03-19 00:00:00

  • Causal Inference for Cross-Modal Action Selection: A Computational Study in a Decision Making Framework.

    abstract::Animals try to make sense of sensory information from multiple modalities by categorizing them into perceptions of individual or multiple external objects or internal concepts. For example, the brain constructs sensory, spatial representations of the locations of visual and auditory stimuli in the visual and auditory ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00062

    authors: Daemi M,Harris LR,Crawford JD

    更新日期:2016-06-23 00:00:00

  • Probabilistic inference of short-term synaptic plasticity in neocortical microcircuits.

    abstract::Short-term synaptic plasticity is highly diverse across brain area, cortical layer, cell type, and developmental stage. Since short-term plasticity (STP) strongly shapes neural dynamics, this diversity suggests a specific and essential role in neural information processing. Therefore, a correct characterization of sho...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00075

    authors: Costa RP,Sjöström PJ,van Rossum MC

    更新日期:2013-06-06 00:00:00

  • A micro-pool model for decision-related signals in visual cortical areas.

    abstract::The study of sensory signaling in the visual cortex has been greatly advanced by the recording of neural activity simultaneously with the performance of a specific psychophysical task. Individual nerve cells may also increase their firing leading up to the particular choice or decision made on a single psychophysical ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00115

    authors: Parker AJ

    更新日期:2013-08-13 00:00:00

  • Analysis of Nociceptive Information Encoded in the Temporal Discharge Patterns of Cutaneous C-Fibers.

    abstract::The generation of pain signals from primary afferent neurons is explained by a labeled-line code. However, this notion cannot apply in a simple way to cutaneous C-fibers, which carry signals from a variety of receptors that respond to various stimuli including agonist chemicals. To represent the discharge patterns of ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00118

    authors: Cho K,Jang JH,Kim SP,Lee SH,Chung SC,Kim IY,Jang DP,Jung SJ

    更新日期:2016-11-18 00:00:00

  • Neural Coding With Bursts-Current State and Future Perspectives.

    abstract::Neuronal action potentials or spikes provide a long-range, noise-resistant means of communication between neurons. As point processes single spikes contain little information in themselves, i.e., outside the context of spikes from other neurons. Moreover, they may fail to cross a synapse. A burst, which consists of a ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章,评审

    doi:10.3389/fncom.2018.00048

    authors: Zeldenrust F,Wadman WJ,Englitz B

    更新日期:2018-07-06 00:00:00

  • Computational models of neuron-astrocyte interaction in epilepsy.

    abstract::Astrocytes actively shape the dynamics of neurons and neuronal ensembles by affecting several aspects critical to neuronal function, such as regulating synaptic plasticity, modulating neuronal excitability, and maintaining extracellular ion balance. These pathways for astrocyte-neuron interaction can also enhance the ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2012.00058

    authors: Volman V,Bazhenov M,Sejnowski TJ

    更新日期:2012-08-13 00:00:00

  • Visual Cortex Inspired CNN Model for Feature Construction in Text Analysis.

    abstract::Recently, biologically inspired models are gradually proposed to solve the problem in text analysis. Convolutional neural networks (CNN) are hierarchical artificial neural networks, which include a various of multilayer perceptrons. According to biological research, CNN can be improved by bringing in the attention mod...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00064

    authors: Fu H,Niu Z,Zhang C,Ma J,Chen J

    更新日期:2016-07-14 00:00:00

  • Neuromodulation impact on nonlinear firing behavior of a reduced model motoneuron with the active dendrite.

    abstract::Neuromodulatory inputs from brainstem systems modulate the normal function of spinal motoneurons by altering the activation properties of persistent inward currents (PICs) in their dendrites. However, the effect of the PIC on firing outputs also depends on its location in the dendritic tree. To investigate the interac...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2014.00110

    authors: Kim H,Heckman CJ

    更新日期:2014-09-09 00:00:00

  • Model selection for the extraction of movement primitives.

    abstract::A wide range of blind source separation methods have been used in motor control research for the extraction of movement primitives from EMG and kinematic data. Popular examples are principal component analysis (PCA), independent component analysis (ICA), anechoic demixing, and the time-varying synergy model (d'Avella ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00185

    authors: Endres DM,Chiovetto E,Giese MA

    更新日期:2013-12-20 00:00:00

  • Quantized response times are a signature of a neuronal bottleneck in decision.

    abstract::The histograms of response times of optimal YES/NO decisions that are computed from a single sensory Poisson neuron are highly structured. In particular, response times in NO decisions are quantized to a small set of times, while response times in YES decisions have a multimodal structure. Both the times of NO decisio...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2014.00042

    authors: Perona P

    更新日期:2014-04-11 00:00:00

  • Segmental Bayesian estimation of gap-junctional and inhibitory conductance of inferior olive neurons from spike trains with complicated dynamics.

    abstract::The inverse problem for estimating model parameters from brain spike data is an ill-posed problem because of a huge mismatch in the system complexity between the model and the brain as well as its non-stationary dynamics, and needs a stochastic approach that finds the most likely solution among many possible solutions...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2015.00056

    authors: Hoang H,Yamashita O,Tokuda IT,Sato MA,Kawato M,Toyama K

    更新日期:2015-05-21 00:00:00

  • Neural variability, or lack thereof.

    abstract::We do not claim that the brain is completely deterministic, and we agree that noise may be beneficial in some cases. But we suggest that neuronal variability may be often overestimated, due to uncontrolled internal variables, and/or the use of inappropriate reference times. These ideas are not new, but should be re-ex...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00007

    authors: Masquelier T

    更新日期:2013-02-25 00:00:00

  • Cancer Risk Analysis Based on Improved Probabilistic Neural Network.

    abstract::The problem of cancer risk analysis is of great importance to health-service providers and medical researchers. In this study, we propose a novel Artificial Neural Network (ANN) algorithm based on the probabilistic framework, which aims to investigate patient patterns associated with their disease development. Compare...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2020.00058

    authors: Yang C,Yang J,Liu Y,Geng X

    更新日期:2020-07-21 00:00:00

  • Learning view invariant recognition with partially occluded objects.

    abstract::This paper investigates how a neural network model of the ventral visual pathway, VisNet, can form separate view invariant representations of a number of objects seen rotating together. In particular, in the current work one of the rotating objects is always partially occluded by the other objects present during train...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2012.00048

    authors: Tromans JM,Higgins I,Stringer SM

    更新日期:2012-07-25 00:00:00

  • A Role for Electrotonic Coupling Between Cortical Pyramidal Cells.

    abstract::Many brain regions communicate information through synchronized network activity. Electrical coupling among the dendrites of interneurons in the cortex has been implicated in forming and sustaining such activity in the cortex. Evidence for the existence of electrical coupling among cortical pyramidal cells, however, h...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2019.00033

    authors: Crodelle J,Zhou D,Kovačič G,Cai D

    更新日期:2019-05-28 00:00:00

  • Disinhibition-Induced Delayed Onset of Epileptic Spike-Wave Discharges in a Five Variable Model of Cortex and Thalamus.

    abstract::Based on a modified neural field network model composed of cortex and thalamus, we here propose a computational framework to investigate the onset control of absence seizure, which is characterized by the spike-wave discharges. Firstly, we briefly demonstrate the existence of various transition types in Taylor's model...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00028

    authors: Liu S,Wang Q,Fan D

    更新日期:2016-04-05 00:00:00

  • Structural Plasticity Denoises Responses and Improves Learning Speed.

    abstract::Despite an abundance of computational models for learning of synaptic weights, there has been relatively little research on structural plasticity, i.e., the creation and elimination of synapses. Especially, it is not clear how structural plasticity works in concert with spike-timing-dependent plasticity (STDP) and wha...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00093

    authors: Spiess R,George R,Cook M,Diehl PU

    更新日期:2016-09-08 00:00:00

  • Network Models Predict That Pyramidal Neuron Hyperexcitability and Synapse Loss in the dlPFC Lead to Age-Related Spatial Working Memory Impairment in Rhesus Monkeys.

    abstract::Behavioral studies have shown spatial working memory impairment with aging in several animal species, including humans. Persistent activity of layer 3 pyramidal dorsolateral prefrontal cortex (dlPFC) neurons during delay periods of working memory tasks is important for encoding memory of the stimulus. In vitro studies...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2019.00089

    authors: Ibañez S,Luebke JI,Chang W,Draguljić D,Weaver CM

    更新日期:2020-01-17 00:00:00

  • Simultaneous learning and filtering without delusions: a Bayes-optimal combination of Predictive Inference and Adaptive Filtering.

    abstract::Predictive coding appears to be one of the fundamental working principles of brain processing. Amongst other aspects, brains often predict the sensory consequences of their own actions. Predictive coding resembles Kalman filtering, where incoming sensory information is filtered to produce prediction errors for subsequ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2015.00047

    authors: Kneissler J,Drugowitsch J,Friston K,Butz MV

    更新日期:2015-04-30 00:00:00