Multimodal Neural Network for Rapid Serial Visual Presentation Brain Computer Interface.

Abstract:

:Brain computer interfaces allow users to preform various tasks using only the electrical activity of the brain. BCI applications often present the user a set of stimuli and record the corresponding electrical response. The BCI algorithm will then have to decode the acquired brain response and perform the desired task. In rapid serial visual presentation (RSVP) tasks, the subject is presented with a continuous stream of images containing rare target images among standard images, while the algorithm has to detect brain activity associated with target images. In this work, we suggest a multimodal neural network for RSVP tasks. The network operates on the brain response and on the initiating stimulus simultaneously, providing more information for the BCI application. We present two variants of the multimodal network, a supervised model, for the case when the targets are known in advanced, and a semi-supervised model for when the targets are unknown. We test the neural networks with a RSVP experiment on satellite imagery carried out with two subjects. The multimodal networks achieve a significant performance improvement in classification metrics. We visualize what the networks has learned and discuss the advantages of using neural network models for BCI applications.

journal_name

Front Comput Neurosci

authors

Manor R,Mishali L,Geva AB

doi

10.3389/fncom.2016.00130

subject

Has Abstract

pub_date

2016-12-20 00:00:00

pages

130

issn

1662-5188

journal_volume

10

pub_type

杂志文章
  • Deep Learning-Based Concurrent Brain Registration and Tumor Segmentation.

    abstract::Image registration and segmentation are the two most studied problems in medical image analysis. Deep learning algorithms have recently gained a lot of attention due to their success and state-of-the-art results in variety of problems and communities. In this paper, we propose a novel, efficient, and multi-task algori...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2020.00017

    authors: Estienne T,Lerousseau M,Vakalopoulou M,Alvarez Andres E,Battistella E,Carré A,Chandra S,Christodoulidis S,Sahasrabudhe M,Sun R,Robert C,Talbot H,Paragios N,Deutsch E

    更新日期:2020-03-20 00:00:00

  • Bayesian Inference of Synaptic Quantal Parameters from Correlated Vesicle Release.

    abstract::Synaptic transmission is both history-dependent and stochastic, resulting in varying responses to presentations of the same presynaptic stimulus. This complicates attempts to infer synaptic parameters and has led to the proposal of a number of different strategies for their quantification. Recently Bayesian approaches...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00116

    authors: Bird AD,Wall MJ,Richardson MJ

    更新日期:2016-11-25 00:00:00

  • Sensitivity analysis for the EEG forward problem.

    abstract::Sensitivity analysis can provide useful information when one is interested in identifying the parameter θ of a system since it measures the variations of the output u when θ changes. In the literature two different sensitivity functions are frequently used: the traditional sensitivity functions (TSF) and the generaliz...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2010.00138

    authors: Troparevsky MI,Rubio D,Saintier N

    更新日期:2010-09-30 00:00:00

  • Statistical physics of pairwise probability models.

    abstract::Statistical models for describing the probability distribution over the states of biological systems are commonly used for dimensional reduction. Among these models, pairwise models are very attractive in part because they can be fit using a reasonable amount of data: knowledge of the mean values and correlations betw...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/neuro.10.022.2009

    authors: Roudi Y,Aurell E,Hertz JA

    更新日期:2009-11-17 00:00:00

  • Is attentional blink a byproduct of neocortical attractors?

    abstract::This study proposes a computational model for attentional blink or "blink of the mind," a phenomenon where a human subject misses perception of a later expected visual pattern as two expected visual patterns are presented less than 500 ms apart. A neocortical patch modeled as an attractor network is stimulated with a ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2011.00013

    authors: Silverstein DN,Lansner A

    更新日期:2011-05-03 00:00:00

  • Causal Role of Thalamic Interneurons in Brain State Transitions: A Study Using a Neural Mass Model Implementing Synaptic Kinetics.

    abstract::Experimental studies on the Lateral Geniculate Nucleus (LGN) of mammals and rodents show that the inhibitory interneurons (IN) receive around 47.1% of their afferents from the retinal spiking neurons, and constitute around 20-25% of the LGN cell population. However, there is a definite gap in knowledge about the role ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00115

    authors: Bhattacharya BS,Bond TP,O'Hare L,Turner D,Durrant SJ

    更新日期:2016-11-16 00:00:00

  • A model-based approach to predict muscle synergies using optimization: application to feedback control.

    abstract::This paper presents a new model-based method to define muscle synergies. Unlike the conventional factorization approach, which extracts synergies from electromyographic data, the proposed method employs a biomechanical model and formally defines the synergies as the solution of an optimal control problem. As a result,...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2015.00121

    authors: Sharif Razavian R,Mehrabi N,McPhee J

    更新日期:2015-10-06 00:00:00

  • Analog Signaling With the "Digital" Molecular Switch CaMKII.

    abstract::Molecular switches, such as the protein kinase CaMKII, play a fundamental role in cell signaling by decoding inputs into either high or low states of activity; because the high activation state can be turned on and persist after the input ceases, these switches have earned a reputation as "digital." Although this on/o...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2018.00092

    authors: Clarke SE

    更新日期:2018-11-22 00:00:00

  • Dopamine-signaled reward predictions generated by competitive excitation and inhibition in a spiking neural network model.

    abstract::Dopaminergic neurons in the mammalian substantia nigra display characteristic phasic responses to stimuli which reliably predict the receipt of primary rewards. These responses have been suggested to encode reward prediction-errors similar to those used in reinforcement learning. Here, we propose a model of dopaminerg...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2011.00021

    authors: Chorley P,Seth AK

    更新日期:2011-05-18 00:00:00

  • Dynamical Graph Theory Networks Methods for the Analysis of Sparse Functional Connectivity Networks and for Determining Pinning Observability in Brain Networks.

    abstract::Neuroimaging in combination with graph theory has been successful in analyzing the functional connectome. However almost all analysis are performed based on static graph theory. The derived quantitative graph measures can only describe a snap shot of the disease over time. Neurodegenerative disease evolution is poorly...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2017.00087

    authors: Meyer-Bäse A,Roberts RG,Illan IA,Meyer-Bäse U,Lobbes M,Stadlbauer A,Pinker-Domenig K

    更新日期:2017-10-05 00:00:00

  • MACOP modular architecture with control primitives.

    abstract::Walking, catching a ball and reaching are all tasks in which humans and animals exhibit advanced motor skills. Findings in biological research concerning motor control suggest a modular control hierarchy which combines movement/motor primitives into complex and natural movements. Engineers inspire their research on th...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00099

    authors: Waegeman T,Hermans M,Schrauwen B

    更新日期:2013-07-23 00:00:00

  • Learning modular policies for robotics.

    abstract::A promising idea for scaling robot learning to more complex tasks is to use elemental behaviors as building blocks to compose more complex behavior. Ideally, such building blocks are used in combination with a learning algorithm that is able to learn to select, adapt, sequence and co-activate the building blocks. Whil...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章,评审

    doi:10.3389/fncom.2014.00062

    authors: Neumann G,Daniel C,Paraschos A,Kupcsik A,Peters J

    更新日期:2014-06-11 00:00:00

  • Learning Generative State Space Models for Active Inference.

    abstract::In this paper we investigate the active inference framework as a means to enable autonomous behavior in artificial agents. Active inference is a theoretical framework underpinning the way organisms act and observe in the real world. In active inference, agents act in order to minimize their so called free energy, or p...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2020.574372

    authors: Çatal O,Wauthier S,De Boom C,Verbelen T,Dhoedt B

    更新日期:2020-11-16 00:00:00

  • Direct connections assist neurons to detect correlation in small amplitude noises.

    abstract::We address a question on the effect of common stochastic inputs on the correlation of the spike trains of two neurons when they are coupled through direct connections. We show that the change in the correlation of small amplitude stochastic inputs can be better detected when the neurons are connected by direct excitat...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00108

    authors: Bolhasani E,Azizi Y,Valizadeh A

    更新日期:2013-08-14 00:00:00

  • A high-capacity model for one shot association learning in the brain.

    abstract::We present a high-capacity model for one-shot association learning (hetero-associative memory) in sparse networks. We assume that basic patterns are pre-learned in networks and associations between two patterns are presented only once and have to be learned immediately. The model is a combination of an Amit-Fusi like ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2014.00140

    authors: Einarsson H,Lengler J,Steger A

    更新日期:2014-11-07 00:00:00

  • Architectural constraints are a major factor reducing path integration accuracy in the rat head direction cell system.

    abstract::Head direction cells fire to signal the direction in which an animal's head is pointing. They are able to track head direction using only internally-derived information (path integration)In this simulation study we investigate the factors that affect path integration accuracy. Specifically, two major limiting factors ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2015.00010

    authors: Page HJ,Walters D,Stringer SM

    更新日期:2015-02-06 00:00:00

  • A stimulus-dependent spike threshold is an optimal neural coder.

    abstract::A neural code based on sequences of spikes can consume a significant portion of the brain's energy budget. Thus, energy considerations would dictate that spiking activity be kept as low as possible. However, a high spike-rate improves the coding and representation of signals in spike trains, particularly in sensory sy...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2015.00061

    authors: Jones DL,Johnson EC,Ratnam R

    更新日期:2015-06-02 00:00:00

  • A model of food reward learning with dynamic reward exposure.

    abstract::The process of conditioning via reward learning is highly relevant to the study of food choice and obesity. Learning is itself shaped by environmental exposure, with the potential for such exposures to vary substantially across individuals and across place and time. In this paper, we use computational techniques to ex...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2012.00082

    authors: Hammond RA,Ornstein JT,Fellows LK,Dubé L,Levitan R,Dagher A

    更新日期:2012-10-11 00:00:00

  • Stability constraints on large-scale structural brain networks.

    abstract::Stability is an important dynamical property of complex systems and underpins a broad range of coherent self-organized behavior. Based on evidence that some neurological disorders correspond to linear instabilities, we hypothesize that stability constrains the brain's electrical activity and influences its structure a...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00031

    authors: Gray RT,Robinson PA

    更新日期:2013-04-12 00:00:00

  • An Approximation to the Adaptive Exponential Integrate-and-Fire Neuron Model Allows Fast and Predictive Fitting to Physiological Data.

    abstract::For large-scale network simulations, it is often desirable to have computationally tractable, yet in a defined sense still physiologically valid neuron models. In particular, these models should be able to reproduce physiological measurements, ideally in a predictive sense, and under different input regimes in which n...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2012.00062

    authors: Hertäg L,Hass J,Golovko T,Durstewitz D

    更新日期:2012-09-06 00:00:00

  • Inhibition potentiates the synchronizing action of electrical synapses.

    abstract::In vivo and in vitro experimental studies have found that blocking electrical interactions connecting GABAergic interneurons reduces oscillatory activity in the gamma range in cortex. However, recent theoretical works have shown that the ability of electrical synapses to promote or impede synchrony, when alone, depend...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/neuro.10.008.2007

    authors: Pfeuty B,Golomb D,Mato G,Hansel D

    更新日期:2007-11-02 00:00:00

  • Spectral Entropy Based Neuronal Network Synchronization Analysis Based on Microelectrode Array Measurements.

    abstract::Synchrony and asynchrony are essential aspects of the functioning of interconnected neuronal cells and networks. New information on neuronal synchronization can be expected to aid in understanding these systems. Synchronization provides insight in the functional connectivity and the spatial distribution of the informa...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00112

    authors: Kapucu FE,Välkki I,Mikkonen JE,Leone C,Lenk K,Tanskanen JM,Hyttinen JA

    更新日期:2016-10-18 00:00:00

  • Structural Plasticity Denoises Responses and Improves Learning Speed.

    abstract::Despite an abundance of computational models for learning of synaptic weights, there has been relatively little research on structural plasticity, i.e., the creation and elimination of synapses. Especially, it is not clear how structural plasticity works in concert with spike-timing-dependent plasticity (STDP) and wha...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00093

    authors: Spiess R,George R,Cook M,Diehl PU

    更新日期:2016-09-08 00:00:00

  • Unsupervised Domain Adaptation With Optimal Transport in Multi-Site Segmentation of Multiple Sclerosis Lesions From MRI Data.

    abstract::Automatic segmentation of Multiple Sclerosis (MS) lesions from Magnetic Resonance Imaging (MRI) images is essential for clinical assessment and treatment planning of MS. Recent years have seen an increasing use of Convolutional Neural Networks (CNNs) for this task. Although these methods provide accurate segmentation,...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2020.00019

    authors: Ackaouy A,Courty N,Vallée E,Commowick O,Barillot C,Galassi F

    更新日期:2020-03-09 00:00:00

  • ARTIE: An Integrated Environment for the Development of Affective Robot Tutors.

    abstract::Over the last decade robotics has attracted a great deal of interest from teachers and researchers as a valuable educational tool from preschool to highschool levels. The implementation of social-support behaviors in robot tutors, in particular in the emotional dimension, can make a significant contribution to learnin...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2016.00077

    authors: Imbernón Cuadrado LE,Manjarrés Riesco Á,De La Paz López F

    更新日期:2016-08-03 00:00:00

  • Stochastic Resonance Based Visual Perception Using Spiking Neural Networks.

    abstract::Our aim is to propose an efficient algorithm for enhancing the contrast of dark images based on the principle of stochastic resonance in a global feedback spiking network of integrate-and-fire neurons. By linear approximation and direct simulation, we disclose the dependence of the peak signal-to-noise ratio on the sp...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2020.00024

    authors: Fu Y,Kang Y,Chen G

    更新日期:2020-05-15 00:00:00

  • Steering the Volume of Tissue Activated With a Directional Deep Brain Stimulation Lead in the Globus Pallidus Pars Interna: A Modeling Study With Heterogeneous Tissue Properties.

    abstract::Objective: To study the effect of directional deep brain stimulation (DBS) electrode configuration and vertical electrode spacing on the volume of tissue activated (VTA) in the globus pallidus, pars interna (GPi). Background: Directional DBS leads may allow clinicians to precisely direct current fields to different fu...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2020.561180

    authors: Zhang S,Tagliati M,Pouratian N,Cheeran B,Ross E,Pereira E

    更新日期:2020-09-25 00:00:00

  • Multiple Frequency Bands Analysis of Large Scale Intrinsic Brain Networks and Its Application in Schizotypal Personality Disorder.

    abstract::The human brain is a complex system composed by several large scale intrinsic networks with distinct functions. The low frequency oscillation (LFO) signal of blood oxygen level dependent (BOLD), measured through resting-state fMRI, reflects the spontaneous neural activity of these networks. We propose to characterize ...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2018.00064

    authors: Qi S,Gao Q,Shen J,Teng Y,Xie X,Sun Y,Wu J

    更新日期:2018-08-03 00:00:00

  • Visual motion integration is mediated by directional ambiguities in local motion signals.

    abstract::The output of primary visual cortex (V1) is a piecemeal representation of the visual scene and the response of any one cell cannot unambiguously guide sensorimotor behavior. It remains unsolved how subsequent stages of cortical processing combine ("pool") these early visual signals into a coherent representation. We (...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00167

    authors: Rocchi F,Ledgeway T,Webb BS

    更新日期:2013-11-18 00:00:00

  • Anisotropic connectivity implements motion-based prediction in a spiking neural network.

    abstract::Predictive coding hypothesizes that the brain explicitly infers upcoming sensory input to establish a coherent representation of the world. Although it is becoming generally accepted, it is not clear on which level spiking neural networks may implement predictive coding and what function their connectivity may have. W...

    journal_title:Frontiers in computational neuroscience

    pub_type: 杂志文章

    doi:10.3389/fncom.2013.00112

    authors: Kaplan BA,Lansner A,Masson GS,Perrinet LU

    更新日期:2013-09-17 00:00:00