当前位置： SCI文献检索 > NEURAL COMPUTATION期刊下所有文献 > State-Space Representations of Deep Neural Networks.

State-Space Representations of Deep Neural Networks.

Abstract：

:This letter deals with neural networks as dynamical systems governed by finite difference equations. It shows that the introduction of k -many skip connections into network architectures, such as residual networks and additive dense networks, defines k th order dynamical equations on the layer-wise transformations. Closed-form solutions for the state-space representations of general k th order additive dense networks, where the concatenation operation is replaced by addition, as well as k th order smooth networks, are found. The developed provision endows deep neural networks with an algebraic structure. Furthermore, it is shown that imposing k th order smoothness on network architectures with d -many nodes per layer increases the state-space dimension by a multiple of k , and so the effective embedding dimension of the data manifold by the neural network is k·d -many dimensions. It follows that network architectures of these types reduce the number of parameters needed to maintain the same embedding dimension by a factor of k2 when compared to an equivalent first-order, residual network. Numerical simulations and experiments on CIFAR10, SVHN, and MNIST have been conducted to help understand the developed theory and efficacy of the proposed concepts.

journal_name

Neural Comput

journal_title

Neural computation

authors

Hauser M,Gunn S,Saab S Jr,Ray A

doi

10.1162/neco_a_01165

subject

Has Abstract

pub_date

2019-03-01 00:00:00

pages

538-554

issue

3

eissn

0899-7667

issn

1530-888X

journal_volume

31

pub_type

杂志文章

相关文献

NEURAL COMPUTATION文献大全
  • Feature selection in simple neurons: how coding depends on spiking dynamics.

    abstract：:The relationship between a neuron's complex inputs and its spiking output defines the neuron's coding strategy. This is frequently and effectively modeled phenomenologically by one or more linear filters that extract the components of the stimulus that are relevant for triggering spikes and a nonlinear function that r...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco.2009.02-09-956

    authors： Famulare M,Fairhall A

    更新日期：2010-03-01 00:00:00

  • Enhanced stimulus encoding capabilities with spectral selectivity in inhibitory circuits by STDP.

    abstract：:The ability to encode and transmit a signal is an essential property that must demonstrate many neuronal circuits in sensory areas in addition to any processing they may provide. It is known that an appropriate level of lateral inhibition, as observed in these areas, can significantly improve the encoding ability of a...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/NECO_a_00100

    authors： Coulon A,Beslon G,Soula HA

    更新日期：2011-04-01 00:00:00

  • The computational structure of spike trains.

    abstract：:Neurons perform computations, and convey the results of those computations through the statistical structure of their output spike trains. Here we present a practical method, grounded in the information-theoretic analysis of prediction, for inferring a minimal representation of that structure and for characterizing it...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco.2009.12-07-678

    authors： Haslinger R,Klinkner KL,Shalizi CR

    更新日期：2010-01-01 00:00:00

  • Random embedding machines for pattern recognition.

    abstract：:Real classification problems involve structured data that can be essentially grouped into a relatively small number of clusters. It is shown that, under a local clustering condition, a set of points of a given class, embedded in binary space by a set of randomly parameterized surfaces, is linearly separable from other...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/089976601753196012

    authors： Baram Y

    更新日期：2001-11-01 00:00:00

  • Robust boosting algorithm against mislabeling in multiclass problems.

    abstract：:We discuss robustness against mislabeling in multiclass labels for classification problems and propose two algorithms of boosting, the normalized Eta-Boost.M and Eta-Boost.M, based on the Eta-divergence. Those two boosting algorithms are closely related to models of mislabeling in which the label is erroneously exchan...

    journal_title：Neural computation

    pub_type： 信件

    doi：10.1162/neco.2007.11-06-400

    authors： Takenouchi T,Eguchi S,Murata N,Kanamori T

    更新日期：2008-06-01 00:00:00

  • A neural-network-based approach to the double traveling salesman problem.

    abstract：:The double traveling salesman problem is a variation of the basic traveling salesman problem where targets can be reached by two salespersons operating in parallel. The real problem addressed by this work concerns the optimization of the harvest sequence for the two independent arms of a fruit-harvesting robot. This a...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/08997660252741194

    authors： Plebe A,Anile AM

    更新日期：2002-02-01 00:00:00

  • Deficient GABAergic gliotransmission may cause broader sensory tuning in schizophrenia.

    abstract：:We examined how the depression of intracortical inhibition due to a reduction in ambient GABA concentration impairs perceptual information processing in schizophrenia. A neural network model with a gliotransmission-mediated ambient GABA regulatory mechanism was simulated. In the network, interneuron-to-glial-cell and ...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/NECO_a_00519

    authors： Hoshino O

    更新日期：2013-12-01 00:00:00

  • Convergence of the IRWLS Procedure to the Support Vector Machine Solution.

    abstract：:An iterative reweighted least squares (IRWLS) procedure recently proposed is shown to converge to the support vector machine solution. The convergence to a stationary point is ensured by modifying the original IRWLS procedure. ...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/0899766052530875

    authors： Pérez-Cruz F,Bousoño-Calzón C,Artés-Rodríguez A

    更新日期：2005-01-01 00:00:00

  • Correlational Neural Networks.

    abstract：:Common representation learning (CRL), wherein different descriptions (or views) of the data are embedded in a common subspace, has been receiving a lot of attention recently. Two popular paradigms here are canonical correlation analysis (CCA)-based approaches and autoencoder (AE)-based approaches. CCA-based approaches...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/NECO_a_00801

    authors： Chandar S,Khapra MM,Larochelle H,Ravindran B

    更新日期：2016-02-01 00:00:00

  • Active Learning for Enumerating Local Minima Based on Gaussian Process Derivatives.

    abstract：:We study active learning (AL) based on gaussian processes (GPs) for efficiently enumerating all of the local minimum solutions of a black-box function. This problem is challenging because local solutions are characterized by their zero gradient and positive-definite Hessian properties, but those derivatives cannot be ...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco_a_01307

    authors： Inatsu Y,Sugita D,Toyoura K,Takeuchi I

    更新日期：2020-10-01 00:00:00

  • Dynamics of learning near singularities in layered networks.

    abstract：:We explicitly analyze the trajectories of learning near singularities in hierarchical networks, such as multilayer perceptrons and radial basis function networks, which include permutation symmetry of hidden nodes, and show their general properties. Such symmetry induces singularities in their parameter space, where t...

    journal_title：Neural computation

    pub_type： 信件

    doi：10.1162/neco.2007.12-06-414

    authors： Wei H,Zhang J,Cousseau F,Ozeki T,Amari S

    更新日期：2008-03-01 00:00:00

  • Rapid processing and unsupervised learning in a model of the cortical macrocolumn.

    abstract：:We study a model of the cortical macrocolumn consisting of a collection of inhibitorily coupled minicolumns. The proposed system overcomes several severe deficits of systems based on single neurons as cerebral functional units, notably limited robustness to damage and unrealistically large computation time. Motivated ...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/089976604772744893

    authors： Lücke J,von der Malsburg C

    更新日期：2004-03-01 00:00:00

  • A bio-inspired, computational model suggests velocity gradients of optic flow locally encode ordinal depth at surface borders and globally they encode self-motion.

    abstract：:Visual navigation requires the estimation of self-motion as well as the segmentation of objects from the background. We suggest a definition of local velocity gradients to compute types of self-motion, segment objects, and compute local properties of optical flow fields, such as divergence, curl, and shear. Such veloc...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/NECO_a_00479

    authors： Raudies F,Ringbauer S,Neumann H

    更新日期：2013-09-01 00:00:00

  • Investigating the fault tolerance of neural networks.

    abstract：:Particular levels of partial fault tolerance (PFT) in feedforward artificial neural networks of a given size can be obtained by redundancy (replicating a smaller normally trained network), by design (training specifically to increase PFT), and by a combination of the two (replicating a smaller PFT-trained network). Th...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/0899766053723096

    authors： Tchernev EB,Mulvaney RG,Phatak DS

    更新日期：2005-07-01 00:00:00

  • Asynchronous Event-Based Motion Processing: From Visual Events to Probabilistic Sensory Representation.

    abstract：:In this work, we propose a two-layered descriptive model for motion processing from retina to the cortex, with an event-based input from the asynchronous time-based image sensor (ATIS) camera. Spatial and spatiotemporal filtering of visual scenes by motion energy detectors has been implemented in two steps in a simple...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco_a_01191

    authors： Khoei MA,Ieng SH,Benosman R

    更新日期：2019-06-01 00:00:00

  • Local and global gating of synaptic plasticity.

    abstract：:Mechanisms influencing learning in neural networks are usually investigated on either a local or a global scale. The former relates to synaptic processes, the latter to unspecific modulatory systems. Here we study the interaction of a local learning rule that evaluates coincidences of pre- and postsynaptic action pote...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/089976600300015682

    authors： Sánchez-Montañés MA,Verschure PF,König P

    更新日期：2000-03-01 00:00:00

  • Positive Neural Networks in Discrete Time Implement Monotone-Regular Behaviors.

    abstract：:We study the expressive power of positive neural networks. The model uses positive connection weights and multiple input neurons. Different behaviors can be expressed by varying the connection weights. We show that in discrete time and in the absence of noise, the class of positive neural networks captures the so-call...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/NECO_a_00789

    authors： Ameloot TJ,Van den Bussche J

    更新日期：2015-12-01 00:00:00

  • Conductance-based integrate-and-fire models.

    abstract：:A conductance-based model of Na+ and K+ currents underlying action potential generation is introduced by simplifying the quantitative model of Hodgkin and Huxley (HH). If the time course of rate constants can be approximated by a pulse, HH equations can be solved analytically. Pulse-based (PB) models generate action p...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco.1997.9.3.503

    authors： Destexhe A

    更新日期：1997-04-01 00:00:00

  • Statistical procedures for spatiotemporal neuronal data with applications to optical recording of the auditory cortex.

    abstract：:This article presents new procedures for multisite spatiotemporal neuronal data analysis. A new statistical model - the diffusion model - is considered, whose parameters can be estimated from experimental data thanks to mean-field approximations. This work has been applied to optical recording of the guinea pig's audi...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/089976600300015150

    authors： François O,Abdallahi LM,Horikawa J,Taniguchi I,Hervé T

    更新日期：2000-08-01 00:00:00

  • A Gaussian attractor network for memory and recognition with experience-dependent learning.

    abstract：:Attractor networks are widely believed to underlie the memory systems of animals across different species. Existing models have succeeded in qualitatively modeling properties of attractor dynamics, but their computational abilities often suffer from poor representations for realistic complex patterns, spurious attract...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco.2010.02-09-957

    authors： Hu X,Zhang B

    更新日期：2010-05-01 00:00:00

  • Changes in GABAB modulation during a theta cycle may be analogous to the fall of temperature during annealing.

    abstract：:Changes in GABA modulation may underlie experimentally observed changes in the strength of synaptic transmission at different phases of the theta rhythm (Wyble, Linster, & Hasselmo, 1997). Analysis demonstrates that these changes improve sequence disambiguation by a neural network model of CA3. We show that in the fra...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/089976698300017539

    authors： Sohal VS,Hasselmo ME

    更新日期：1998-05-15 00:00:00

  • Direct estimation of inhomogeneous Markov interval models of spike trains.

    abstract：:A necessary ingredient for a quantitative theory of neural coding is appropriate "spike kinematics": a precise description of spike trains. While summarizing experiments by complete spike time collections is clearly inefficient and probably unnecessary, the most common probabilistic model used in neurophysiology, the ...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco.2009.07-08-828

    authors： Wójcik DK,Mochol G,Jakuczun W,Wypych M,Waleszczyk WJ

    更新日期：2009-08-01 00:00:00

  • Supervised Determined Source Separation with Multichannel Variational Autoencoder.

    abstract：:This letter proposes a multichannel source separation technique, the multichannel variational autoencoder (MVAE) method, which uses a conditional VAE (CVAE) to model and estimate the power spectrograms of the sources in a mixture. By training the CVAE using the spectrograms of training examples with source-class label...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco_a_01217

    authors： Kameoka H,Li L,Inoue S,Makino S

    更新日期：2019-09-01 00:00:00

  • Piecewise-linear neural networks and their relationship to rule extraction from data.

    abstract：:This article addresses the topic of extracting logical rules from data by means of artificial neural networks. The approach based on piecewise linear neural networks is revisited, which has already been used for the extraction of Boolean rules in the past, and it is shown that this approach can be important also for t...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco.2006.18.11.2813

    authors： Holena M

    更新日期：2006-11-01 00:00:00

  • Density-weighted Nyström method for computing large kernel eigensystems.

    abstract：:The Nyström method is a well-known sampling-based technique for approximating the eigensystem of large kernel matrices. However, the chosen samples in the Nyström method are all assumed to be of equal importance, which deviates from the integral equation that defines the kernel eigenfunctions. Motivated by this observ...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco.2008.11-07-651

    authors： Zhang K,Kwok JT

    更新日期：2009-01-01 00:00:00

  • Modeling short-term synaptic depression in silicon.

    abstract：:We describe a model of short-term synaptic depression that is derived from a circuit implementation. The dynamics of this circuit model is similar to the dynamics of some theoretical models of short-term depression except that the recovery dynamics of the variable describing the depression is nonlinear and it also dep...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/089976603762552942

    authors： Boegerhausen M,Suter P,Liu SC

    更新日期：2003-02-01 00:00:00

  • On the performance of voltage stepping for the simulation of adaptive, nonlinear integrate-and-fire neuronal networks.

    abstract：:In traditional event-driven strategies, spike timings are analytically given or calculated with arbitrary precision (up to machine precision). Exact computation is possible only for simplified neuron models, mainly the leaky integrate-and-fire model. In a recent paper, Zheng, Tonnelier, and Martinez (2009) introduced ...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/NECO_a_00112

    authors： Kaabi MG,Tonnelier A,Martinez D

    更新日期：2011-05-01 00:00:00

  • On the slow convergence of EM and VBEM in low-noise linear models.

    abstract：:We analyze convergence of the expectation maximization (EM) and variational Bayes EM (VBEM) schemes for parameter estimation in noisy linear models. The analysis shows that both schemes are inefficient in the low-noise limit. The linear model with additive noise includes as special cases independent component analysis...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/0899766054322991

    authors： Petersen KB,Winther O,Hansen LK

    更新日期：2005-09-01 00:00:00

  • Hidden Quantum Processes, Quantum Ion Channels, and 1/ fθ-Type Noise.

    abstract：:In this letter, we perform a complete and in-depth analysis of Lorentzian noises, such as those arising from [Formula: see text] and [Formula: see text] channel kinetics, in order to identify the source of [Formula: see text]-type noise in neurological membranes. We prove that the autocovariance of Lorentzian noise de...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/NECO_a_01067

    authors： Paris A,Vosoughi A,Berman SA,Atia G

    更新日期：2018-07-01 00:00:00

  • Statistics of Visual Responses to Image Object Stimuli from Primate AIT Neurons to DNN Neurons.

    abstract：:Under the goal-driven paradigm, Yamins et al. ( 2014 ; Yamins & DiCarlo, 2016 ) have shown that by optimizing only the final eight-way categorization performance of a four-layer hierarchical network, not only can its top output layer quantitatively predict IT neuron responses but its penultimate layer can also automat...

    journal_title：Neural computation

    pub_type： 杂志文章

    doi：10.1162/neco_a_01039

    authors： Dong Q,Wang H,Hu Z

    更新日期：2018-02-01 00:00:00