Why Does Large Batch Training Result in Poor Generalization? A Comprehensive Explanation and a Better Strategy from the Viewpoint of Stochastic Optimization.

abstract：:Recent experimental and computational evidence suggests that several dynamical properties may characterize the operating point of functioning neural networks: critical branching, neutral stability, and production of a wide range of firing patterns. We seek the simplest setting in which these properties emerge, clarify...

journal_title：Neural computation

authors： Cayco-Gajic NA,Shea-Brown E

更新日期：2013-07-01 00:00:00

abstract：:We study active learning (AL) based on gaussian processes (GPs) for efficiently enumerating all of the local minimum solutions of a black-box function. This problem is challenging because local solutions are characterized by their zero gradient and positive-definite Hessian properties, but those derivatives cannot be ...

journal_title：Neural computation

authors： Inatsu Y,Sugita D,Toyoura K,Takeuchi I

更新日期：2020-10-01 00:00:00

abstract：:Inner-product operators, often referred to as kernels in statistical learning, define a mapping from some input space into a feature space. The focus of this letter is the construction of biologically motivated kernels for cortical activities. The kernels we derive, termed Spikernels, map spike count sequences into an...

journal_title：Neural computation

authors： Shpigelman L,Singer Y,Paz R,Vaadia E

更新日期：2005-03-01 00:00:00

abstract：:Much experimental evidence suggests that during decision making, neural circuits accumulate evidence supporting alternative options. A computational model well describing this accumulation for choices between two options assumes that the brain integrates the log ratios of the likelihoods of the sensory inputs given th...

journal_title：Neural computation

authors： Kurzawa N,Summerfield C,Bogacz R

更新日期：2017-02-01 00:00:00

abstract：:Neuroscience is progressing vigorously, and knowledge at different levels of description is rapidly accumulating. To establish relationships between results found at these different levels is one of the central challenges. In this simulation study, we demonstrate how microscopic cellular properties, taking the example...

journal_title：Neural computation

authors： Verschure PF,König P

更新日期：1999-07-01 00:00:00

abstract：:In considering a statistical model selection of neural networks and radial basis functions under an overrealizable case, the problem of unidentifiability emerges. Because the model selection criterion is an unbiased estimator of the generalization error based on the training error, this article analyzes the expected t...

journal_title：Neural computation

authors： Hagiwara K

更新日期：2002-08-01 00:00:00

abstract：:A necessary ingredient for a quantitative theory of neural coding is appropriate "spike kinematics": a precise description of spike trains. While summarizing experiments by complete spike time collections is clearly inefficient and probably unnecessary, the most common probabilistic model used in neurophysiology, the ...

journal_title：Neural computation

authors： Wójcik DK,Mochol G,Jakuczun W,Wypych M,Waleszczyk WJ

更新日期：2009-08-01 00:00:00

abstract：:A key problem in computational neuroscience is to find simple, tractable models that are nevertheless flexible enough to capture the response properties of real neurons. Here we examine the capabilities of recurrent point process models known as Poisson generalized linear models (GLMs). These models are defined by a s...

journal_title：Neural computation

authors： Weber AI,Pillow JW

更新日期：2017-12-01 00:00:00

abstract：:Decision making is a complex task, and its underlying mechanisms that regulate behavior, such as the implementation of the coupling between physiological states and neural networks, are hard to decipher. To gain more insight into neural computations underlying ongoing binary decision-making tasks, we consider a neural...

journal_title：Neural computation

authors： Bose T,Reina A,Marshall JAR

更新日期：2019-05-01 00:00:00

abstract：:In this letter, we develop a gaussian process model for clustering. The variances of predictive values in gaussian processes learned from a training data are shown to comprise an estimate of the support of a probability density function. The constructed variance function is then applied to construct a set of contours ...

journal_title：Neural computation

authors： Kim HC,Lee J

更新日期：2007-11-01 00:00:00

abstract：:The brain is known to be active even when not performing any overt cognitive tasks, and often it engages in involuntary mind wandering. This resting state has been extensively characterized in terms of fMRI-derived brain networks. However, an alternate method has recently gained popularity: EEG microstate analysis. Pr...

journal_title：Neural computation

authors： Shaw SB,Dhindsa K,Reilly JP,Becker S

更新日期：2019-11-01 00:00:00

abstract：:The goal of sufficient dimension reduction in supervised learning is to find the low-dimensional subspace of input features that contains all of the information about the output values that the input features possess. In this letter, we propose a novel sufficient dimension-reduction method using a squared-loss variant...

journal_title：Neural computation

authors： Suzuki T,Sugiyama M

更新日期：2013-03-01 00:00:00

abstract：:We investigate the approximation ability of a multilayer perceptron (MLP) network when it is extended to the complex domain. The main challenge for processing complex data with neural networks has been the lack of bounded and analytic complex nonlinear activation functions in the complex domain, as stated by Liouville...

journal_title：Neural computation

authors： Kim T,Adali T

更新日期：2003-07-01 00:00:00

abstract：:Recurrent neural architectures having oscillatory dynamics use rhythmic network activity to represent patterns stored in short-term memory. Multiple stored patterns can be retained in memory over the same neural substrate because the network's state persistently switches between them. Here we present a simple oscillat...

journal_title：Neural computation

authors： Winder RK,Reggia JA,Weems SA,Bunting MF

更新日期：2009-03-01 00:00:00

abstract：:When we move our body to perform a movement task, our central nervous system selects a movement trajectory from an infinite number of possible trajectories under constraints that have been acquired through evolution and learning. Minimization of the energy cost has been suggested as a potential candidate for a constra...

journal_title：Neural computation

authors： Taniai Y,Nishii J

更新日期：2015-08-01 00:00:00

abstract：:The successor representation was introduced into reinforcement learning by Dayan ( 1993 ) as a means of facilitating generalization between states with similar successors. Although reinforcement learning in general has been used extensively as a model of psychological and neural processes, the psychological validity o...

journal_title：Neural computation

authors： Gershman SJ,Moore CD,Todd MT,Norman KA,Sederberg PB

更新日期：2012-06-01 00:00:00

abstract：:We study a model of the cortical macrocolumn consisting of a collection of inhibitorily coupled minicolumns. The proposed system overcomes several severe deficits of systems based on single neurons as cerebral functional units, notably limited robustness to damage and unrealistically large computation time. Motivated ...

journal_title：Neural computation

authors： Lücke J,von der Malsburg C

更新日期：2004-03-01 00:00:00

abstract：:Visual navigation requires the estimation of self-motion as well as the segmentation of objects from the background. We suggest a definition of local velocity gradients to compute types of self-motion, segment objects, and compute local properties of optical flow fields, such as divergence, curl, and shear. Such veloc...

journal_title：Neural computation

authors： Raudies F,Ringbauer S,Neumann H

更新日期：2013-09-01 00:00:00

abstract：:We generalize recent theoretical work on the minimal number of layers of narrow deep belief networks that can approximate any probability distribution on the states of their visible units arbitrarily well. We relax the setting of binary units (Sutskever & Hinton, 2008 ; Le Roux & Bengio, 2008 , 2010 ; Montúfar & Ay, 2...

journal_title：Neural computation

authors： Montúfar GF

更新日期：2014-07-01 00:00:00

abstract：:The bias/variance decomposition of mean-squared error is well understood and relatively straightforward. In this note, a similar simple decomposition is derived, valid for any kind of error measure that, when using the appropriate probability model, can be derived from a Kullback-Leibler divergence or log-likelihood. ...

journal_title：Neural computation

authors： Heskes T

更新日期：1998-07-28 00:00:00

abstract：:We propose a scalable semiparametric Bayesian model to capture dependencies among multiple neurons by detecting their cofiring (possibly with some lag time) patterns over time. After discretizing time so there is at most one spike at each interval, the resulting sequence of 1s (spike) and 0s (silence) for each neuron ...

journal_title：Neural computation

authors： Shahbaba B,Zhou B,Lan S,Ombao H,Moorman D,Behseta S

更新日期：2014-09-01 00:00:00

abstract：:Pairwise correlations among spike trains recorded in vivo have been frequently reported. It has been argued that correlated activity could play an important role in the brain, because it efficiently modulates the response of a postsynaptic neuron. We show here that a neuron's output firing rate critically depends on t...

journal_title：Neural computation

authors： Kuhn A,Aertsen A,Rotter S

更新日期：2003-01-01 00:00:00

abstract：:The relationship between a neuron's complex inputs and its spiking output defines the neuron's coding strategy. This is frequently and effectively modeled phenomenologically by one or more linear filters that extract the components of the stimulus that are relevant for triggering spikes and a nonlinear function that r...

journal_title：Neural computation

authors： Famulare M,Fairhall A

更新日期：2010-03-01 00:00:00

abstract：:This article presents a reinforcement learning framework for continuous-time dynamical systems without a priori discretization of time, state, and action. Based on the Hamilton-Jacobi-Bellman (HJB) equation for infinite-horizon, discounted reward problems, we derive algorithms for estimating value functions and improv...

journal_title：Neural computation

authors： Doya K

更新日期：2000-01-01 00:00:00

abstract：:The Kalman filter provides a simple and efficient algorithm to compute the posterior distribution for state-space models where both the latent state and measurement models are linear and gaussian. Extensions to the Kalman filter, including the extended and unscented Kalman filters, incorporate linearizations for model...

journal_title：Neural computation

authors： Burkhart MC,Brandman DM,Franco B,Hochberg LR,Harrison MT

更新日期：2020-05-01 00:00:00

abstract：:We show how a Hopfield network with modifiable recurrent connections undergoing slow Hebbian learning can extract the underlying geometry of an input space. First, we use a slow and fast analysis to derive an averaged system whose dynamics derives from an energy function and therefore always converges to equilibrium p...

journal_title：Neural computation

authors： Galtier MN,Faugeras OD,Bressloff PC

更新日期：2012-09-01 00:00:00

abstract：:Changes in GABA modulation may underlie experimentally observed changes in the strength of synaptic transmission at different phases of the theta rhythm (Wyble, Linster, & Hasselmo, 1997). Analysis demonstrates that these changes improve sequence disambiguation by a neural network model of CA3. We show that in the fra...

journal_title：Neural computation

authors： Sohal VS,Hasselmo ME

更新日期：1998-05-15 00:00:00

abstract：:In the past decade the importance of synchronized dynamics in the brain has emerged from both empirical and theoretical perspectives. Fast dynamic synchronous interactions of an oscillatory or nonoscillatory nature may constitute a form of temporal coding that underlies feature binding and perceptual synthesis. The re...

journal_title：Neural computation

authors： Chawla D,Lumer ED,Friston KJ

更新日期：1999-08-15 00:00:00

abstract：:We argue that when faced with big data sets, learning and inference algorithms should compute updates using only subsets of data items. We introduce algorithms that use sequential hypothesis tests to adaptively select such a subset of data points. The statistical properties of this subsampling process can be used to c...

journal_title：Neural computation

authors： Korattikara A,Chen Y,Welling M

更新日期：2016-01-01 00:00:00

abstract：:Independent component analysis (ICA) finds a linear transformation to variables that are maximally statistically independent. We examine ICA and algorithms for finding the best transformation from the point of view of maximizing the likelihood of the data. In particular, we discuss the way in which scaling of the unmi...

journal_title：Neural computation

authors： Everson R,Roberts S

更新日期：1999-11-15 00:00:00