Scalable training of artificial neural networks with adaptive sparse connectivity inspired by network science.

Abstract:

:Through the success of deep learning in various domains, artificial neural networks are currently among the most used artificial intelligence methods. Taking inspiration from the network properties of biological neural networks (e.g. sparsity, scale-freeness), we argue that (contrary to general practice) artificial neural networks, too, should not have fully-connected layers. Here we propose sparse evolutionary training of artificial neural networks, an algorithm which evolves an initial sparse topology (Erdős-Rényi random graph) of two consecutive layers of neurons into a scale-free topology, during learning. Our method replaces artificial neural networks fully-connected layers with sparse ones before training, reducing quadratically the number of parameters, with no decrease in accuracy. We demonstrate our claims on restricted Boltzmann machines, multi-layer perceptrons, and convolutional neural networks for unsupervised and supervised learning on 15 datasets. Our approach has the potential to enable artificial neural networks to scale up beyond what is currently possible.

journal_name

Nat Commun

journal_title

Nature communications

authors

Mocanu DC,Mocanu E,Stone P,Nguyen PH,Gibescu M,Liotta A

doi

10.1038/s41467-018-04316-3

subject

Has Abstract

pub_date

2018-06-19 00:00:00

pages

2383

issue

1

issn

2041-1723

pii

10.1038/s41467-018-04316-3

journal_volume

9

pub_type

杂志文章
  • Reliable detection of subclonal single-nucleotide variants in tumour cell populations.

    abstract::According to the clonal evolution model, tumour growth is driven by competing subclones in somatically evolving cancer cell populations, which gives rise to genetically heterogeneous tumours. Here we present a comparative targeted deep-sequencing approach combined with a customised statistical algorithm, called deepSN...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms1814

    authors: Gerstung M,Beisel C,Rechsteiner M,Wild P,Schraml P,Moch H,Beerenwinkel N

    更新日期:2012-05-01 00:00:00

  • Direct electrical quantification of glucose and asparagine from bodily fluids using nanopores.

    abstract::Crucial steps in the miniaturisation of biosensors are the conversion of a biological signal into an electrical current as well as the direct sampling of bodily fluids. Here we show that protein sensors in combination with a nanopore, acting as an electrical transducer, can accurately quantify metabolites in real time...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-018-06534-1

    authors: Galenkamp NS,Soskine M,Hermans J,Wloka C,Maglia G

    更新日期:2018-10-05 00:00:00

  • Skin-touch-actuated textile-based triboelectric nanogenerator with black phosphorus for durable biomechanical energy harvesting.

    abstract::Textiles that are capable of harvesting biomechanical energy via triboelectric effects are of interest for self-powered wearable electronics. Fabrication of conformable and durable textiles with high triboelectric outputs remains challenging. Here we propose a washable skin-touch-actuated textile-based triboelectric n...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-018-06759-0

    authors: Xiong J,Cui P,Chen X,Wang J,Parida K,Lin MF,Lee PS

    更新日期:2018-10-15 00:00:00

  • Electron-vibration coupling induced renormalization in the photoemission spectrum of diamondoids.

    abstract::The development of theories and methods devoted to the accurate calculation of the electronic quasi-particle states and levels of molecules, clusters and solids is of prime importance to interpret the experimental data. These quantum systems are often modelled by using the Born-Oppenheimer approximation where the coup...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms11327

    authors: Gali A,Demján T,Vörös M,Thiering G,Cannuccia E,Marini A

    更新日期:2016-04-22 00:00:00

  • Self-similarity of contact line depinning from textured surfaces.

    abstract::The mobility of drops on surfaces is important in many biological and industrial processes, but the phenomena governing their adhesion, which is dictated by the morphology of the three-phase contact line, remain unclear. Here we describe a technique for measuring the dynamic behaviour of the three-phase contact line a...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms2482

    authors: Paxson AT,Varanasi KK

    更新日期:2013-01-01 00:00:00

  • Genome-scale proteome quantification by DEEP SEQ mass spectrometry.

    abstract::Advances in chemistry and massively parallel detection underlie DNA-sequencing platforms that are poised for application in personalized medicine. In stark contrast, systematic generation of protein-level data lags well behind genomics in virtually every aspect: depth of coverage, throughput, ease of sample preparatio...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms3171

    authors: Zhou F,Lu Y,Ficarro SB,Adelmant G,Jiang W,Luckey CJ,Marto JA

    更新日期:2013-01-01 00:00:00

  • Tuneable reflexes control antennal positioning in flying hawkmoths.

    abstract::Complex behaviours may be viewed as sequences of modular actions, each elicited by specific sensory cues in their characteristic timescales. From this perspective, we can construct models in which unitary behavioural modules are hierarchically placed in context of related actions. Here, we analyse antennal positioning...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-019-13595-3

    authors: Natesan D,Saxena N,Ekeberg Ö,Sane SP

    更新日期:2019-12-06 00:00:00

  • Plasmodium falciparum heat shock protein 110 stabilizes the asparagine repeat-rich parasite proteome during malarial fevers.

    abstract::One-fourth of Plasmodium falciparum proteins have asparagine repeats that increase the propensity for aggregation, especially at elevated temperatures that occur routinely in malaria-infected patients. Here we report that a Plasmodium Asn repeat-containing protein (PFI1155w) formed aggregates in mammalian cells at feb...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms2306

    authors: Muralidharan V,Oksman A,Pal P,Lindquist S,Goldberg DE

    更新日期:2012-01-01 00:00:00

  • An experimental test on the probability of extinction of new genetic variants.

    abstract::In 1927, J.B.S. Haldane reasoned that the probability of fixation of new beneficial alleles is twice their fitness effect. This result, later generalized by M. Kimura, has since become the cornerstone of modern population genetics. There is no experimental test of Haldane's insight that new beneficial alleles are lost...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms3417

    authors: Chelo IM,Nédli J,Gordo I,Teotónio H

    更新日期:2013-01-01 00:00:00

  • Genome-wide association study identifies multiple susceptibility loci for glioma.

    abstract::Previous genome-wide association studies (GWASs) have shown that common genetic variation contributes to the heritable risk of glioma. To identify new glioma susceptibility loci, we conducted a meta-analysis of four GWAS (totalling 4,147 cases and 7,435 controls), with imputation using 1000 Genomes and UK10K Project d...

    journal_title:Nature communications

    pub_type: 杂志文章,meta分析

    doi:10.1038/ncomms9559

    authors: Kinnersley B,Labussière M,Holroyd A,Di Stefano AL,Broderick P,Vijayakrishnan J,Mokhtari K,Delattre JY,Gousias K,Schramm J,Schoemaker MJ,Fleming SJ,Herms S,Heilmann S,Schreiber S,Wichmann HE,Nöthen MM,Swerdlow A,Lathro

    更新日期:2015-10-01 00:00:00

  • Synthetic strategies of supported atomic clusters for heterogeneous catalysis.

    abstract::Supported atomic clusters with uniform metal sites and definite low-nuclearity are intermediate states between single-atom catalysts (SACs) and nanoparticles in size. Benefiting from the presence of metal-metal bonds, supported atomic clusters can trigger synergistic effects among every metal atom, which contributes t...

    journal_title:Nature communications

    pub_type: 杂志文章,评审

    doi:10.1038/s41467-020-19571-6

    authors: Rong H,Ji S,Zhang J,Wang D,Li Y

    更新日期:2020-11-18 00:00:00

  • Key drivers of cloud response to surface-active organics.

    abstract::Aerosol-cloud interactions constitute the largest source of uncertainty in global radiative forcing estimates, hampering our understanding of climate evolution. Recent empirical evidence suggests surface tension depression by organic aerosol to significantly influence the formation of cloud droplets, and hence cloud o...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-019-12982-0

    authors: Lowe SJ,Partridge DG,Davies JF,Wilson KR,Topping D,Riipinen I

    更新日期:2019-11-18 00:00:00

  • A generalized non-local optical response theory for plasmonic nanostructures.

    abstract::Metallic nanostructures exhibit a multitude of optical resonances associated with localized surface plasmon excitations. Recent observations of plasmonic phenomena at the sub-nanometre to atomic scale have stimulated the development of various sophisticated theoretical approaches for their description. Here instead we...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms4809

    authors: Mortensen NA,Raza S,Wubs M,Søndergaard T,Bozhevolnyi SI

    更新日期:2014-05-02 00:00:00

  • CMOS-based carbon nanotube pass-transistor logic integrated circuits.

    abstract::Field-effect transistors based on carbon nanotubes have been shown to be faster and less energy consuming than their silicon counterparts. However, ensuring these advantages are maintained for integrated circuits is a challenge. Here we demonstrate that a significant reduction in the use of field-effect transistors ca...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms1682

    authors: Ding L,Zhang Z,Liang S,Pei T,Wang S,Li Y,Zhou W,Liu J,Peng LM

    更新日期:2012-02-14 00:00:00

  • Clonal selection in the human Vδ1 T cell repertoire indicates γδ TCR-dependent adaptive immune surveillance.

    abstract::γδ T cells are considered to be innate-like lymphocytes that respond rapidly to stress without clonal selection and differentiation. Here we use next-generation sequencing to probe how this paradigm relates to human Vδ2neg T cells, implicated in responses to viral infection and cancer. The prevalent Vδ1 T cell recepto...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms14760

    authors: Davey MS,Willcox CR,Joyce SP,Ladell K,Kasatskaya SA,McLaren JE,Hunter S,Salim M,Mohammed F,Price DA,Chudakov DM,Willcox BE

    更新日期:2017-03-01 00:00:00

  • Klf5 acetylation regulates luminal differentiation of basal progenitors in prostate development and regeneration.

    abstract::Prostate development depends on balanced cell proliferation and differentiation, and acetylated KLF5 is known to alter epithelial proliferation. It remains elusive whether post-translational modifications of transcription factors can differentially determine adult stem/progenitor cell fate. Here we report that, in hum...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-020-14737-8

    authors: Zhang B,Ci X,Tao R,Ni JJ,Xuan X,King JL,Xia S,Li Y,Frierson HF,Lee DK,Xu J,Osunkoya AO,Dong JT

    更新日期:2020-02-21 00:00:00

  • How ligand binds to the type 1 insulin-like growth factor receptor.

    abstract::Human type 1 insulin-like growth factor receptor is a homodimeric receptor tyrosine kinase that signals into pathways directing normal cellular growth, differentiation and proliferation, with aberrant signalling implicated in cancer. Insulin-like growth factor binding is understood to relax conformational restraints w...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-018-03219-7

    authors: Xu Y,Kong GK,Menting JG,Margetts MB,Delaine CA,Jenkin LM,Kiselyov VV,De Meyts P,Forbes BE,Lawrence MC

    更新日期:2018-02-26 00:00:00

  • Combined activation of MAP kinase pathway and β-catenin signaling cause deep penetrating nevi.

    abstract::Deep penetrating nevus (DPN) is characterized by enlarged, pigmented melanocytes that extend through the dermis. DPN can be difficult to distinguish from melanoma but rarely displays aggressive biological behavior. Here, we identify a combination of mutations of the β-catenin and mitogen-activated protein kinase pathw...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-017-00758-3

    authors: Yeh I,Lang UE,Durieux E,Tee MK,Jorapur A,Shain AH,Haddad V,Pissaloux D,Chen X,Cerroni L,Judson RL,LeBoit PE,McCalmont TH,Bastian BC,de la Fouchardière A

    更新日期:2017-09-21 00:00:00

  • Self-assembled nanoscale coordination polymers with trigger release properties for effective anticancer therapy.

    abstract::Nanoscale coordination polymers (NCPs) are self-assembled from metal ions and organic bridging ligands, and can overcome many drawbacks of existing drug delivery systems by virtue of tunable compositions, sizes and shapes, high drug loadings, ease of surface modification and intrinsic biodegradability. Here we report ...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms5182

    authors: Liu D,Poon C,Lu K,He C,Lin W

    更新日期:2014-06-25 00:00:00

  • Structure of a microtubule-bound axonemal dynein.

    abstract::Axonemal dyneins are tethered to doublet microtubules inside cilia to drive ciliary beating, a process critical for cellular motility and extracellular fluid flow. Axonemal dyneins are evolutionarily and biochemically distinct from cytoplasmic dyneins that transport cargo, and the mechanisms regulating their localizat...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-020-20735-7

    authors: Walton T,Wu H,Brown A

    更新日期:2021-01-20 00:00:00

  • Ultra-narrow metallic armchair graphene nanoribbons.

    abstract::Graphene nanoribbons (GNRs)-narrow stripes of graphene-have emerged as promising building blocks for nanoelectronic devices. Recent advances in bottom-up synthesis have allowed production of atomically well-defined armchair GNRs with different widths and doping. While all experimentally studied GNRs have exhibited wid...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms10177

    authors: Kimouche A,Ervasti MM,Drost R,Halonen S,Harju A,Joensuu PM,Sainio J,Liljeroth P

    更新日期:2015-12-14 00:00:00

  • Two-dimensional molecular brush-functionalized porous bilayer composite separators toward ultrastable high-current density lithium metal anodes.

    abstract::Lithium metal batteries have been considerably limited by the problems of uncontrolled dendritic lithium formation and the highly reactive nature of lithium with electrolytes. Herein, we have developed functional porous bilayer composite separators by simply blade-coating polyacrylamide-grafted graphene oxide molecula...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-019-09211-z

    authors: Li C,Liu S,Shi C,Liang G,Lu Z,Fu R,Wu D

    更新日期:2019-03-25 00:00:00

  • Cascading ecological effects of eliminating fishery discards.

    abstract::Discarding by fisheries is perceived as contrary to responsible harvesting. Legislation seeking to end the practice is being introduced in many jurisdictions. However, discarded fish are food for a range of scavenging species; so, ending discarding may have ecological consequences. Here we investigate the sensitivity ...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms4893

    authors: Heath MR,Cook RM,Cameron AI,Morris DJ,Speirs DC

    更新日期:2014-05-13 00:00:00

  • Probing nanoscale spatial distribution of plasmonically excited hot carriers.

    abstract::Surface plasmons (SPs) of metals enable the tight focusing and strong absorption of light to realize an efficient utilization of photons at nanoscale. In particular, the SP-generated hot carriers have emerged as a promising way to efficiently drive photochemical and photoelectric processes under moderate conditions. I...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-020-18016-4

    authors: Huang SC,Wang X,Zhao QQ,Zhu JF,Li CW,He YH,Hu S,Sartin MM,Yan S,Ren B

    更新日期:2020-08-24 00:00:00

  • Non-equilibrium 8π Josephson effect in atomic Kitaev wires.

    abstract::The identification of fractionalized excitations, such as Majorana quasi-particles, would be a striking signal of the realization of exotic quantum states of matter. While the paramount demonstration of such excitations would be a probe of their non-Abelian statistics via controlled braiding operations, alternative pr...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms12280

    authors: Laflamme C,Budich JC,Zoller P,Dalmonte M

    更新日期:2016-08-02 00:00:00

  • Stiff filamentous virus translocations through solid-state nanopores.

    abstract::The ionic conductance through a nanometer-sized pore in a membrane changes when a biopolymer slides through it, making nanopores sensitive to single molecules in solution. Their possible use for sequencing has motivated numerous studies on how DNA, a semi-flexible polymer, translocates nanopores. Here we study voltage...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms5171

    authors: McMullen A,de Haan HW,Tang JX,Stein D

    更新日期:2014-06-16 00:00:00

  • Dynamic CpG methylation delineates subregions within super-enhancers selectively decommissioned at the exit from naive pluripotency.

    abstract::Clusters of enhancers, referred as to super-enhancers (SEs), control the expression of cell identity genes. The organisation of these clusters, and how they are remodelled upon developmental transitions remain poorly understood. Here, we report the existence of two types of enhancer units within SEs typified by distin...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-020-14916-7

    authors: Bell E,Curry EW,Megchelenbrink W,Jouneau L,Brochard V,Tomaz RA,Mau KHT,Atlasi Y,de Souza RA,Marks H,Stunnenberg HG,Jouneau A,Azuara V

    更新日期:2020-02-28 00:00:00

  • Nonequilibrium self-assembly dynamics of icosahedral viral capsids packaging genome or polyelectrolyte.

    abstract::The survival of viruses partly relies on their ability to self-assemble inside host cells. Although coarse-grained simulations have identified different pathways leading to assembled virions from their components, experimental evidence is severely lacking. Here, we use time-resolved small-angle X-ray scattering to unc...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/s41467-018-05426-8

    authors: Chevreuil M,Law-Hine D,Chen J,Bressanelli S,Combet S,Constantin D,Degrouard J,Möller J,Zeghal M,Tresset G

    更新日期:2018-08-06 00:00:00

  • Publisher Correction: Unravelling the immune signature of Plasmodium falciparum transmission-reducing immunity.

    abstract::The original version of this Article contained errors in Fig. 3. In panel a, bars from a chart depicting the percentage of antibody-positive individuals in non-infectious and infectious groups were inadvertently included in place of bars depicting the percentage of infectious individuals, as described in the Article a...

    journal_title:Nature communications

    pub_type: 杂志文章,已发布勘误

    doi:10.1038/s41467-018-03769-w

    authors: Stone WJR,Campo JJ,Ouédraogo AL,Meerstein-Kessel L,Morlais I,Da D,Cohuet A,Nsango S,Sutherland CJ,van de Vegte-Bolmer M,Siebelink-Stoter R,van Gemert GJ,Graumans W,Lanke K,Shandling AD,Pablo JV,Teng AA,Jones S,de Jong

    更新日期:2018-04-11 00:00:00

  • Lateral orbitofrontal cortex anticipates choices and integrates prior with current information.

    abstract::Adaptive behavior requires integrating prior with current information to anticipate upcoming events. Brain structures related to this computation should bring relevant signals from the recent past into the present. Here we report that rats can integrate the most recent prior information with sensory information, there...

    journal_title:Nature communications

    pub_type: 杂志文章

    doi:10.1038/ncomms14823

    authors: Nogueira R,Abolafia JM,Drugowitsch J,Balaguer-Ballester E,Sanchez-Vives MV,Moreno-Bote R

    更新日期:2017-03-24 00:00:00