Mastering Atari, Go, chess and shogi by planning with a learned model.

Abstract:

:Constructing agents with planning capabilities has long been one of the main challenges in the pursuit of artificial intelligence. Tree-based planning methods have enjoyed huge success in challenging domains, such as chess1 and Go2, where a perfect simulator is available. However, in real-world problems, the dynamics governing the environment are often complex and unknown. Here we present the MuZero algorithm, which, by combining a tree-based search with a learned model, achieves superhuman performance in a range of challenging and visually complex domains, without any knowledge of their underlying dynamics. The MuZero algorithm learns an iterable model that produces predictions relevant to planning: the action-selection policy, the value function and the reward. When evaluated on 57 different Atari games3-the canonical video game environment for testing artificial intelligence techniques, in which model-based planning approaches have historically struggled4-the MuZero algorithm achieved state-of-the-art performance. When evaluated on Go, chess and shogi-canonical environments for high-performance planning-the MuZero algorithm matched, without any knowledge of the game dynamics, the superhuman performance of the AlphaZero algorithm5 that was supplied with the rules of the game.

journal_name

Nature

journal_title

Nature

authors

Schrittwieser J,Antonoglou I,Hubert T,Simonyan K,Sifre L,Schmitt S,Guez A,Lockhart E,Hassabis D,Graepel T,Lillicrap T,Silver D

doi

10.1038/s41586-020-03051-4

subject

Has Abstract

pub_date

2020-12-01 00:00:00

pages

604-609

issue

7839

eissn

0028-0836

issn

1476-4687

pii

10.1038/s41586-020-03051-4

journal_volume

588

pub_type

杂志文章

相关文献

NATURE文献大全
  • A haplotype map of the human genome.

    abstract::Inherited genetic variation has a critical but as yet largely uncharacterized role in human disease. Here we report a public database of common variation in the human genome: more than one million single nucleotide polymorphisms (SNPs) for which accurate and complete genotypes have been obtained in 269 DNA samples fro...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature04226

    authors: International HapMap Consortium.

    更新日期:2005-10-27 00:00:00

  • Isolation of a cDNA encoding the vascular type-1 angiotensin II receptor.

    abstract::Angiotensin II is an important effector molecule controlling blood pressure and volume in the cardiovascular system. Its importance is manifested by the efficacy of angiotensin-converting enzyme inhibitors in the treatment of hypertension and congestive heart failure. Angiotensin II interacts with two pharmacologicall...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/351233a0

    authors: Murphy TJ,Alexander RW,Griendling KK,Runge MS,Bernstein KE

    更新日期:1991-05-16 00:00:00

  • No mixing of superconductivity and antiferromagnetism in a high-temperature superconductor.

    abstract::There is still no universally accepted theory of high-temperature superconductivity. Most models assume that doping creates 'holes' in the valence band of an insulating, antiferromagnetic 'parent' compound, and that antiferromagnetism and high-temperature superconductivity are intimately related. If their respective e...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature01544

    authors: Bozovic I,Logvenov G,Verhoeven MA,Caputo P,Goldobin E,Geballe TH

    更新日期:2003-04-24 00:00:00

  • A nucleotide regulatory site for somatostatin inhibition of adenylate cyclase in S49 lymphoma cells.

    abstract::The cyc- variants of S49 lymphoma cells have served as powerful tools for studying the components and mechanisms of hormone-induced adenylate cyclase stimulation, as these cells are deficient in the guanine nucleotide regulatory site (Ns) mediating hormone, guanine nucleotide, cholera toxin and fluoride-induced stimul...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/303177a0

    authors: Jakobs KH,Aktories K,Schultz G

    更新日期:1983-05-12 00:00:00

  • The operator-binding domain of lambda repressor: structure and DNA recognition.

    abstract::The structure of the operator-binding domain of the lambda repressor has been determined at 3.2 A resolution. This domain contains an extended N-terminal arm and five alpha-helices. Model-building studies of the repressor-operator complex suggest that alpha-helices, especially the N-terminal parts of these helices, ma...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/298443a0

    authors: Pabo CO,Lewis M

    更新日期:1982-07-29 00:00:00

  • The protein Sex-lethal antagonizes the splicing factor U2AF to regulate alternative splicing of transformer pre-mRNA.

    abstract::Somatic sexual differentiation in Drosophila melanogaster involves a cascade of regulated splicing events and provides an attractive model system for the analysis of alternative splicing mechanisms. The protein Sex-lethal (Sxl) activates a female-specific 3' splice site in the first intron of transformer (tra) pre-mRN...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/362171a0

    authors: Valcárcel J,Singh R,Zamore PD,Green MR

    更新日期:1993-03-11 00:00:00

  • Hot streaks in artistic, cultural, and scientific careers.

    abstract::The hot streak-loosely defined as 'winning begets more winnings'-highlights a specific period during which an individual's performance is substantially better than his or her typical performance. Although hot streaks have been widely debated in sports1,2, gambling3-5 and financial markets6,7 over the past several deca...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/s41586-018-0315-8

    authors: Liu L,Wang Y,Sinatra R,Giles CL,Song C,Wang D

    更新日期:2018-07-01 00:00:00

  • Structure of the Vdelta domain of a human gammadelta T-cell antigen receptor.

    abstract::Antigen recognition by T lymphocytes is mediated by cell-surface glycoproteins known as T-cell antigen receptors (TCRs). These are composed of alpha and beta, or gamma and delta, polypeptide chains with variable (V) and constant (C) regions. In contrast to alphabeta TCRs, which recognize antigen only as peptide fragme...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/35172

    authors: Li H,Lebedeva MI,Llera AS,Fields BA,Brenner MB,Mariuzza RA

    更新日期:1998-01-29 00:00:00

  • Regulation of Arabidopsis cryptochrome 2 by blue-light-dependent phosphorylation.

    abstract::Cryptochromes are blue/ultraviolet-A light receptors that mediate various light responses in plants and animals. But the initial photochemical reaction of cryptochrome is still unclear. For example, although most photoreceptors are known to undergo light-dependent protein modification such as phosphorylation, no blue-...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature00815

    authors: Shalitin D,Yang H,Mockler TC,Maymon M,Guo H,Whitelam GC,Lin C

    更新日期:2002-06-13 00:00:00

  • FeO2 and FeOOH under deep lower-mantle conditions and Earth's oxygen-hydrogen cycles.

    abstract::The distribution, accumulation and circulation of oxygen and hydrogen in Earth's interior dictate the geochemical evolution of the hydrosphere, atmosphere and biosphere. The oxygen-rich atmosphere and iron-rich core represent two end-members of the oxygen-iron (O-Fe) system, overlapping with the entire pressure-temper...

    journal_title:Nature

    pub_type: 历史文章,杂志文章

    doi:10.1038/nature18018

    authors: Hu Q,Kim DY,Yang W,Yang L,Meng Y,Zhang L,Mao HK

    更新日期:2016-06-09 00:00:00

  • Long-lived magnetism from solidification-driven convection on the pallasite parent body.

    abstract::Palaeomagnetic measurements of meteorites suggest that, shortly after the birth of the Solar System, the molten metallic cores of many small planetary bodies convected vigorously and were capable of generating magnetic fields. Convection on these bodies is currently thought to have been thermally driven, implying that...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature14114

    authors: Bryson JF,Nichols CI,Herrero-Albillos J,Kronast F,Kasama T,Alimadadi H,van der Laan G,Nimmo F,Harrison RJ

    更新日期:2015-01-22 00:00:00

  • An unexpected specialization for horizontal disparity in primate primary visual cortex.

    abstract::The horizontal separation of the eyes means that objects nearer or farther than the fixation point project to different locations on the two retinae, differing principally in their horizontal coordinates (horizontal binocular disparity). Disparity-selective neurons have generally been studied with disparities applied ...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature00909

    authors: Cumming BG

    更新日期:2002-08-08 00:00:00

  • Stress changes from the 2008 Wenchuan earthquake and increased hazard in the Sichuan basin.

    abstract::On 12 May 2008, the devastating magnitude 7.9 (Wenchuan) earthquake struck the eastern edge of the Tibetan plateau, collapsing buildings and killing thousands in major cities aligned along the western Sichuan basin in China. After such a large-magnitude earthquake, rearrangement of stresses in the crust commonly leads...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature07177

    authors: Parsons T,Ji C,Kirby E

    更新日期:2008-07-24 00:00:00

  • Nuclear reprogramming and pluripotency.

    abstract::The cloning of mammals from differentiated donor cells has refuted the old dogma that development is an irreversible process. It has demonstrated that the oocyte can reprogramme an adult nucleus into an embryonic state that can direct development of a new organism. The prospect of deriving patient-specific embryonic s...

    journal_title:Nature

    pub_type: 杂志文章,评审

    doi:10.1038/nature04955

    authors: Hochedlinger K,Jaenisch R

    更新日期:2006-06-29 00:00:00

  • Quantum phase transition from a superfluid to a Mott insulator in a gas of ultracold atoms.

    abstract::For a system at a temperature of absolute zero, all thermal fluctuations are frozen out, while quantum fluctuations prevail. These microscopic quantum fluctuations can induce a macroscopic phase transition in the ground state of a many-body system when the relative strength of two competing energy terms is varied acro...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/415039a

    authors: Greiner M,Mandel O,Esslinger T,Hänsch TW,Bloch I

    更新日期:2002-01-03 00:00:00

  • Chaos and threshold for irreversibility in sheared suspensions.

    abstract::Systems governed by time reversible equations of motion often give rise to irreversible behaviour. The transition from reversible to irreversible behaviour is fundamental to statistical physics, but has not been observed experimentally in many-body systems. The flow of a newtonian fluid at low Reynolds number can be r...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature04380

    authors: Pine DJ,Gollub JP,Brady JF,Leshansky AM

    更新日期:2005-12-15 00:00:00

  • Strain-resolved community proteomics reveals recombining genomes of acidophilic bacteria.

    abstract::Microbes comprise the majority of extant organisms, yet much remains to be learned about the nature and driving forces of microbial diversification. Our understanding of how microorganisms adapt and evolve can be advanced by genome-wide documentation of the patterns of genetic exchange, particularly if analyses target...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature05624

    authors: Lo I,Denef VJ,Verberkmoes NC,Shah MB,Goltsman D,DiBartolo G,Tyson GW,Allen EE,Ram RJ,Detter JC,Richardson P,Thelen MP,Hettich RL,Banfield JF

    更新日期:2007-03-29 00:00:00

  • Food-web interactions govern the resistance of communities after non-random extinctions.

    abstract::Growing concern about how loss of biodiversity will affect ecosystems has stimulated numerous studies. Although most studies have assumed that species go extinct randomly, species often go extinct in order of their sensitivity to a stress that intensifies through time (such as climate change). Here we show that the co...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature02515

    authors: Ives AR,Cardinale BJ

    更新日期:2004-05-13 00:00:00

  • Direct measurement of proton transfer rates to a group controlling the dihydropyridine-sensitive Ca2+ channel.

    abstract::Protons participate in most biologically important reactions, as substrates, products, cofactors and modulators, and proton transport is an essential step in energy transduction. The dynamics of protonation reactions have been studied extensively in solution and in model systems involving lipid-water interfaces, but h...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/329243a0

    authors: Prod'hom B,Pietrobon D,Hess P

    更新日期:1987-09-17 00:00:00

  • Circulating mitochondrial DAMPs cause inflammatory responses to injury.

    abstract::Injury causes a systemic inflammatory response syndrome (SIRS) that is clinically much like sepsis. Microbial pathogen-associated molecular patterns (PAMPs) activate innate immunocytes through pattern recognition receptors. Similarly, cellular injury can release endogenous 'damage'-associated molecular patterns (DAMPs...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature08780

    authors: Zhang Q,Raoof M,Chen Y,Sumi Y,Sursal T,Junger W,Brohi K,Itagaki K,Hauser CJ

    更新日期:2010-03-04 00:00:00

  • Directional dominance on stature and cognition in diverse human populations.

    abstract::Homozygosity has long been associated with rare, often devastating, Mendelian disorders, and Darwin was one of the first to recognize that inbreeding reduces evolutionary fitness. However, the effect of the more distant parental relatedness that is common in modern human populations is less well understood. Genomic da...

    journal_title:Nature

    pub_type: 杂志文章,meta分析

    doi:10.1038/nature14618

    authors: Joshi PK,Esko T,Mattsson H,Eklund N,Gandin I,Nutile T,Jackson AU,Schurmann C,Smith AV,Zhang W,Okada Y,Stančáková A,Faul JD,Zhao W,Bartz TM,Concas MP,Franceschini N,Enroth S,Vitart V,Trompet S,Guo X,Chasman DI,

    更新日期:2015-07-23 00:00:00

  • MHC class II function preserved by low-affinity peptide interactions preceding stable binding.

    abstract::Major histocompatibility complex class II molecules and their peptide ligands show unusual interaction kinetics, with slow association and dissociation rates that yield an apparent equilibrium constant of approximately 10(-6)-10(-8) M (refs 1-5). However, there is evidence for a specific, rapidly formed, short-lived c...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/370647a0

    authors: Sadegh-Nasseri S,Stern LJ,Wiley DC,Germain RN

    更新日期:1994-08-25 00:00:00

  • Identification and inhibition of the ICE/CED-3 protease necessary for mammalian apoptosis.

    abstract::The protease responsible for the cleavage of poly(ADP-ribose) polymerase and necessary for apoptosis has been purified and characterized. This enzyme, named apopain, is composed of two subunits of relative molecular mass (M(r)) 17K and 12K that are derived from a common proenzyme identified as CPP32. This proenzyme is...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/376037a0

    authors: Nicholson DW,Ali A,Thornberry NA,Vaillancourt JP,Ding CK,Gallant M,Gareau Y,Griffin PR,Labelle M,Lazebnik YA

    更新日期:1995-07-06 00:00:00

  • Heterodimerization of the Drosophila ecdysone receptor with retinoid X receptor and ultraspiracle.

    abstract::Ecdysone in Drosophila has been a paradigm for steroid hormones since its ability to induce gene activity directly was demonstrated by its effects on moulting and polytene chromosome puffing. The ecdysone receptor (EcR) was recently confirmed as a member of the nuclear receptor superfamily by cloning and characterizat...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/362471a0

    authors: Thomas HE,Stunnenberg HG,Stewart AF

    更新日期:1993-04-01 00:00:00

  • Novel neurotrophic factor CDNF protects and rescues midbrain dopamine neurons in vivo.

    abstract::In Parkinson's disease, brain dopamine neurons degenerate most prominently in the substantia nigra. Neurotrophic factors promote survival, differentiation and maintenance of neurons in developing and adult vertebrate nervous system. The most potent neurotrophic factor for dopamine neurons described so far is the glial...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature05957

    authors: Lindholm P,Voutilainen MH,Laurén J,Peränen J,Leppänen VM,Andressoo JO,Lindahl M,Janhunen S,Kalkkinen N,Timmusk T,Tuominen RK,Saarma M

    更新日期:2007-07-05 00:00:00

  • Volcanic carbon dioxide vents show ecosystem effects of ocean acidification.

    abstract::The atmospheric partial pressure of carbon dioxide (p(CO(2))) will almost certainly be double that of pre-industrial levels by 2100 and will be considerably higher than at any time during the past few million years. The oceans are a principal sink for anthropogenic CO(2) where it is estimated to have caused a 30% incr...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature07051

    authors: Hall-Spencer JM,Rodolfo-Metalpa R,Martin S,Ransome E,Fine M,Turner SM,Rowley SJ,Tedesco D,Buia MC

    更新日期:2008-07-03 00:00:00

  • A comprehensive analysis of protein-protein interactions in Saccharomyces cerevisiae.

    abstract::Two large-scale yeast two-hybrid screens were undertaken to identify protein-protein interactions between full-length open reading frames predicted from the Saccharomyces cerevisiae genome sequence. In one approach, we constructed a protein array of about 6,000 yeast transformants, with each transformant expressing on...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/35001009

    authors: Uetz P,Giot L,Cagney G,Mansfield TA,Judson RS,Knight JR,Lockshon D,Narayan V,Srinivasan M,Pochart P,Qureshi-Emili A,Li Y,Godwin B,Conover D,Kalbfleisch T,Vijayadamodar G,Yang M,Johnston M,Fields S,Rothberg JM

    更新日期:2000-02-10 00:00:00

  • Evidence of Xist RNA-independent initiation of mouse imprinted X-chromosome inactivation.

    abstract::XX female mammals undergo transcriptional silencing of most genes on one of their two X chromosomes to equalize X-linked gene dosage with XY males in a process referred to as X-chromosome inactivation (XCI). XCI is an example of epigenetic regulation. Once enacted in individual cells of the early female embryo, XCI is...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature08161

    authors: Kalantry S,Purushothaman S,Bowen RB,Starmer J,Magnuson T

    更新日期:2009-07-30 00:00:00

  • A single amino acid of the cholecystokinin-B/gastrin receptor determines specificity for non-peptide antagonists.

    abstract::The brain cholecystokinin-B/gastrin receptor (CCK-B/gastrin) has been implicated in mediating anxiety, panic attacks, satiety, and the perception of pain. The canine and human CCK-B/gastrin receptors share 90% amino-acid identity and have similar agonist affinities. These receptors can be selectively blocked by the no...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/362348a0

    authors: Beinborn M,Lee YM,McBride EW,Quinn SM,Kopin AS

    更新日期:1993-03-25 00:00:00

  • Species-area relationships always overestimate extinction rates from habitat loss.

    abstract::Extinction from habitat loss is the signature conservation problem of the twenty-first century. Despite its importance, estimating extinction rates is still highly uncertain because no proven direct methods or reliable data exist for verifying extinctions. The most widely used indirect method is to estimate extinction...

    journal_title:Nature

    pub_type: 杂志文章

    doi:10.1038/nature09985

    authors: He F,Hubbell SP

    更新日期:2011-05-19 00:00:00