当前位置： SCI文献检索 > NATURE期刊下所有文献 > Vector-based navigation using grid-like representations in artificial agents.

Vector-based navigation using grid-like representations in artificial agents.

Abstract：

:Deep neural networks have achieved impressive successes in fields ranging from object recognition to complex games such as Go1,2. Navigation, however, remains a substantial challenge for artificial agents, with deep neural networks trained by reinforcement learning3-5 failing to rival the proficiency of mammalian spatial behaviour, which is underpinned by grid cells in the entorhinal cortex 6 . Grid cells are thought to provide a multi-scale periodic representation that functions as a metric for coding space7,8 and is critical for integrating self-motion (path integration)6,7,9 and planning direct trajectories to goals (vector-based navigation)7,10,11. Here we set out to leverage the computational functions of grid cells to develop a deep reinforcement learning agent with mammal-like navigational abilities. We first trained a recurrent network to perform path integration, leading to the emergence of representations resembling grid cells, as well as other entorhinal cell types 12 . We then showed that this representation provided an effective basis for an agent to locate goals in challenging, unfamiliar, and changeable environments-optimizing the primary objective of navigation through deep reinforcement learning. The performance of agents endowed with grid-like representations surpassed that of an expert human and comparison agents, with the metric quantities necessary for vector-based navigation derived from grid-like units within the network. Furthermore, grid-like representations enabled agents to conduct shortcut behaviours reminiscent of those performed by mammals. Our findings show that emergent grid-like representations furnish agents with a Euclidean spatial metric and associated vector operations, providing a foundation for proficient navigation. As such, our results support neuroscientific theories that see grid cells as critical for vector-based navigation7,10,11, demonstrating that the latter can be combined with path-based strategies to support navigation in challenging environments.

journal_name

Nature

journal_title

Nature

authors

Banino A,Barry C,Uria B,Blundell C,Lillicrap T,Mirowski P,Pritzel A,Chadwick MJ,Degris T,Modayil J,Wayne G,Soyer H,Viola F,Zhang B,Goroshin R,Rabinowitz N,Pascanu R,Beattie C,Petersen S,Sadik A,Gaffney S,King H

doi

10.1038/s41586-018-0102-6

subject

Has Abstract

pub_date

2018-05-01 00:00:00

pages

429-433

issue

7705

eissn

0028-0836

issn

1476-4687

pii

10.1038/s41586-018-0102-6

journal_volume

557

pub_type

杂志文章

在线工具

Vector-based navigation using grid-like representations in artificial agents.