Saurabh Gupta
UC Berkeley

Cognitive Mapping and Planning for Visual Navigation

Wednesday 07th of June 2017 at 12:00pm
560 Evans

We introduce a novel neural architecture for navigation in novel environments that learns a cognitive map from first person viewpoints and plans a sequence of actions towards goals in the environment. The Cognitive Mapper and Planner (CMP) is based on two key ideas: a) a unified joint architecture for mapping and planning, such that the mapping is driven by the needs of the planner, and b) a spatial memory with the ability to plan given an incomplete set of observations about the world. CMP constructs a top-down belief map of the world and applies a differentiable neural net planner to produce the next action at each time step. The accumulated belief of the world enables the agent to track visited regions of the environment. Our experiments demonstrate that CMP outperforms both reactive strategies and standard memory-based architectures and performs well even in novel environments. Furthermore, we show that CMP can also achieve semantically specified goals, such as “go to a chair”. This is joint work with James Davidson, Sergey Levine, Rahul Sukthankar and Jitendra Malik.
(video)

Join Email List

You can subscribe to our weekly seminar email list by sending an email to majordomo@lists.berkeley.edu that contains the words subscribe redwood in the body of the message.
(Note: The subject line can be arbitrary and will be ignored)