Link Search Menu Expand Document

Learning to Compose Hierarchical Object-Centric Controllers for Robotic Manipulation

Paper PDF


Mohit Sharma (Carnegie Mellon University); Jacky Liang (Carnegie Mellon University); Jialiang Zhao (Carnegie Mellon University); Alex Lagrassa (Carnegie Mellon University); Oliver Kroemer (Carnegie Mellon University)

Interactive Session

2020-11-16, 11:50 - 12:20 PST | PheedLoop Session


Manipulation tasks can often be decomposed into multiple subtasks performed in parallel, e.g., sliding an object to a goal pose while maintaining contact with a table. Individual subtasks can be achieved by task-axis controllers defined relative to the objects being manipulated, and a set of object-centric controllers can be combined in an hierarchy. In prior works, such combinations are defined manually or learned from demonstrations. By contrast, we propose using reinforcement learning to dynamically compose hierarchical object-centric controllers for manipulation tasks. Experiments in both simulation and real world show how the proposed approach leads to improved sample efficiency, zero-shot generalization to novel test environments, and simulation-to-reality transfer without fine-tuning.


Reviews and Rebuttal

Reviews & Rebuttal

Conference on Robot Learning 2020