Theses Doctoral

Diverse and Scalable Skill Acquisition for Robot Manipulation

Xu, Zhenjia

The acquisition of capable robot manipulation skills is a critical prerequisite for the widespread deployment of robots in real-world environments, from household tasks to industrial applications. However, current robot manipulation systems remain limited in their ability to handle the diversity of objects, materials, and manipulation actions required in the real world. Data-driven methods have shown impressive results toward generalizing across a variety of problems, but existing approaches often require costly data collection using real robot platforms, hindering the scalability of skill acquisition.

In this dissertation, we aim to push the limits of robotic manipulation task diversity by providing mechanisms to acquire new skills in a scalable manner. Achieving the "right" data with large quantity and high quality is of vital importance. We approach this problem by leveraging physics simulators. Different from commonly used rigid body simulators, we have customized simulators to support deformable objects with diverse materials and dynamics. Aerodynamics and fracture effects are also included to enable a wider range of manipulation actions such as blowing and cutting. With sophisticated system design, including proper representation selection and customized hardware design, the policies trained in simulation can be seamlessly applied to real robots.

More specifically, this dissertation presents a series of works to address the challenges of diversity and scalability in robot manipulation skill acquisition. First, we introduce UMPNet, a universal policy network that can infer closed-loop action sequences for manipulating a wide range of articulated objects using only visual input. Second, we present DextAIRity, a system that leverages active airflow to enable safe and effective deformable object manipulation, expanding the repertoire of skills beyond traditional contact-based methods. Third, we describe RoboNinja, a cutting system for multi-material objects. With an interactive state estimator and an adaptive cutting policy, RoboNinja successfully removes the soft part of an object while preserving the rigid core.

Files

  • thumnail for Xu_columbia_0054D_18756.pdf Xu_columbia_0054D_18756.pdf application/pdf 5.65 MB Download File

More About This Work

Academic Units
Computer Science
Thesis Advisors
Song, Shuran
Degree
Ph.D., Columbia University
Published Here
September 18, 2024