Motion planning from demonstrations

Extracting the constraints of a task like pouring or scooping from a kinesthetic demonstration, and reusing them for new object positions, new objects, and harvesting in vertical farms.

Representing complex manipulation tasks, like scooping and pouring, as a sequence of constant screw motions in SE(3) allows us to extract the task-related constraints on the end-effector’s motion from kinesthetic demonstrations and transfer them to new instances of the same task. The motion plans are computed with screw linear interpolation (ScLERP), which satisfies these constraints kinematically.

Pouring with a Franka Emika Panda.

We have evaluated this approach on scooping and pouring, and in containerized vertical farms for transplanting and harvesting leafy crops.

Harvesting leafy crops in a containerized vertical farm.

We have also developed an approach to transfer the task-related constraints between objects that are functionally similar but have different geometries. The notion of functional similarity is captured by a knowledge base.

Knowledge-enabled motion generation pipeline
Knowledge-enabled motion generation: the robot queries a knowledge base for a relevant demonstration and the object's geometric attributes, then plans with ScLERP.

More recently, we developed a self-evaluation-based approach that lets the robot compute the minimal set of kinesthetic demonstrations needed to perform tasks like pouring and scooping reliably over a region of its workspace. See How many demonstrations are enough?

Papers