Robotic picking and packing with physical reasoning

This Fellowship focuses on robotic object manipulation. Consider the example of packing different items into a box in a warehouse to ship the box to a customer. To perform this, a robot would need to pick and insert the items into the box one by one, while nudging, pushing and squeezing objects, to achieve a tight packing. The robot would need to plan and control its actions, as well as use sensors to estimate the positions and deformations of the objects in the box.

The dominant approach to object manipulation in the literature is geometry-based, where the world is represented using shapes and configurations only. The central vision of this Fellowship is to go beyond that, by enabling robots to reason about, plan in, and control the full physics of the world. This has the potential to transform robots' object manipulation skills and our lives, because robots will be able to perform a much diverse variety of object manipulation skills applicable to manufacturing, assembly, and services.

This Fellowship will also target a particular application area: picking and packing of objects for warehouse automation. With the rapid advance of e-commerce over the past decade, there is a pressing need to have efficient warehouse automation systems, in the UK and the world. However, existing robotic systems do not have physics-based reasoning, which limits their applications drastically. The physics-based picking and packing approach that I propose will enable robots to reach into cluttered bins/shelves/bags, pushing, nudging, and squeezing arbitrary objects to search and retrieve a particular object or to pack multiple objects tightly for shipping --- skills that do not exist in any current system.
 

Publications and outputs

https://robotpickpack.leeds.ac.uk/
https://gow.epsrc.ukri.org/NGBOViewGrant.aspx?GrantRef=EP/V052659/1