Algorithm allows robot to learn complex grasping skills from experience
14 May 2012
Cornell researchers have developed a new algorithm that allows a robot to learn complex grasping skills from experience, and to apply them in new situations. Inspired by the 'universal jamming gripper' created in the lab of Hod Lipson, associate professor of mechanical engineering and computer science, the new method is 'hardware agnostic' and will work with any type of robot gripper.
Lipson's gripper consists of a flexible bag filled with a granular material. With the new algorithm, the robot uses a 3D image of the object to examines a series of rectangles that match the size of the gripper, and tests each one on a variety of features. The robot is trained on images of many different objects until it has built a library of features common to good-grasping rectangles. Presented with a new object, it chooses the rectangle with the highest score based on the rules it has discovered. The robot also considers the overall size and shape of the object to choose a stable grasping point.
To test the method, researchers fitted an industrial robot arm with the jamming gripper and a Microsoft Kinect 3D camera. In trying to pick up 23 objects, including tools, toys and dishes, the robot succeeded an average of 90 to 100 percent of the time. In most cases, the robot was successfully able to grasp new objects that had not been in the training set. They ran the same tests with a simple 'pick it up at the centre' directive, scoring only 30 percent to 50 percent; except on flat objects, where both approaches tied at 89 percent.
The algorithm also was tested with the standard parallel jaws most modern robots use, with about the same results.
The work was done by Lipson and Ashutosh Saxena, assistant professor of computer science and a specialist in 'machine learning.' It will be presented on May 16 at the International Conference on Robotics and Automation in St. Paul, Minn. Co-authors of their paper are graduate students Yun Jiang and John Amend.
To see the gripper in action, click here.