This website uses cookies primarily for visitor analytics. Certain pages will ask you to fill in contact details to receive additional information. On these pages you have the option of having the site log your details for future visits. Indicating you want the site to remember your details will place a cookie on your device. To view our full cookie policy, please click here. You can also view it at any time by going to our Contact Us page.

New technique enables robot mastery of skills via trial and error

25 May 2015

Researchers have developed algorithms that enable robots to learn motor tasks through trial and error using a process that approximates to the way humans learn.

BRETT (Berkeley Robot for the Elimination of Tedious Tasks), a Willow Garage Personal Robot 2 (PR2) learns a new task through trial and error (image taken from video clip below, courtesy of UC Berkeley Robot Learning Lab)

The University of California (UC) Berkeley researchers demonstrated their technique - a type of reinforcement learning - by having a robot complete various tasks, such as putting a clothes hanger on a rack, assembling a toy plane, screwing a cap on a water bottle and more, without pre-programmed details about its surroundings.

“What we’re reporting on here is a new approach to empowering a robot to learn,” says Professor Pieter Abbeel of UC Berkeley’s Department of Electrical Engineering and Computer Sciences. “The key is that when a robot is faced with something new, we won’t have to reprogram it. The exact same software, which encodes how the robot can learn, was used to allow the robot to learn all the different tasks we gave it.”

Conventional, but impractical, approaches to helping a robot make its way through a 3D world include pre-programming it to handle the vast range of possible scenarios or creating simulated environments within which the robot operates.

Instead, the UC Berkeley researchers turned to a new branch of artificial intelligence known as 'deep learning', which is loosely inspired by the neural circuitry of the human brain when it perceives and interacts with the world.

In artificial intelligence, deep learning programs create 'neural nets' in which layers of artificial neurons process overlapping raw sensory data, whether it be sound waves or image pixels. This helps the robot recognise patterns and categories among the data it is receiving. 

Applying deep reinforcement learning to motor tasks has been far more challenging, however, since the task goes beyond the passive recognition of images and sounds.

In the experiments, the UC Berkeley researchers worked with a Willow Garage Personal Robot 2 (PR2), which they nicknamed BRETT (Berkeley Robot for the Elimination of Tedious Tasks).

They presented BRETT with a series of motor tasks, such as placing blocks into matching openings or stacking Lego blocks. The algorithm controlling BRETT’s learning included a reward function that provided a score based upon how well the robot was doing with the task.

BRETT takes in the scene, including the position of its own arms and hands, as viewed by the camera. The algorithm provides real-time feedback via the score based upon the robot’s movements. Movements that bring the robot closer to completing the task will score higher than those that do not. The score feeds back through the neural net, so the robot can learn which movements are better for the task at hand.

This end-to-end training process underlies the robot’s ability to learn on its own. As the PR2 moves its joints and manipulates objects, the algorithm calculates good values for the 92,000 parameters of the neural net it needs to learn.

With this approach, when given the relevant coordinates for the beginning and end of the task, the PR2 could master a typical assignment in about ten minutes. When the robot is not given the location for the objects in the scene and needs to learn vision and control together, the learning process takes about three hours.

Professor Abbeel says the field will likely see significant improvements as the ability to process vast amounts of data improves.

“With more data, you can start learning more complex things,” he says. “We still have a long way to go before our robots can learn to clean a house or sort laundry, but our initial results indicate that these kinds of deep learning techniques can have a transformative effect in terms of enabling robots to learn complex tasks entirely from scratch. In the next five to ten years, we may see significant advances in robot learning capabilities through this line of work.”

Print this page | E-mail this page