Open Benchmark Evaluates Skills of Robot Collectors

Scientists from the Oxford Institute of Robotics, the Center for Production Technologies (MTC), and the University of Birmingham have developed a special benchmark for evaluating and comparing the abilities of different robots that collect various products from individual details. The benchmark, called RAMP (Robotic Assembly Manipulation and Planning), is an open benchmark that is available to anyone for use and supplementation.

RAMP is designed to enable the evaluation of the performance of robots through a set of tasks and criteria. The benchmark consists of a series of basic parts in the form of beams of different colors and sizes that can be connected to each other. The task of the robot is to collect a certain object from these parts, such as a cube or a pyramid, while independently choosing the assembly procedure to avoid getting stuck in a dead end.

In order to test their benchmark, the scientists used the real UR10 robot and the computer simulation of NVIDIA ISAAC SIM. They also proposed a basic method for solving the RAMP problem based on the deep learning algorithm with reinforcement. This method allows the robot to learn from its own experience by receiving rewards for correct actions and a fine for errors.

The results showed that the basic method works well in simulation but is not very effective on a real robot. This indicates that the RAMP task is challenging and requires further research and improvements.

“We want to create a platform that will stimulate the development of robotics from an applied point of view in production and assembly,” said Professor Ingmar Pozner from the Oxford Institute of Robotics. The RAMP benchmark is open to everyone for participation, Professor Pozner added.

Overall, the development of RAMP is significant because it creates a standard for the assessment of robots that collect various products from individual details.