Intelligent Robot Lab
Brown University, Providence RI

Motion Planning Benchmarks

One of the challenges in evaluating robot motion planning algorithms is a well-known lack of standard benchmarks. We aimed to resolve this problem by developing a benchmark suite of four different randomized planning problem generators. Each of the four parametrized scenarios uses a different robot to solve a different task; the random task generator fixes these parameters to produce a collection of motion planning tasks for each scenario.

We have bundled these environment generators as a ROS package for general use by the the robotics community. You can download the ROS package here: It was developed and tested in ROS Melodic on Ubuntu 18.04, by Sean Murray (sean at rtr dot ai). If you use these benchmarks, please cite the paper in which they were introduced:


  1. Place in a catkin workspace.
  2. From the root of the workspace, run:
    rosdep install --from-paths src --ignore-src -r -y
    (if you've never run rosdep before, you may get an error message that is clear and easily rectified).
  3. Build, with catkin build or catkin_make, depending on your preference.

Scene descriptions

  1. The shelf_place generator creates a set of shelves in front of a Fetch robot. The height of the shelves is randomly generated, and clutter on the shelves is randomly placed. A link named goal region and visualized with a red disk, is placed on a random location on a random shelf. Success is defined in this scenario by bringing the can in the Fetch's hand to the goal region.
  2. The power_strip generator involves a UR3 robot reaching to place a plug into a power strip, navigating around the wires already present. The power strip is randomly placed around the robot, and the strip is randomly populated with other plugs. The goal socket is indicated with a link called goal_region, and success is defined by bringing the plug in the UR3's hand to the goal region.
  3. The plate_grab generator involves a JACO robot reaching to grasp a plate in a dishwasher. The robot is randomly placed along one of the three sides of the dishwasher, and one of the plates is selected to be the goal, with the center of the target plate being indicated with a goal_region link. The JACO should reach to bring its palm link within reach to grasp the target plate.
  4. The machine_tend generator tried to model a more industrial task. A UR5 robot is placed outside of a CNC machine, with a piece of material attached to its end-effector. The goal is to bring the stock material above and aligned with a goal region link which is randomly placed inside the machine (yes, the placement of the link is not where a piece of stock would be placed in a real CNC machine).

Useful Commands

  • To create a set of randomized urdfs:
    $ roslaunch random_scene_gen gen_worlds.launch scene:=[scene_type] num_worlds:=[num_worlds]
    ... where scene_type is one of shelf_place|power_strip|plate_grab|machine_tend and num_worlds the number of desired test urdfs. The results will be places in the test_urdfs/ directory
  • To view one of the created worlds in rviz:
    $ roslaunch random_scene_gen view.launch scene:=[scene_type] world_num:=[world_num]
    ... where scene_type is one of shelf_place|power_strip|plate_grab|machine_tend and world_num is an id number less than the num_worlds you generated.


  • src/ contains the source files for generating the random urdfs
  • include/ contains some utility functions
  • base_urdfs/ contains the boiler plate for the random scenes
  • meshes/ contains the mesh data for the scenes
  • test_urdfs/ directory will be created the first time you run gen_worlds.launch