The purpose of this benchmark is to evaluate the planning and control aspects of robotic in-hand manipulation systems. The goal is to assess the system’s ability to change the position of a handheld object using the fingers, environment, or a combination of both. We provide examples of initial and goal states (i.e., static object poses and fingertip locations) for various in-hand manipulation tasks, given an object surface mesh from the YCB dataset. We further propose metrics that measure the error in reaching the goal state from a specific initial state. When aggregated across all tasks, these metrics also serve as a measure of the system’s in-hand manipulation capability. We provide supporting software, task examples, and evaluation results associated with the benchmark.
Supplementary notes can be added here, including code and math.