Existing manipulation benchmarks still provide limited support for systematically evaluating dual-arm coordination. They often underrepresent the diversity of coordination patterns required by two-arm manipulation, provide little support for reproducible sim-and-real evaluation, or rely mainly on binary task success.
That makes it hard to tell whether failures come from grasp acquisition, stabilization, object transfer, parallel execution, or later semantic stages of the task.
DuoBench is built around the Franka Research 3 Duo setup and exposes the same benchmark logic through a shared environment interface for simulation, teleoperation, replay, and evaluation.
The goal is not just to report whether a policy succeeds on a single lab setup, but to make the benchmark itself reproducible across labs. Tasks can be recreated in simulation and, for a subset, in the real world with 3D-printable assets and shared teleoperation tools.
duobench/<task_id> Gymnasium environments
DuoBench has eleven tasks in four coordination categories, defined by the functional roles the two arms take. Each task is split into stages that the environment reports at every step; four tasks are also replicated on the real robot with 3D-printable assets (switch their cards between simulated and real teleoperated demonstrations). Every task is a Gymnasium environment under duobench/<task_id> with its own language instruction.
One arm stabilizes or holds an object while the other arm interacts with it indirectly, e.g. holding a lid open while the other arm inserts a cube.
Holding the lid of a small chest open while inserting a cube. One arm must hold the lid while the other inserts the cube.
“open the box with the right arm and place the cube inside the box with the left arm”
duobench/hinge_chest
A spring-loaded microwave door requires one arm to hold it open while the other inserts a box.
“use the left arm to open the microwave door, then use the right arm to place the box inside the microwave, and close the door again”
duobench/spring_door
Two cups, one containing marbles. Both cups must be picked up, and the marbles must be poured into the other cup before both cups are placed back.
“grasp and lift both cups, then pour the marbles from one cup into the other and place the cups back to their original location inside the green square”
duobench/pour_marbles
Both arms jointly manipulate the same object, forming a closed kinematic chain, e.g. lifting a heavy pot.
Pick up a maze board with both arms and tilt it so a ball rolls into a target region.
“pick up the board and tilt it so the ball rolls onto the red square”
duobench/ball_maze
Carry a pot using both side handles and place it on a stove. Both arms are needed to lift the pot.
“use two arms to carry the pot at the handle on the stove”
duobench/carry_pot
Place a beam on a support cube and then place two rectangular blocks on the beam simultaneously.
“place the beam on the cube and then place the other blocks on the beam simultaneously using one arm for each cube”
duobench/block_balance
Connect two movable blocks together and then attach them to a peg on a third stationary block.
“join the two blocks using the peg on the left block and join the free socket of the right block with the peg on the wall”
duobench/join_blocks
An object is transferred between both arms due to workspace or task constraints; the role of the acting arm changes at the handover.
Hand over a cube between arms before placing it into a bowl.
“grasp the white cube with the right arm, hand it over to the left arm and place it in the white bowl with the left arm”
duobench/transfer_cube
Hand over a box between arms before placing it onto a mat. The box has to be passed through a gate.
“use the right arm to pick up the white box, and hand it over to the left arm through the hoop, then place it on the green mat with the left arm”
duobench/transfer_gate
The right arm picks up a peg and hands it over to the left arm so that the left arm can insert it into a socket.
“grasp the block with the right arm, hand it over to the left arm such that the left arm can easily insert the piece later, then insert the block into the socket with the left arm”
duobench/transfer_reorient
Both arms execute independent unimanual tasks, simultaneously or sequentially, without direct interdependence.
Sort two cubes into the matching bowls, testing simultaneous execution instead of direct cooperation.
“use the left arm to place the white cube in the white bowl; use the right arm to place the black cube in the black bowl”
duobench/bin_sort
Stage-conditional pass rates locate the same bottlenecks in both domains. In Ball-Maze, the bimanual grasp is reliable in simulation and on the real robot (266 to 284 of 300 simulated, 10 to 11 of 15 real episodes), and guiding the ball to the goal is the bottleneck, succeeding in 2 to 9% (sim) and 0 to 27% (real) of grasped episodes. In Transfer-Cube, picking up the cube is the bottleneck in simulation for all policies and on the real robot for ACT and X-VLA (0 and 1 of 15), whereas π0.5 picks more reliably on the real robot (11 of 15) than in simulation (140 of 300). Simulation thus locates bottlenecks faithfully even where it misestimates success rates.
We evaluate three representative policies with dual-arm support: ACT as a task-specific imitation-learning baseline, and the vision-language-action models π0.5 and X-VLA. ACT is trained per task on 50 demonstrations, while the VLAs are trained on all 550 simulation episodes with task-instruction conditioning. Each policy is evaluated for 300 rollouts per task.
DuoBench is far from saturated. Across policies, many failures already occur in the earliest stages: initial grasp acquisition and task setup remain the dominant bottlenecks even before more complex coordination is required. The bottleneck is a stage that requires establishing a grasp in 17 of the 33 task-policy combinations.
Parallel execution is especially difficult: Bin-Sort remains a weak point across policies, suggesting that current policies struggle with independent control of both arms and with demonstrations that admit multiple valid execution orders.
Mixed simulation-and-real training can help on some real-world tasks, but the sim-to-real gap remains clearly visible. That makes DuoBench useful not only for comparing policies within a domain, but also for studying transfer strategies and data-mixing choices.
| Task | ACT | π0.5 | X-VLA | ||||||
|---|---|---|---|---|---|---|---|---|---|
| SR [CI] | pfinal | bottleneck (ρk) | SR [CI] | pfinal | bottleneck (ρk) | SR [CI] | pfinal | bottleneck (ρk) | |
| Asymmetric Support | |||||||||
| Hinge-Chest | 26.7 [22–32] | 0.55 | cube + lid (0.39) | 26.7 [22–32] | 0.39 | lid or cube (0.56) | 12.0 [9–16] | 0.30 | cube + lid (0.37) |
| Spring-Door | 22.3 [18–27] | 0.50 | box + door (0.33) | 19.7 [16–25] | 0.44 | box + door (0.39) | 13.7 [10–18] | 0.38 | box + door (0.37) |
| Pour-Marbles | 0.3 [0–2] | 0.12 | pour all (0.23) | 0.0 [0–1] | 0.08 | pour all (0.00) | 0.0 [0–1] | 0.07 | place cups (0.00) |
| Bimanual Manipulation | |||||||||
| Carry-Pot | 54.0 [48–60] | 0.70 | grasp + lift (0.55) | 26.0 [21–31] | 0.51 | grasp + lift (0.30) | 12.3 [9–17] | 0.40 | grasp + lift (0.14) |
| Ball-Maze | 8.0 [5–12] | 0.71 | ball to goal (0.09) | 5.0 [3–8] | 0.67 | ball to goal (0.06) | 2.0 [1–4] | 0.71 | ball to goal (0.02) |
| Block-Balance | 0.0 [0–1] | 0.12 | grasp rects (0.00) | 0.0 [0–1] | 0.06 | grasp rects (0.00) | 0.0 [0–1] | 0.15 | release (0.00) |
| Join-Blocks | 2.3 [1–5] | 0.17 | connect (0.06) | 0.0 [0–1] | 0.16 | to wall (0.00) | 0.0 [0–1] | 0.20 | to wall (0.00) |
| Sequential Handoff | |||||||||
| Transfer-Cube | 10.0 [7–14] | 0.14 | pick cube (0.20) | 13.0 [10–17] | 0.28 | pick cube (0.47) | 9.3 [7–13] | 0.17 | pick cube (0.33) |
| Transfer-Gate | 42.7 [37–48] | 0.49 | pick box (0.59) | 23.3 [19–28] | 0.35 | pick box (0.49) | 26.3 [22–32] | 0.36 | pick box (0.51) |
| Transfer-Reorient | 0.3 [0–2] | 0.35 | insert (0.01) | 1.3 [1–3] | 0.36 | insert (0.04) | 0.0 [0–1] | 0.30 | insert (0.00) |
| Parallel Execution | |||||||||
| Bin-Sort | 0.7 [0–2] | 0.11 | pick 2nd (0.09) | 1.7 [1–4] | 0.14 | pick 2nd (0.16) | 1.0 [0–3] | 0.13 | pick 2nd (0.21) |
Real-world evaluation on the four replicated tasks shows the same overall pattern as simulation. Many failures happen early, with grasping and initial setup remaining the dominant bottlenecks. At the same time, once grasping succeeds, later stages are often completed reliably, especially in tasks such as Transfer-Cube and Bin-Sort.
Pick a task, a policy and the stage a rollout ends in to watch how that policy fails there. The bars show how many of the policy’s 300 evaluation rollouts end at each stage. Every clip is one of these rollouts, recorded by the head camera and played in real time.
Asymmetric Support
Bimanual Manipulation
Sequential Handoff
Parallel Execution
Fails at
The same task on the real Franka Research 3 Duo: the arms grasp and lift the printed maze and tilt it until the ball reaches the red goal. X-VLA succeeds in 3 of 15 real rollouts.
success failure
@misc{duobench,
title={{DuoBench}: A Reproducible Benchmark for Bimanual Manipulation in Simulation and the Real World},
author={Tobias J{\"u}lg and Seongjin Bien and Simon Hilber and Yannik Blei and Pierre Krack and Maximilian Li and Sven Parusel and Rudolf Lioutikov and Florian Walter and Wolfram Burgard},
year={2026},
url={https://arxiv.org/abs/2606.11901}
}