Data and benchmarks
LADDER Probes Where Long Robot Tasks Break Down
The LADDER authors say their benchmark scales tasks from one to ten subgoals in the same environment to distinguish planning limits from physical execution errors.
Robot policies that handle short skills can deteriorate sharply when those skills must be chained, the LADDER authors report. A benchmark can separate planning from execution by extending tasks from one to ten subgoals in the same environment, then comparing direct low-level control with systems that add high-level orchestration.
LADDER applies this design across three suites, 145 tasks, eight categories and two robot embodiments, with 2,024 simulated episodes per system. The benchmark evaluates two orchestration paradigms: text subgoals for learned low-level policies and tool-selection orchestration.
When given the full instruction directly, the low-level policies achieved 85.4% on single-subgoal tasks but fell to near zero on extended tasks, according to the preprint. Text-subgoal orchestration improved on that collapse but reached only about 25%. The authors attribute this ceiling to accumulated physical-execution errors in the low-level policy rather than to high-level planning.
The reported comparison also includes tool-selection systems. GPT-Policy running on GPT-6 Astra significantly outperformed the learned-policy methods and succeeded about 62% of the time, although the authors report substantial time and cost demands. They also say experiments with two physical robot embodiments confirmed both the low-level-policy collapse and the benefit of orchestration, but the abstract provides no detailed real-world metrics. These results come from the authors’ preprint and are not independent confirmation; the useful diagnostic is the controlled increase in horizon combined with comparisons between planning layers and the physical policies that execute their decisions.