Topics
Explore 14 articles on Data and benchmarks from How Robots Learn.
Use difficulty-weighted gates for unequal subtasks, preserve which conjunctive conditions were satisfied and report outstanding achievability checks separately from observed failures. Fiatlux applies this structure in simulation, although the available excerpt does not expose its exact aggregate-score arithmetic.
Written
A benchmark can distinguish temporal task understanding from prior-driven guessing by scoring recognition, matching, ordering, prediction, correspondence, and localization separately, then repeating matched questions without visual observations. The reported matching and ordering scores diverge sharply, and next-action prediction changed far less than current-action recognition when observations were removed.
Published
Spa-Bench compares in-distribution and out-of-distribution spatial configurations while retaining training support for relevant objects, concepts, and actions. The authors use diagnostic conditions to probe physical execution, sensitivity to wording, object-task associations, and recognition that no action is required; matched direct instructions improved success in the reported results.
Published
The admitted excerpts support coordinating data collection, scene reset, policy execution, result logging, and task-factor analysis, while using success classifiers and reset policies as mechanisms for repeatable trials; they do not provide a complete implementation recipe.
Published
In the LeRobot demo dataset, RGB observations have shape `(height, width, 4)`, use `numpy.uint8`, range from 0 to 255, and have resolutions of 720 × 720 for the head camera and 480 × 480 for wrist cameras. Linear depth measures the distance from the camera to scene geometry, has shape `(height, width)`, uses `numpy.float32`, ranges from 0 to 10 meters, and is stored in videos that are log-quantized during replay and then dequantized to metric depth values.
Written
Raw HDF5 files retain the information required for exact trajectory replay in OmniGibson and for collecting additional visual observations. The corresponding LeRobot release presents 20,000 demonstrations across 100 tasks in a customized LeRobot V3 structure with documented RGB and depth data.
Written
Variation across tasks, environments, arms, and grippers can widen the conditions a robot policy handles, but the available studies do not isolate diversity from dataset scale or establish universal gains.
Written
The EgoVerse authors report that robot policy performance generally improves with more human demonstrations when the data align with the learning objectives, although the available abstract does not quantify the gains or define alignment.
Written
Stage-level evaluation can show when and how a bimanual policy breaks down, separating alignment, slipping, coordination, arm-use asymmetry, efficiency and safety problems that binary success can conceal.
Written
A dual-arm benchmark can probe recomposition by testing unseen combinations of familiar per-arm skills, enforcing order and timing requirements, and comparing policies under shared data and evaluation conditions.
Written
A guide to checking a robot success detector, using AutoEval and a separate study of visual changes.
Written
VOMMI reconstructs motion from synchronized body and hand RGB views, then uses local-motion tokens to condition vision-language-action post-training. The authors report improvements in trajectory reconstruction, base-velocity error, and mean success across three real-robot tasks.
Written
RawVLA tests how camera image processing changes robot manipulation results under altered light, with a small physical trial and clear limits.
Written
Why robot benchmark results need task, metric and hardware context, with examples from physical pairwise comparisons and object-shape stress tests.
Written