The LIBERO-RIG preprint, submitted to arXiv on October 13, 2026, examines equipment changes while robot tasks remain fixed. Its authors test nine vision-language-action models across eleven perturbation axes and five severity levels. They report pooled success of 0.932 under nominal conditions, 0.767 at middle severity and 0.509 at the highest severity. These are author-reported simulation results; our newsroom has not repeated the tests.
The task can stay the same while the setup changes
In our editorial interpretation, this evaluation matters because the request to a robot does not always change when its equipment changes. A cup may still need to move to the same place, while the camera now observes it from another angle. This is an explanatory example, not an additional measured result from the preprint.
A useful test should identify exactly what changed. If an object, instruction and setup all change together, the cause of failure is difficult to establish. We would preserve separate records for each configuration and check whether the task itself remains feasible. That connects an overall percentage to a specific limitation, rather than turning it into a label for the entire model.
What LIBERO means
The original LIBERO repository describes a benchmark for knowledge transfer in multitask and lifelong robot learning. It provides tasks and human teleoperation demonstrations, with suites addressing spatial relations, objects and goals. This is context for the foundation of the new work, not an independent reproduction of LIBERO-RIG.
Our analysis is that common tasks help comparisons when the protocol is clearly preserved. A ranking should also be read alongside training and execution conditions. If teams use different setups, closely spaced scores do not necessarily demonstrate a capability difference. Practical evaluation additionally requires conditions resembling the intended application.
Earlier context on benchmark limitations
A separate June 2 preprint, What Are We Actually Benchmarking in Robot Manipulation?, considers shortcut solving, statistically inconclusive differences, creeping overfitting and dependence on data sources. Its authors propose diagnostic checks. It is a different study and should not be merged with the new numerical findings.
Our conclusion from this context is that a good evaluation needs more than a high percentage. It should explain the capability measured, which alternative explanations were checked and what remains outside its scope. Repeating a familiar task can be useful without automatically demonstrating general understanding of an unfamiliar environment.
What developers should examine next
We would use such a benchmark to identify where to concentrate testing, rather than as automatic deployment approval. A physical machine requires evaluation of its particular sensors, mechanics, operating area and safe stopping behavior. Simulation checks and physical trials answer different questions.
The news is valuable for changing the focus: evaluating an AI robot together with the setup on which its actions depend. None of the percentages is a universal success probability in a factory or home. The next verifiable step is an independent repetition of the protocol and an explicit comparison with physical trials, before using the result to support broad promises of autonomy.
Sources: LIBERO-RIG — original preprint, LIBERO — original benchmark repository, What Are We Actually Benchmarking in Robot Manipulation?.
Editorial verification: 2026-10-14T16:34:39.030582+00:00 (UTC).
