Robot learning's reproducibility problem is definitional, not technical

The bottleneck in teaching robots new skills is not hardware or data volume. It is that the field has no shared definition of "progress," so two papers that grade a robot on how it is doing may not be measuring the same thing at all.

A comprehensive survey submitted to arXiv on 22 July 2026 makes the field's own diagnosis explicit. The survey's authors argue the literature lacks a shared framework because methods differ along five axes at once: the observations they read, the goals they accept, the output signals they emit, the supervision sources they use, and the benchmarks they test on. The robot that scores "75 percent progress" in one paper and the one that scores the same number in another may be answering different questions.

This is a measurement problem before it is a technical one. The survey's three-layer model — interface, internal methods, data and benchmarks — is what alignment would have to look like, and the survey's own admission is that alignment is missing. The next paper to claim a state-of-the-art number will inherit the same caveat until researchers settle what "progress" means before they try to measure it.