Frontier stage

Closing the 100,000 year “data gap” in robotics

In person
  • Date
    8 July 2025
    Timeframe
    15:40 - 16:00 CEST
    Duration
    20 minutes
    Share this session

    AI is rapidly advancing the way we think, but we live in a material world. We still need to move things, make things, and maintain things. We need AI-driven humanoid robots to support an aging human population that does not have enough workers.  Large models based on internet-scale data can now pass the Turing Test for intelligence. In this sense, data has “solved” language and many analogously claim that data has solved speech recognition and computer vision.

    Will data also solve robotics?  Rich Sutton points out in the “Bitter Lesson” that data has surpassed all the best-laid analytic work in AI, and I accept that this trend will eventually hold sway in robotics. But when?

    Using commonly accepted metrics for converting word and image tokens into time, the amount of internet-scale data used to train contemporary large vision language models (VLMs) is on the order of 100,000 years. However, the data needed to train robots is not yet available.  One way to obtain such data is through human teleoperation, but the largest such dataset reported is on the order of 1 year. This suggests that humanoid robots will be available in…100,000 years.

    This 100,000-year “data gap” could be closed in three other ways: simulation, spatial analysis of internet videos, and collecting data from real robots operating in real environments.  Simulation works well for robots that fly or walk, but is notoriously unreliable for robot manipulation.  Spatial analysis of videos is progressing, but currently far too noisy to be useful for robot manipulation.  The last option – where data is collected as real robots operate in real commercial environments — requires bootstrapping with AI and “good old-fashioned engineering” to create robots with real return on investment that will be adopted by industry.  Such robots can create a “data flywheel” to increase performance and enable new functionality, accelerating the timeline to achieve reliable, general-purpose robots.