World Labs turns one real-world robot task into thousands of simulated variations for training

World Labs, the startup founded by AI pioneer Fei-Fei Li, has unveiled a simulation engine that trains robot control systems entirely in virtual environments. The models then run reliably for hours on real hardware.

The company’s “Real-to-Sim-to-Real” (R2S2R) engine turns real-world robot tasks into simulations for training and evaluating control models, cutting out expensive tests on actual hardware. The technology comes from SceniX, a startup World Labs acquired in July. The main bottleneck in robot deployment isn’t model architecture, World Labs says, but the sheer volume of experience a robot needs to operate reliably. Real-world data is expensive and hard to control, and even online videos don’t systematically cover the full range of objects, physical conditions, and failure states. One real-world task becomes thousands of controlled variations The engine captures robots, sensors, the environment, and task demos, then rebuilds them as an interactive virtual world that doesn’t just look like the original but behaves the same way physically. World Labs pulls this off by combining generative world models with task-oriented robot simulation. From a single real-world task, the system generates thousands of variations by changing lighting, object position and count, the surrounding environment, physical properties like friction, and camera angle. To check accuracy, World Labs runs the same action sequence in simulation and reality side by side and compares observations, object movements, and outcomes. The examples shown include rigid, movable, and deformable objects such as cable routing, inserting an elastic cable end into a hole, and packing a box with both hands. Control models that never trained on real hardware Control models train in simulation and then transfer to real robots. One of the test platforms was ALOHA, an open-source dual-arm design from Stanford operated through puppeteering with two smaller control arms. The setup costs a fraction of commercial systems, and all blueprints are public, making ALOHA the go-to reference platform in robotics research. According to World Labs, the models each ran for one hour across four additional robot platforms without any human intervention. Tasks ranged from wrapping a power cord around a refrigerator with both hands to precisely repositioning test tubes and separating thin objects like markers or pencils from a dense jumble. The system isn’t tied to a specific control model or robot type, so a world that’s been reconstructed once can be reused later for new models and different robots, the company says. Simulation can stand in for hardware during policy evaluation