Runway’s open-source Praxis-1 turns video AI into a robot controller

NVIDIA Corp. — aI company Runway has released its robot model Praxis-1 with open weights and source code. The model uses what it learned from video to control robots and shows promising results in the company’s own tests.

Runway has released Praxis-1, its AI model for controlling robots. The company first introduced Praxis-1 in late September but initially gave access only to select partners. Alongside the release, Runway published a detailed research report with new performance tests. Runway’s model weights and source code are available under the Apache 2.0 license, which allows commercial use.Ad Praxis-1 takes in camera images, text instructions, and information about a robot’s position. From those inputs, it predicts how the environment will change and calculates the right robot movements. As new camera frames come in, the model continuously adjusts its control.Ad Praxis-1 builds on video training to control robots Praxis-1 builds on a video model that learned how objects move and interact from ordinary footage. Runway then fine-tuned it with videos of human actions and robot data. This approach could reduce the need for expensive robot-specific datasets, which have traditionally been collected just to train these kinds of systems. Runway is best known for generative video models like Gen-4.5. With Praxis-1, the company is bringing its video research into robotics. Runway calls Praxis-1 a World Action Model that predicts how a scene will change and derives matching robot movements from those predictions. Since September, robotics companies Standard Bots, Noble Machines, and Ultra have been testing Praxis-1 on different systems, including a humanoid robot, an industrial robot arm, and a dual-arm warehouse robot.Ad Praxis-1 still needs some specialized robot data. To adapt it to a Franka robot arm, Runway used about 277 hours of recordings from the public DROID research dataset, according to its research report. Those recordings show robots grasping and moving objects, for example. For a dual-arm robot, the company used between 1,400 and 2,600 hours of training material.