NVIDIA introduces Cosmos Policy for AI-powered robots

NVIDIA has introduced Cosmos Policy, an artificial intelligence model that turns camera images into robot movements and can also predict what will happen after each action. The approach aims to help robots perform complex tasks with less training data.

A model that observes, acts and plans

A robotics policy is the system that decides what a machine should do based on what it sees. For example, it receives an image of a cup and calculates how to move an arm to grab it without knocking over other objects.

Cosmos Policy builds on Cosmos Predict, a model trained to predict how scenes change over time. NVIDIA has adapted it to generate not only future frames, but also robot actions and an estimate of whether those actions will achieve the goal.

The difference is that the model treats actions, physical states and the probabilities of success as internal representations similar to video frames. This lets it use what it has already learned about movement, gravity and changes within a scene.

In practice, a single model can:

• Generate movement blocks to coordinate vision and the robotic arm.
• Predict what the scene will look like after an action.
• Evaluate different options and choose the one most likely to succeed.

It can work directly, producing the necessary movements, or as a planning system. In the second mode, it mentally tests several sequences, predicts their consequences and chooses the most promising one.

Results on manipulation tasks

NVIDIA evaluated Cosmos Policy on LIBERO and RoboCasa, two commonly used benchmarks for measuring whether a robot can complete multiple tasks, including those requiring many steps and precise coordination.

On LIBERO, it achieved an average success rate of 98.5%. That figure was higher than other models included in the comparison data shared by NVIDIA, such as OpenVLA-OFT, at 97.1%, and CogVLA, at 97.4%. On long-horizon tasks, Cosmos Policy reached 97.6%.

On RoboCasa, which simulates a range of household tasks, it achieved an average success rate of 67.1% using 50 demonstrations per task. That result was higher than models evaluated with more examples, such as π0, at 62.5% using 300 demonstrations, and GR00T-N1.5, at 64.1% using the same number.

The comparison does not mean that the robot can solve every household task. These results were obtained under the specific conditions of those benchmarks and are mainly useful for measuring generalization and data efficiency.

Planning improves results

When Cosmos Policy acts directly, it already achieves competitive results on most of the evaluated tasks. After adding model-based planning, NVIDIA observed 12.5% more tasks completed on average across two demanding real-world manipulation tasks.

It was also tested on bimanual tasks with ALOHA robots. According to NVIDIA, the system was able to execute long sequences using only visual observations, although the publication does not provide an overall success rate for those tests.

What changes for robotics

Many current systems are built on vision-language models. These models can understand an instruction such as "pick up the purple cup," but they do not necessarily know how to calculate the precise movements needed to control an arm, maintain balance or react if the object changes position.

Cosmos Policy aims to bridge that gap by using a model that already learns how scenes change over time. For developers, this could mean less need to train separate systems from scratch for perception, action and planning.

NVIDIA presents the project as an initial step and has released Cosmos Cookbook recipes so teams can test and adapt it. The company also notes that Cosmos Predict 2.5 is now available as an update to the model family.

A hackathon to test Cosmos

The company also announced Cosmos Cookoff, an open hackathon for building applications with Cosmos models in robotics, autonomous vehicles and video analysis. The event runs from January 29 to February 26, with teams of up to four people.

Prizes include $5,000 in cash, an NVIDIA DGX Spark workstation and a GeForce RTX 5090 GPU, among others. The goal is to take these models from benchmarks to prototypes that work in specific physical settings.

The key question now is whether Cosmos Policy's advantages hold up outside controlled tests, with new objects, different lighting and unexpected errors. If they do, models that predict the world could become a central part of teaching robots not only to see, but also to anticipate the consequences of their movements.