NVIDIA launches Cosmos Reason 2 for physical AI

NVIDIA has launched Cosmos Reason 2, an open AI model that analyzes images and videos to help robots and agents understand what is happening in the physical world, plan multiple steps and decide what to do next.

The company says the model is more accurate than its predecessor and ranks first among open models on the Physical AI Bench and Physical Reasoning leaderboards. These results come from the evaluations cited by NVIDIA.

From recognizing objects to understanding situations

Vision-language models can already identify objects, read text and describe an image. The problem appears when they need to interpret an entire situation: predict how something will move, act with incomplete information or adapt a plan when conditions change.

Cosmos Reason 2 aims to address that limitation. It analyzes space and time, uses common-sense and physics knowledge, and breaks a task down into steps. For example, it can study a factory video, locate a part, track its path and identify when an incident occurs.

The model can also provide:

• 2D and 3D point coordinates.
• Bounding boxes to locate objects.
• Motion trajectories.
• Text recognition through OCR, the technology that converts visible text into information a machine can read.

More context and two model sizes

Cosmos Reason 2 is available in versions with 2 billion and 8 billion parameters, an approximate measure of a model's internal size. The smaller version may make it easier to run on devices with limited resources, while the larger one offers more capacity for complex tasks and can be deployed in the cloud.

It also expands the context window to 256,000 tokens, compared with 16,000 for Cosmos Reason 1. In practice, this lets it work with much longer videos, instructions or conversations without losing the initial information as quickly.

NVIDIA presents it as a component for several types of systems:

• Video analysis: event searches, summaries and safety monitoring in workplaces.
• Data preparation: automatic video annotation for training other AI systems.
• Robotics: motion planning and choosing the next step.
• Autonomous driving: searching for specific situations in large collections of recordings.

Early uses in video and autonomous driving

Uber is exploring Cosmos Reason 2 to create searchable captions for videos used to train autonomous vehicles. In a recipe developed with the 8-billion-parameter model, adapting it to driving videos raised the BLEU score by 10.6%, from 0.113 to 0.125.

It also improved by 0.67% on a multiple-choice video question-answering test, from 80.18% to 80.85%, and by 13.8% on LingoQA, from 63.2% to 77.0%. These are results from a specific evaluation, not a guarantee of performance for every application.

Salesforce, meanwhile, uses agents to analyze videos captured by Cobalt robots during workplace safety and compliance tasks. Companies including Hitachi, Milestone and VAST Data are also using the Cosmos family in robotics, autonomous driving and video analysis.

What changes for you

This is not a consumer application that will immediately replace a phone assistant. Its main audience is developers building robots, surveillance systems, video analysis tools or autonomous vehicles.

The important difference is that the model does not simply say what appears in an image. It can try to explain what is happening, where objects are, how they are moving and what action might come next. That could reduce some of the manual work required to review hours of video or prepare training data.

Cosmos Reason 2 is already available to download on Hugging Face in its 2B and 8B versions, and to try on NVIDIA's platform. The company says it will soon arrive on Amazon Web Services, Google Cloud and Microsoft Azure.

The next step will be seeing whether its results in labs and demonstrations hold up in real-world environments, where there is poor lighting, unexpected objects and physical consequences. In AI applied to the physical world, better reasoning matters, but reliable action matters even more.