NVIDIA introduces Nemotron Diffusion for text generation

NVIDIA has introduced Nemotron-Labs Diffusion, a family of models that can generate several tokens at once instead of writing text one word at a time. The promise is lower latency and better GPU utilization, especially when only one request is active or the number of users changes constantly.

Most current language models are autoregressive. That means they generate one token, a unit of text that can be a word or part of one, and then use it to produce the next. Each step requires running the model again, leaving much of the GPU's time devoted to moving data through memory.

Nemotron-Labs Diffusion takes a different approach. It first creates several tokens in parallel and then reviews them over multiple stages. If part of the text does not fit, the model can modify it before continuing. The process is more like writing a draft and revising it than completing a sentence with no chance to go back.

Three ways to generate text

The same model family can run in three modes, selected during deployment without major application changes:

• Autoregressive mode: generates text from left to right, like a conventional model. It preserves a familiar workflow and serves as a reference for accuracy.
• Diffusion mode: generates blocks of text and refines them iteratively. In NVIDIA's described implementation, it works with blocks of 32 tokens and decides which tokens are confident enough to lock in.
• Self-speculation: produces several tokens as a draft and then verifies them with autoregressive generation. The idea is to combine diffusion's parallelism with the traditional model's step-by-step checking.

This lets you adjust the trade-off between speed and resource consumption. Using fewer refinement steps lowers the compute cost, although it can also change the result or its quality.

NVIDIA's reported performance

The Nemotron-Labs Diffusion 8B model achieves 1.2% higher average accuracy than Qwen3 8B on the tasks evaluated by NVIDIA. For speed, the company uses TPF, or tokens per forward pass, a metric that measures how many tokens are processed in each model execution.

In diffusion mode, the model reaches 2.6 times more TPF than the autoregressive comparison models. Self-speculation reaches 6 times more with LinearSpec and 6.4 times more with QuadraticSpec, with comparable accuracy in the reported tests.

In another measurement on an NVIDIA B200 GPU using the SpeedBench test suite, the LinearSpec variant reached around 865 tokens per second, close to four times the reference autoregressive performance on the same hardware. These figures reflect the evaluation conditions reported by NVIDIA, not a guaranteed speed for every device or application.

Open models and training

The collection includes text models with 3,000, 8,000 and 14,000 million parameters, along with an 8,000-million-parameter vision-language model that can also work with images. There are base versions and variants tuned for conversation.

The text models are released under NVIDIA's open Nemotron license, which is designed to allow commercial use. The vision-language model uses an NVIDIA source-code license with different conditions, so you should review the terms before integrating it into a product.

NVIDIA trained the family with a combined objective: preserve the capabilities of an autoregressive model while adding diffusion-based generation. Pretraining used 1.3 trillion tokens, followed by another 45 billion for supervised fine-tuning.

What changes for developers

The practical advantage is not just achieving a higher score in a test. A team can use the same model in three ways depending on the situation: autoregressive when compatibility comes first, diffusion when more throughput is needed, and self-speculation when it wants to accelerate generation without moving too far from the conventional result.

Support for deploying these models will soon arrive in SGLang's main branch, a tool for serving language models. For now, the integration is available through a pull request in the project's repository.

NVIDIA's direction is clear: autoregressive generation and diffusion do not have to be separate model families. If this type of model maintains its accuracy outside controlled tests, the next major change will not be how you talk to an AI, but how long you wait for its response and how much hardware it needs to produce it.