DiffusionGemma speeds up AI text generation

Google has introduced DiffusionGemma, an experimental open-source model that generates text up to 4 times faster in certain configurations. Instead of writing one word at a time, it processes complete blocks of up to 256 tokens and corrects them over several passes.

The model is mainly designed to run AI locally, on a computer with a dedicated GPU. Traditional models often leave some of the hardware unused because they have to wait for each token to finish before producing the next one.

A different way to generate text

Most language models work sequentially: they predict a token, add it to the text, and then calculate which one comes next. DiffusionGemma changes that process.

First, it creates a block with provisional tokens. It then reviews the entire block several times, keeps the parts that are correct, and modifies those that need adjustment. The process resembles a diffusion image generator, which starts with noise and gradually transforms it into a defined image.

The practical difference matters: the model can work with many tokens at the same time. According to Google, DiffusionGemma reaches more than 1,000 tokens per second on an NVIDIA H100 and more than 700 tokens per second on a GeForce RTX 5090, with an announced maximum of up to four times the speed of conventional models under specific conditions.

What you can do with it

DiffusionGemma is aimed at tasks where a fast response matters more than getting the highest possible quality in every sentence:

• Online text editing without noticeable delays.
• Real-time code autocompletion and generation.
• Filling in code or text fragments in the middle of a document.
• Creating non-linear structures, such as mathematical graphs or amino acid sequences.
• Local tools that need to respond quickly to a single person.

Its bidirectional attention allows each token to take the other tokens in the block into account. That is useful when the response is not built only from left to right. Sudoku is one example: to place a number correctly, it helps to consider what is happening in other parts of the board at the same time.

The model can also review its own output during the different passes. This allows it to identify and correct errors while it finishes building the block, instead of simply moving forward without looking back.

A large model that activates fewer parameters

DiffusionGemma has 26 billion parameters and uses a MoE, or mixture-of-experts, architecture. This technique divides the work among different internal groups and activates only the ones needed at each moment.

During inference, the model activates approximately 3.8 billion parameters. After quantization, a technique that reduces the numerical size of parameters to save memory, it can run within the 18 GB of VRAM available on some high-end consumer GPUs.

That makes it possible to try it on a desktop computer without always relying on a cloud service. Google has also optimized it for GeForce RTX 4090 and 5090 cards, as well as professional and enterprise systems with Hopper and Blackwell hardware.

Speed comes at a price

DiffusionGemma does not replace the standard Gemma 4 models. Google acknowledges that its overall quality is lower because it prioritizes parallel generation and speed.

It is also designed for local workloads or a small number of simultaneous requests. In a cloud service processing thousands of batched requests, sequential models can make better use of the hardware. In those cases, DiffusionGemma's advantage is reduced and its operation may be more expensive.

So if you need more precise and consistent responses for production, the recommendation is still to use Gemma 4. DiffusionGemma is a better fit for prototypes, interactive tools, and experiments where every millisecond counts.

Available to developers

Google is releasing the model weights under the Apache 2.0 license, a permissive license that allows developers to study, modify, and integrate the model into projects with few restrictions. Developers can try it through Hugging Face and use tools such as MLX, vLLM, and Transformers.

There are also options for adapting it to specific tasks through fine-tuning, meaning retraining it with specialized examples. Google mentions integrations with Unsloth, NVIDIA NeMo, and its own JAX-based tool called Hackable Diffusion. Official support for llama.cpp will arrive later.

For you, the most relevant change is not that all chatbots will immediately multiply their speed. It is that local generation is beginning to explore a different strategy: producing entire blocks, reviewing them, and making better use of the GPU. The next step will be to see how far those time savings can go without quality and operating costs suffering too much.