Google’s new open model DiffusionGemma generates text from noise instead of word by word

Google released DiffusionGemma, a 26-billion-parameter model that generates text not token by token but through diffusion, similar to how image AI turns noise into a picture. According to Nvidia, it hits about 1,000 tokens per second on a single H100 GPU, roughly four times faster than comparable autoregressive models.

Nvidia handled the optimization. Most language models generate one token after another, basing each new token on the previous one. DiffusionGemma takes a different approach. It starts with a block of 256 random placeholder tokens and refines them across several passes until readable text emerges. The idea comes from image AI, where diffusion models turn noise into clear images. The model has 26 billion parameters total but only activates 3.8 billion per step. That’s thanks to a mixture-of-experts architecture, where several specialized sub-networks sit side by side and only the right ones fire depending on the input. When quantized to lower precision, the model fits into 18 GB of VRAM on high-end consumer GPUs, according to Google. It builds on the Gemma 4 family and borrows its diffusion process from Google’s earlier research on Gemini Diffusion.Ad Nvidia says the speed advantage comes down to hardware usage. With autoregressive models, single-user inference is often bottlenecked by memory bandwidth. The GPU’s compute units sit idle most of the time, just waiting for data from memory. Engineers call this memory-bound. DiffusionGemma sidesteps the problem by processing up to 256 tokens in parallel, pushing the bottleneck toward raw compute instead. The result is that GPUs actually stay busy.AdDEC_D_Incontent-1 Nvidia reports about 1,000 tokens per second on an H100 when processing a single request, 150 tokens per second on the DGX Spark deskside system, and up to 2,000 tokens per second on the DGX Station. On the GeForce RTX 5090, Google claims more than 700 tokens per second.