Google’s DiffusionGemma proves you don’t need to train from scratch to build a text diffusion model

Instead of training a new model from scratch, Google DeepMind retrofitted Gemma 4 into a diffusion model. The newly published report explains how it works and where the tradeoffs are.

Google DeepMind released DiffusionGemma as a model in mid-June and has now followed up with the technical report. Unlike standard language models that generate text one token at a time, DiffusionGemma refines blocks of 256 tokens in parallel, similar to how image AIs pull a picture out of noise. On an Nvidia H100 accelerator, the model hits about 1,500 tokens per second. Building a new model from scratch wasn’t necessary. The team started with the existing Gemma-4-26B-A4B and converted it into a diffusion model using less than ten percent of the original training token budget, according to the report. In the first of two steps, the model learns to reconstruct noisy text blocks from example data. A combined phase of reinforcement learning and sampler distillation follows, which Google calls SD·RL. Reinforcement learning typically boosts answer quality, while sampler distillation lets the model get by with fewer compute steps. Google merges both into a single process. According to the report, this combined approach raises quality on reasoning benchmarks by an average of ten points while nearly quadrupling the number of tokens per compute step. As a side effect, DiffusionGemma’s answers run about 50 percent shorter, which further boosts speed. Bidirectional reasoning lets the model correct itself Standard language models have to commit to the first digit of an answer before they’ve worked through the reasoning. In a math problem from the report, Gemma 4 starts its response with “-1,” realizes during its derivation that “-25” is correct, and tacks on a correction afterward. DiffusionGemma develops the answer and reasoning in parallel, so it can fix mistakes before the output is finalized. Sudoku solving works on the same principle, since every entry depends on entries that come later.