Diffusion-draft speculative decoding: an eight-prompt probe

A diffusion model drafts, an autoregressive model verifies. Acceptance measured on eight prompts.

Speculative decoding pairs a draft model with a verifier. The draft proposes several tokens; the verifier checks them in one forward pass and keeps the longest prefix it would have produced itself, so the output is identical to the verifier run alone. We measured acceptance when the draft is a diffusion model (DiffusionGemma) and the verifier is autoregressive (Gemma), with no training. Acceptance is the number of draft tokens kept per block — the proxy for how much verifier work is skipped.

Scope. Eight prompts, greedy decoding, one draft-block length, one run on a cloud GPU (~$3). No wall-clock latency was measured. This is too small to generalize from; it is a first measurement, reported as-is.

Results

Each row is one prompt. “Accepted” is the count of draft tokens the verifier kept before its next choice diverged, out of the draft block length.

Prompt	Type	Accepted
”Translate ‘good morning’ into Spanish…“	formulaic	37 / 37
”List three fruits as JSON”	structured	13 / 33
”The capital of France is…“	factual	8 / 9
”Write a Python function that returns…“	semi-open	5 / 48
”What is 17 multiplied by 23? Show your work”	reasoning	1 / 26
”Summarize the plot of Romeo and Juliet”	open	1 / 48
”Explain why the sky is blue”	open	0 / 42
”Give me a haiku about the ocean”	creative	0 / 19

Mean accepted: 8.1 tokens per draft block.

Acceptance was high on formulaic, structured, and factual prompts (37, 13, 8) and near zero on open and creative prompts (0–1).

Method

Verification was exact-greedy: a draft token is kept only if it equals the token the verifier would have produced. Where the next token is nearly determined — "fruits": [ "apple", — the draft and verifier agree over long stretches and many tokens are accepted at once. Where several continuations are equally valid, the draft and verifier pick different fluent options (“Two young lovers…” vs “Two star-crossed lovers…”), which exact matching counts as a miss from the first content word. The draft token is a synonym, not an error, but exact-greedy verification does not admit synonyms. Acceptance therefore tracks how constrained the next token is, not whether the draft is correct.

Setup

draft: DiffusionGemma · verifier: Gemma (exact-greedy, lossless)
no training · vLLM · bf16 / int8 weights · one cloud GPU, single run (~$3)
metric: tokens accepted per draft block

Limitations

Eight prompts; one model pair; greedy only; one draft-block length; cloud GPU; acceptance measured, latency not. Relaxed (sampling-based) verification that admits synonyms was not tested and would likely change the open-text numbers. These results are a probe, not a benchmark.