TD-Labs/Qwen3-8B-DCOCO

DCOCO drafter for Qwen/Qwen3-8B

This repository contains the DCOCO (DFlash Contextual Corrector) speculative-decoding drafter for Qwen/Qwen3-8B, from the paper Contextual Correction of Parallel Drafts for Speculative Decoding. It is a draft model: serve it together with Qwen/Qwen3-8B, which verifies every token, so the output distribution is exactly the target model's.

Paper: Contextual Correction of Parallel Drafts for Speculative Decoding
Code: github.com/two-delta/DCOCO

How it works

Diffusion-style drafters such as DFlash propose a whole block of tokens in a single pass, but each position is predicted without seeing which tokens were chosen before it. DCOCO keeps the top-16 candidates at every drafted position and scores how well each candidate fits each candidate at the next position, using candidate representations and a pair state over adjacent slots, all in one small batched pass. A prefix-first walk over those scores then picks the draft. This restores intra-block dependencies at a cost of a few percent of the drafting cycle.

Model details

Target (verifier) model	Qwen/Qwen3-8B
Architecture	DCOCODraftModel: 5-layer DFlash drafter (hidden size 4096, bidirectional in-block attention) plus the DCOCO corrector
Corrector	K = 16 candidates per slot, candidate width 128, pair-state width 128, 2-layer scoring MLP (hidden 256); 2,229,505 parameters
Draft block	15 drafted tokens per round (anchor position predicts); up to 16 tokens committed per verification
Target layers read	1, 9, 17, 25, 33
Training data	~800K prompts from the stem, math and code splits of Nemotron Post-Training Dataset v2 and CodeAlpaca-20k, with continuations regenerated by Qwen/Qwen3-8B (greedy, thinking off, up to 4,096 tokens)
Training	6 epochs of DFlash drafter training, then 6 epochs of joint training with the corrector; AdamW, lr 6e-4, cosine schedule
Format	BF16 safetensors (4.59 GB), vLLM-native config

Usage

DCOCO runs in vLLM, through the speculative-decoding dflash method; vLLM selects the corrector from the checkpoint's DCOCODraftModel architecture. This requires a vLLM build with DCOCO support, such as the DCOCO image (vllm/vllm-openai:v0.31.0 with the DCOCO overlay).

vllm serve Qwen/Qwen3-8B \
  --speculative-config '{"method": "dflash", "model": "TD-Labs/Qwen3-8B-DCOCO", "num_speculative_tokens": 15}'

Check that the server log shows Resolved architecture: DCOCODraftModel. With any other architecture, vLLM serves the drafter as plain DFlash without the corrector. Then query it like any OpenAI-compatible server:

curl localhost:8000/v1/chat/completions -H 'content-type: application/json' -d '{
  "model": "Qwen/Qwen3-8B",
  "messages": [{"role": "user", "content": "How many positive whole-number divisors does 196 have?"}],
  "chat_template_kwargs": {"enable_thinking": false}
}'

num_speculative_tokens can be lowered (down to 1) but not raised above 15.

Results

All results below are from the paper (Tables 1, 7, 8, 9 and 19). Setup: one NVIDIA B200 GPU, vLLM 0.27, concurrency 1 unless noted, thinking off with at most 2,048 generated tokens (4,096 with thinking on), 128 prompts per benchmark (all 30 of AIME-25, all 80 of MT-Bench, 880 of SPEED-Bench). Throughput (tok/s) is the median per-request rate (output tokens / request latency, including prefill); speedup is relative to the target model alone. τ is the mean accepted length per verification round, including the bonus token. T=0 is a single greedy run; T=1 is the mean over seeds 42, 43 and 44.

Per-benchmark results (thinking off)

Greedy decoding (T=0) in the first four columns, sampling (T=1) in the last four.

Benchmark	No-spec tok/s	DCOCO tok/s	Speedup	τ	No-spec tok/s	DCOCO tok/s	Speedup	τ
MATH-500	265.2	1625.2	6.13×	8.039	264.5	1145.5	4.33×	5.755
AIME-25	263.7	1436.8	5.45×	7.263	263.6	828.7	3.14×	4.486
GSM8K	267.0	1175.8	4.40×	6.118	266.4	1081.3	4.06×	5.440
HumanEval	266.6	1075.6	4.03×	5.443	266.4	910.0	3.42×	4.646
MBPP	267.0	979.1	3.67×	5.215	266.6	848.3	3.18×	4.428
LiveCodeBench	264.2	1328.5	5.03×	6.982	265.3	1013.7	3.82×	5.431
MT-Bench	266.0	494.7	1.86×	2.930	265.3	450.8	1.70×	2.581
Arena-Hard-v2	264.1	486.9	1.84×	2.736	264.2	436.8	1.65×	2.286
Alpaca	265.9	496.1	1.87×	2.560	265.5	447.5	1.69×	2.313
SPEED-Bench	266.5	498.4	1.87×	2.561	266.2	459.5	1.73×	2.373
Average	265.6	959.7	3.62×	4.985	265.4	762.2	2.87×	3.974

Comparison with other parallel drafters (average over the ten benchmarks)

Thinking off:

Method	tok/s (T=0)	Speedup (T=0)	τ (T=0)	tok/s (T=1)	Speedup (T=1)	τ (T=1)
DFlash	783.1	2.95×	3.913	649.8	2.45×	3.259
Domino	839.3	3.16×	4.613	670.9	2.53×	3.697
DSpark	765.6	2.88×	4.254	628.7	2.37×	3.494
DCOCO	959.7	3.62×	4.985	762.2	2.87×	3.974

Thinking on (extended reasoning):

Method	tok/s (T=0)	Speedup (T=0)	τ (T=0)	tok/s (T=1)	Speedup (T=1)	τ (T=1)
DFlash	510.0	1.95×	2.501	465.2	1.78×	2.274
Domino	516.1	1.97×	2.772	469.5	1.79×	2.502
DSpark	480.6	1.83×	2.595	438.9	1.68×	2.364
DCOCO	592.5	2.26×	2.999	530.4	2.03×	2.677

Throughput under concurrent requests (thinking off)

Aggregate output tokens/s, averaged over the ten benchmarks.

Concurrent requests	DFlash	Domino	DSpark	DCOCO	DCOCO vs. best other
1 (T=0)	783.1	839.3	765.6	959.7	+14.3%
4 (T=0)	2806.4	3008.6	2719.0	3507.3	+16.6%
8 (T=0)	4889.1	5051.0	4614.4	6082.6	+20.4%
16 (T=0)	7229.1	7098.4	7092.9	8894.3	+23.0%
32 (T=0)	9655.0	9013.9	9490.3	11512.4	+19.2%
1 (T=1)	649.8	670.9	628.7	762.2	+13.6%
4 (T=1)	2300.3	2334.1	2198.9	2710.0	+16.1%
8 (T=1)	4006.3	3894.9	3758.1	4735.3	+18.2%
16 (T=1)	5964.5	5441.1	5730.3	6871.9	+15.2%
32 (T=1)	7857.9	6821.8	7443.4	8836.1	+12.4%

Limitations

• The drafter is trained for Qwen/Qwen3-8B and must be served with exactly that target model.
• It needs the target's full vocabulary (no reduced draft vocabulary).
• It was trained on thinking-off generations; the speedup is lower with extended thinking (see above).
• The speedup depends on the task: highest on math and code, lower on open-ended chat.

Citation

@misc{eisenstadt2026contextualcorrectionparalleldrafts,
      title={Contextual Correction of Parallel Drafts for Speculative Decoding}, 
      author={Roy Eisenstadt and Harel Lidar and Tal Gonen and Omer Luxembourg and Itamar Zimerman and Lior Wolf and Eli David and Ilan Naiman},
      year={2026},
      eprint={2610.12570},
      archivePrefix={arXiv},
      primaryClass={cs.CL},
      url={https://arxiv.org/abs/2610.12570}, 
}