--- base_model: Qwen/Qwen3-8B library_name: vllm license: apache-2.0 pipeline_tag: text-generation tags: - speculative-decoding - dcoco - dflash - draft-model --- # DCOCO drafter for Qwen/Qwen3-8B This repository contains the **DCOCO** (DFlash Contextual Corrector) speculative-decoding drafter for **[Qwen/Qwen3-8B](https://huggingface.co/Qwen/Qwen3-8B)**, from the paper *Contextual Correction of Parallel Drafts for Speculative Decoding*. It is a draft model: serve it together with `Qwen/Qwen3-8B`, which verifies every token, so the output distribution is exactly the target model's. **Paper:** [Contextual Correction of Parallel Drafts for Speculative Decoding](https://huggingface.co/papers/2610.12570) **Code:** [github.com/two-delta/DCOCO](https://github.com/two-delta/DCOCO)

DCOCO overview: (a) parallel correction scores every adjacent candidate pair in one batched pass; (b) prefix-first selection walks the scores slot by slot

## How it works Diffusion-style drafters such as DFlash propose a whole block of tokens in a single pass, but each position is predicted without seeing which tokens were chosen before it. DCOCO keeps the top-16 candidates at every drafted position and scores how well each candidate fits each candidate at the next position, using candidate representations and a pair state over adjacent slots, all in one small batched pass. A prefix-first walk over those scores then picks the draft. This restores intra-block dependencies at a cost of a few percent of the drafting cycle. ## Model details | | | |---|---| | Target (verifier) model | `Qwen/Qwen3-8B` | | Architecture | `DCOCODraftModel`: 5-layer DFlash drafter (hidden size 4096, bidirectional in-block attention) plus the DCOCO corrector | | Corrector | K = 16 candidates per slot, candidate width 128, pair-state width 128, 2-layer scoring MLP (hidden 256); 2,229,505 parameters | | Draft block | 15 drafted tokens per round (anchor position predicts); up to 16 tokens committed per verification | | Target layers read | 1, 9, 17, 25, 33 | | Training data | ~800K prompts from the stem, math and code splits of Nemotron Post-Training Dataset v2 and CodeAlpaca-20k, with continuations regenerated by `Qwen/Qwen3-8B` (greedy, thinking off, up to 4,096 tokens) | | Training | 6 epochs of DFlash drafter training, then 6 epochs of joint training with the corrector; AdamW, lr 6e-4, cosine schedule | | Format | BF16 `safetensors` (4.59 GB), vLLM-native config | ## Usage DCOCO runs in vLLM, through the speculative-decoding `dflash` method; vLLM selects the corrector from the checkpoint's `DCOCODraftModel` architecture. This requires a vLLM build with DCOCO support, such as the DCOCO image (`vllm/vllm-openai:v0.31.0` with the DCOCO overlay). ```bash vllm serve Qwen/Qwen3-8B \ --speculative-config '{"method": "dflash", "model": "TD-Labs/Qwen3-8B-DCOCO", "num_speculative_tokens": 15}' ``` Check that the server log shows `Resolved architecture: DCOCODraftModel`. With any other architecture, vLLM serves the drafter as plain DFlash without the corrector. Then query it like any OpenAI-compatible server: ```bash curl localhost:8000/v1/chat/completions -H 'content-type: application/json' -d '{ "model": "Qwen/Qwen3-8B", "messages": [{"role": "user", "content": "How many positive whole-number divisors does 196 have?"}], "chat_template_kwargs": {"enable_thinking": false} }' ``` `num_speculative_tokens` can be lowered (down to 1) but not raised above 15. ## Results All results below are from the paper (Tables 1, 7, 8, 9 and 19). Setup: one NVIDIA B200 GPU, vLLM 0.27, concurrency 1 unless noted, thinking off with at most 2,048 generated tokens (4,096 with thinking on), 128 prompts per benchmark (all 30 of AIME-25, all 80 of MT-Bench, 880 of SPEED-Bench). Throughput (tok/s) is the median per-request rate (output tokens / request latency, including prefill); speedup is relative to the target model alone. τ is the mean accepted length per verification round, including the bonus token. T=0 is a single greedy run; T=1 is the mean over seeds 42, 43 and 44. ### Per-benchmark results (thinking off) Greedy decoding (T=0) in the first four columns, sampling (T=1) in the last four. | Benchmark | No-spec tok/s | DCOCO tok/s | Speedup | τ | No-spec tok/s | DCOCO tok/s | Speedup | τ | |---|---:|---:|---:|---:|---:|---:|---:|---:| | MATH-500 | 265.2 | 1625.2 | 6.13× | 8.039 | 264.5 | 1145.5 | 4.33× | 5.755 | | AIME-25 | 263.7 | 1436.8 | 5.45× | 7.263 | 263.6 | 828.7 | 3.14× | 4.486 | | GSM8K | 267.0 | 1175.8 | 4.40× | 6.118 | 266.4 | 1081.3 | 4.06× | 5.440 | | HumanEval | 266.6 | 1075.6 | 4.03× | 5.443 | 266.4 | 910.0 | 3.42× | 4.646 | | MBPP | 267.0 | 979.1 | 3.67× | 5.215 | 266.6 | 848.3 | 3.18× | 4.428 | | LiveCodeBench | 264.2 | 1328.5 | 5.03× | 6.982 | 265.3 | 1013.7 | 3.82× | 5.431 | | MT-Bench | 266.0 | 494.7 | 1.86× | 2.930 | 265.3 | 450.8 | 1.70× | 2.581 | | Arena-Hard-v2 | 264.1 | 486.9 | 1.84× | 2.736 | 264.2 | 436.8 | 1.65× | 2.286 | | Alpaca | 265.9 | 496.1 | 1.87× | 2.560 | 265.5 | 447.5 | 1.69× | 2.313 | | SPEED-Bench | 266.5 | 498.4 | 1.87× | 2.561 | 266.2 | 459.5 | 1.73× | 2.373 | | **Average** | 265.6 | **959.7** | **3.62×** | **4.985** | 265.4 | **762.2** | **2.87×** | **3.974** | ### Comparison with other parallel drafters (average over the ten benchmarks) Thinking off: | Method | tok/s (T=0) | Speedup (T=0) | τ (T=0) | tok/s (T=1) | Speedup (T=1) | τ (T=1) | |---|---:|---:|---:|---:|---:|---:| | DFlash | 783.1 | 2.95× | 3.913 | 649.8 | 2.45× | 3.259 | | Domino | 839.3 | 3.16× | 4.613 | 670.9 | 2.53× | 3.697 | | DSpark | 765.6 | 2.88× | 4.254 | 628.7 | 2.37× | 3.494 | | **DCOCO** | **959.7** | **3.62×** | **4.985** | **762.2** | **2.87×** | **3.974** | Thinking on (extended reasoning): | Method | tok/s (T=0) | Speedup (T=0) | τ (T=0) | tok/s (T=1) | Speedup (T=1) | τ (T=1) | |---|---:|---:|---:|---:|---:|---:| | DFlash | 510.0 | 1.95× | 2.501 | 465.2 | 1.78× | 2.274 | | Domino | 516.1 | 1.97× | 2.772 | 469.5 | 1.79× | 2.502 | | DSpark | 480.6 | 1.83× | 2.595 | 438.9 | 1.68× | 2.364 | | **DCOCO** | **592.5** | **2.26×** | **2.999** | **530.4** | **2.03×** | **2.677** | ### Throughput under concurrent requests (thinking off) Aggregate output tokens/s, averaged over the ten benchmarks. | Concurrent requests | DFlash | Domino | DSpark | DCOCO | DCOCO vs. best other | |---:|---:|---:|---:|---:|---:| | 1 (T=0) | 783.1 | 839.3 | 765.6 | **959.7** | +14.3% | | 4 (T=0) | 2806.4 | 3008.6 | 2719.0 | **3507.3** | +16.6% | | 8 (T=0) | 4889.1 | 5051.0 | 4614.4 | **6082.6** | +20.4% | | 16 (T=0) | 7229.1 | 7098.4 | 7092.9 | **8894.3** | +23.0% | | 32 (T=0) | 9655.0 | 9013.9 | 9490.3 | **11512.4** | +19.2% | | 1 (T=1) | 649.8 | 670.9 | 628.7 | **762.2** | +13.6% | | 4 (T=1) | 2300.3 | 2334.1 | 2198.9 | **2710.0** | +16.1% | | 8 (T=1) | 4006.3 | 3894.9 | 3758.1 | **4735.3** | +18.2% | | 16 (T=1) | 5964.5 | 5441.1 | 5730.3 | **6871.9** | +15.2% | | 32 (T=1) | 7857.9 | 6821.8 | 7443.4 | **8836.1** | +12.4% | ## Limitations - The drafter is trained for `Qwen/Qwen3-8B` and must be served with exactly that target model. - It needs the target's full vocabulary (no reduced draft vocabulary). - It was trained on thinking-off generations; the speedup is lower with extended thinking (see above). - The speedup depends on the task: highest on math and code, lower on open-ended chat. ## Citation ```bibtex @misc{eisenstadt2026contextualcorrectionparalleldrafts, title={Contextual Correction of Parallel Drafts for Speculative Decoding}, author={Roy Eisenstadt and Harel Lidar and Tal Gonen and Omer Luxembourg and Itamar Zimerman and Lior Wolf and Eli David and Ilan Naiman}, year={2026}, eprint={2610.12570}, archivePrefix={arXiv}, primaryClass={cs.CL}, url={https://arxiv.org/abs/2610.12570}, } ```