Qwen3-ASR-1.7B DFlash speculator

A DFlash draft model for speculative decoding with both Qwen3-ASR-1.7B-hf and Qwen3-ASR-1.7B.

Model configuration

Setting	Value
Verifier	`Qwen/Qwen3-ASR-1.7B-hf`
Draft transformer layers	5
Hidden size	2,048
Intermediate size	6,144
Draft vocabulary	32,000, with target-vocabulary mappings
Target auxiliary hidden-state layers	2, 14, 25
DFlash block size	8
Speculative tokens per proposal	7, excluding the anchor token
Weight precision	BF16

Use the verifier's tokenizer and audio processor.

Inference

Install the latest version of vLLM from source.

Load the verifier with the DFlash head:

from vllm import LLM

llm = LLM(
    model="Qwen/Qwen3-ASR-1.7B-hf",
    dtype="bfloat16",
    max_model_len=4096,
    max_num_seqs=8,
    max_num_batched_tokens=8192,
    gpu_memory_utilization=0.4,
    enforce_eager=True,
    enable_prefix_caching=False,
    mm_processor_cache_gb=0,
    limit_mm_per_prompt={"audio": 1},
    generation_config="vllm",
    seed=42,
    speculative_config={
        "model": "mattymchen/Qwen3-ASR-1.7B-speculator.dflash",
        "method": "dflash",
        "num_speculative_tokens": 7,
    },
)

Transcribe audio with example.py:

python example.py sample.wav --draft .

For longer recordings, increase the context, output, and batch token budgets:

python example.py recording.wav --draft . \
  --max-model-len 32768 --max-tokens 4096 --max-num-batched-tokens 32768

--max-model-len includes the audio prompt and generated text; larger budgets may require more GPU memory.

Use --draft mattymchen/Qwen3-ASR-1.7B-speculator.dflash to load the head from Hugging Face, or --verifier Qwen/Qwen3-ASR-1.7B to select the original verifier format.