Duyu-small

A Turkish + English speech recognizer for CPUs and edge devices: 19.3 M active parameters, shipped as ONNX graphs at int8 (56 MB) and 4-bit (34 MB) with a small torch-free engine (numpy + onnxruntime + sentencepiece). It writes whatever is spoken, Turkish, English or both in one sentence, cased and punctuated.

Duyu-small is built for on-device use: live, short, two-language speech, where the model runs next to the microphone. It is not meant for long-form transcription.

Architecture

part	what	stored params
encoder	9 Conformer blocks (d 256, 4 heads, conv k15) applied in 3 shared refinement passes with learned per-pass gates, 25 Hz output	15.5 M
sparse acoustic memory	product-key memory after block 4: 250k × 120 values, 4 heads × 16 reads per frame per pass	30.0 M (23k read per frame)
CTC head	2048 SentencePiece pieces (TR + EN), InterCTC + self-conditioning at block 4 of every pass	0.5 M
decoder	3-layer Transformer (d 256, QK-norm), joint CTC/attention decoding	3.8 M

Active per frame: 19.3 M. Stored: 49.3 M. Compute-equivalent of a 46.6 M single-pass model.

Accuracy

Word error rate (wer_trnum: case-, punctuation- and number-format-blind), PyTorch reference decode (attention beam 5 + CTC rescoring).

test set	Duyu-small (49.3 M)	whisper-base (74 M)	mihuai/turkish-stt (66 M)	balbal (45.4 M)	Cactus Whistle
voicedata-TR (conversation, 235 chunks)	12.45	29.51	19.89	—	—
FLEURS tr test (743)	7.87	24.19	9.62	7.95	—
sierra-tr (call-centre turns, 846, wer_sierra)	16.37	39.08	24.95	17.61	—
FLEURS en_us test (500)	22.88	10.81	Turkish only	Turkish only	14.37

whisper-base and Cactus Whistle decoded with the language forced to English on FLEURS-en and to Turkish on the Turkish sets; Duyu-small with <en> on FLEURS-en and <tr> elsewhere.

Quantization check on 62 held-out conversational dev clips (13.7 min, the engine on a laptop CPU with 4 threads, benchmark numerics):

engine	WER	real-time factor
PyTorch reference (GPU, bf16)	20.14	—
fp32, attention beam 5	19.56	0.11
int8, attention beam 5	19.56	0.07
4-bit, attention beam 5	19.67	0.13
int8, CTC greedy only	22.19	0.016

The engine's beam search is a port of the reference PyTorch decoder and is verified identical to it on 20/20 dev clips at matching numerics.

Files

path	size	purpose
int8/, q4/, fp32/	56 / 34 / 201 MB per set	one complete set of the four graphs per precision; DuyuSmall(quant=...) picks the folder
*/encoder_pre.onnx	1.3 MB	fbank → subsampler + style vector
*/encoder_pass.onnx	181 / 47.6 / 26.9 MB	one refinement pass over the shared blocks, incl. the memory (fp32 / uint8 / packed 4-bit); run 3×
*/encoder_post.onnx	2.1 / 0.5 / 0.3 MB	CTC head
*/decoder.onnx	17.4 / 6.4 / 5.0 MB	attention decoder (no KV cache)
passes.npz	—	pass embeddings + gates
fbank.npz, spm.model, engine.json, config.json	—	feature tables, tokenizer, special ids, model config
duyu_small/	—	the engine (Python package)

int8 = onnxruntime dynamic quantization of every MatMul + uint8 memory table with fp16 per-row scales. q4 = 4-bit block-wise (32) MatMul weights + the memory table packed two values per byte, dequantized after the gather with plain ONNX ops (Mod / Div / Reshape), so it runs on stock onnxruntime.

Use

pip install -e .            # or: pip install numpy onnxruntime sentencepiece
duyu-small recording.m4a                    # int8, attention beam 5, writes whatever is spoken
duyu-small a.wav --quant q4 --lang tr       # 4-bit, force Turkish
duyu-small a.wav --mode ctc                 # CTC greedy only (partials / fastest)

from duyu_small import DuyuSmall
m = DuyuSmall()                             # int8 by default; quant="q4" | "fp32"
text = m.transcribe(wav16k)                 # float32 numpy, any length (chunked above 30 s)
text = m.transcribe(wav16k, lang="en", beam=1)

DuyuSmall() finds the model files next to the package (this repository's layout); from elsewhere pass the directory, DuyuSmall("/path/to/Duyu-small"), or set DUYU_SMALL_DIR.

Audio in: 16 kHz mono float32 (duyu_small.audio.load_audio reads anything ffmpeg reads). Languages: tr, en, mix (default), auto.

Limits

• Full-context encoder: utterances are decoded whole (chunked at 25 s on long input); no streaming partials beyond the CTC head yet.
• The decoder re-runs the prefix at every step (no KV cache): fine for conversation turns, not tuned for long dictation.
• English is "understands English in a Turkish conversation" (FLEURS-en 22.9), not Whistle-class English.
• Read speech with rare proper nouns is where it is weakest.

License

Apache License 2.0: model weights, ONNX graphs and the engine.

Credits

The encoder design (a Conformer applied in 3 shared refinement passes with a sparse acoustic memory of 250k × 120 values, read by 4 heads × 16 records per pass) is taken from balbal, described by its author at erayy.com/balbal-blog. Duyu-small adds the attention decoder and the bilingual Turkish + English vocabulary on top.