CakeByVPBank/piper-pgl-v4-vi_VN-version39_epoch39

Piper v3 — Multi-Speaker Vietnamese TTS (5 speakers)

Production-grade multi-speaker Vietnamese text-to-speech built on VITS + Piper, trained on 260 189 ASR-verified synthetic utterances (257 hours) across 5 speakers.

Successor to piper-v2-vietnamese-4speakers. The dataset was rebuilt end-to-end: every clip is distilled from a voice-cloning teacher and then verified by ASR round-trip before entering training — no clip reaches the model unless an independent recognizer reads back what it was asked to say.

• Sample rate: 22 050 Hz
• Phoneme backend: eSpeak-NG (vi)
• Speakers: 5 (4 female, 1 male)
• Format: ONNX (Piper config v1.3.0)
• Runtime: CPU / edge devices

Architecture

Base VITS (~77 M params) with the same two upgrades as v2:

Change	Rationale
Pre-LayerNorm in TextEncoder	Reduces gradient instability vs. post-norm
Snake activation x + (1/α)·sin²(αx) in Generator ResBlocks	Learnable harmonic bias; mirrors BigVGAN's improvement over HiFi-GAN for periodic/tonal speech

Both modifications are baked into the exported ONNX graph, so standard piper-tts / onnxruntime clients run inference without a custom build. Re-training requires the upstream fork: OHF-Voice/piper1-gpl.

The speaker embedding table emb_g was widened from (4, 512) to (5, 512) to admit the fifth voice — existing rows preserved, the new row initialised near the mean of the trained rows rather than from N(0,1), which would otherwise spend many epochs merely returning to the valid manifold.

Speakers & Samples

Speaker 0  femalenorth-01  (also default) — Ngọc Lan
Region North Gender Female Use case General	Sample Xin chào, đây là giọng Ngọc Lan, nữ miền Bắc.

Speaker 1  femalenorth-02 — Minh Anh
Region North Gender Female Use case Collections (firm register)	Sample Xin chào, đây là giọng Minh Anh, nữ miền Bắc.

Speaker 2  malenorth-01 — Quang Huy
Region North Gender Male Use case General (light tone)	Sample Xin chào, đây là giọng Quang Huy, nam miền Bắc.

Speaker 3  femalenorth-03 — Thu Hà
Region North Gender Female Use case Customer support	Sample Xin chào, đây là giọng Thu Hà, nữ miền Bắc.

Speaker 4  femalesouth-01 — Yến Nhi  (new in v3)
Region South Gender Female Use case Marketing	Sample Xin chào, đây là giọng Yến Nhi, nữ miền Nam.

Data

Every training clip is synthetic, distilled from a voice-cloning teacher, and ASR-verified. Two teachers were used, one per voice — never mixed within a single speaker, so each embedding row learns one consistent identity.

Speaker	Teacher	Clips	Hours
0 Ngọc Lan	OmniVoice	54 136	56.8
1 Minh Anh	VoxCPM2 (cfg 3.0, 10 steps)	52 774	46.1
2 Quang Huy	OmniVoice	52 457	47.3
3 Thu Hà	VoxCPM2 (cfg 3.0, 10 steps)	50 942	45.7
4 Yến Nhi	VoxCPM2 (cfg 3.0, 10 steps)	49 880	61.2
Total (train)		260 189	257.1

Plus 1 507 validation and 817 held-out test utterances. Speaker balance spread is 1.64 percentage points. Splits are deterministic on blake2b(text_id), so the held-out set is identical across model versions and directly comparable.

Verification gates

Clips are rejected before training, not after:

Gate	What it catches
ASR round-trip (SER)	Teacher swallowed or mangled syllables
Deleted-run length	Whole syllable groups dropped — invisible to mean SER
Duration per syllable	Dragging or rushed reads
Dynamic range, clipping, DC offset, edge/internal silence	Vocoder artefacts
Speaker-separation gate	Clips whose speaker embedding drifted from the voice's own centroid, or leaked toward another voice

Overall accept rate was ~80 %, plus a 4-attempt retry ladder on borderline clips. The speaker-separation gate alone removed 8 168 clips that passed every text-level check — ASR cannot hear identity drift.

Training

Property	Value
Config	vi_5spk_synth_v6
Warm start	v2-lineage 4-speaker checkpoint, emb_g widened 4 → 5
Preflight	827 / 827 tensors loaded, 0 shape mismatch
Batch size	128, bucketed by spectrogram length (10 buckets)
LR schedule	ExponentialLR, γ = 0.9999915 (G) / 0.9999944 (D) — verified against TensorBoard, not assumed
Shipped checkpoint	Epoch 16 (val_loss 31.042)
Audio preprocessing	VAD + silence trim; no re-trim at train time (clips already trimmed)

length_scale is non-linear on this architecture: roughly 49.8 % of tokens are separators pinned to a single frame by ceil(), so the parameter only "takes" about 0.39 of its nominal effect. Measured on the shipped model: 0.1 → 0.51× duration, 2.0 → 1.85×, 10.0 → 7.93×.

Evaluation

Measured on the shipped ONNX (not the torch checkpoint), 1 137 held-out clips:

Metric	Value
SER (ASR round-trip)	0.0198
Clips with SER = 0	78.2 %
Deleted run ≥ 3 syllables	0.35 %
Tail truncation	0.35 %
Head truncation	0.00 %

Export parity — torch vs. ONNX

Export can silently distort the model; SDP and the non-standard Snake activation are the usual culprits. Parity is measured on the distribution of SER over 500 sentences, because VITS samples stochastically and the two sides never produce identical audio.

	SER
torch	0.01541
ONNX	0.01935
|Δ|	0.00395 (threshold 0.005)
Noise floor (same ONNX, two runs)	0.00238

Passes, though with thinner margin than v2's export — Δ is 1.7× the noise floor.

Speaker identity

Cosine similarity between each speaker's generated audio and the centroid of its own training data, and distance to the nearest other voice:

Check	Value	Threshold
Model matches the voice it was taught	0.988 – 0.995	≥ 0.75
Closest speaker pair (Thu Hà ↔ Minh Anh)	0.149	≥ 0.10
Yến Nhi (new) to nearest other voice	0.344	≥ 0.10

The new southern voice sits well clear of the three northern female voices, which naturally cluster closest together.

Catastrophic forgetting

Widening emb_g and retraining risks damaging voices whose data did not change. Measured against the source checkpoint on the identical held-out set, with the sampling seed pinned so only weights differ:

Epoch	Old voices — SER	Cosine to source
0	0.30 – 0.35	0.86 – 0.91
1	0.02 – 0.05	0.96 – 0.97

The epoch-0 dip is the optimizer restarting with zeroed moments at full learning rate, not forgetting — it clears within one epoch. Do not judge a warm-started checkpoint before epoch 2.

UTMOSv2 naturalness scores are omitted: the metric's weights are unreachable from our training environment, so no honest number can be reported here.

Quick start

Option A — Piper Python package (recommended)

pip install piper-tts huggingface_hub

from huggingface_hub import hf_hub_download
from piper import PiperVoice
import wave

repo = "CakeByVPBank/piper-v3-vietnamese-5speakers"
onnx = hf_hub_download(repo, "vi_VN-csa-voice-piper-v3-medium.onnx")
hf_hub_download(repo, "vi_VN-csa-voice-piper-v3-medium.onnx.json")  # must sit next to .onnx

voice = PiperVoice.load(onnx)

with wave.open("out.wav", "wb") as wav:
    voice.synthesize(
        "Xin chào, đây là giọng đọc tiếng Việt.",
        wav,
        speaker_id=0,        # 0..4
        length_scale=1.0,    # >1 slower, <1 faster
        noise_scale=0.667,
        noise_w=0.8,
    )

Option B — Piper CLI

pip install piper-tts pathvalidate "huggingface_hub[cli]"

hf download CakeByVPBank/piper-pgl-v4-vi_VN-version39_epoch39 \
  vi_VN-csa-voice-piper-v3-medium.onnx vi_VN-csa-voice-piper-v3-medium.onnx.json \
  --local-dir ./models

echo "Xin chào" | piper \
  --model ./models/vi_VN-csa-voice-piper-v3-medium.onnx \
  --speaker 0 \
  --output_file out.wav

Option C — onnxruntime directly

Phonemize with espeak-ng -v vi -q --ipa=3, map IPA characters via phoneme_id_map in the .onnx.json, then run the session with inputs input, input_lengths, scales ([noise_scale, length_scale, noise_w]), and sid.

Recommended inference parameters

Speaker	length_scale
0 Ngọc Lan	1.0
1 Minh Anh	1.2
2 Quang Huy	1.2
3 Thu Hà	1.2
4 Yến Nhi	1.0

noise_scale 0.667 and noise_w 0.8 for all speakers. Per-speaker length_scale differs because the reference recordings differ in pace — treat these as defaults, not limits.

Text normalization matters

The model was trained on labels produced by a specific Vietnamese normalizer. Feeding it raw text without that normalizer creates train/serve skew: numbers, dates, currency, and brand names will be phonemized as literal characters.

"Cake by VPBank"    → "cách bai vi pi banh"
"ngày 15 tháng 3"   → "ngày mười lăm tháng ba"
"8.560.000 đồng"    → "tám triệu năm trăm sáu mươi nghìn đồng"
"OTP"               → "ô tê pê"

The reference implementation ships in csa-voice.

Production server

FastAPI server with streaming output, the Vietnamese normalizer above, per-voice parameter defaults, and an OpenAI-compatible /v1/audio/speech endpoint: csa-voice.

Limitations

• Vietnamese only; English text falls back to eSpeak phonemes and sounds accented.
• No voice cloning — speaker selection is fixed to the 5 trained speakers.
• Entirely synthetic training data. Quality is bounded by the teacher models; prosodic patterns absent from the synthetic distribution may degrade.
• Speaker 4's reference recording is band-limited (most energy below ~1.3 kHz), so that voice is duller than the others. This is a property of the source audio, not of the model.
• speaker_id outside 0–4 is not validated by the ONNX graph; clamp it upstream.

License & attribution

Weights released under the MIT License, inheriting from the upstream rhasspy/piper-voices lineage. When redistributing, please credit:

• Piper TTS by Michael Hansen
• rhasspy/piper-voices
• OHF-Voice/piper1-gpl (training fork)

Citation

@software{piper_tts,
  author = {Hansen, Michael},
  title  = {Piper: A fast, local neural text-to-speech system},
  url    = {https://github.com/rhasspy/piper}
}

@software{kim2021conditional,
  author = {Kim, Jaehyeon and Kong, Jungil and Son, Juhee},
  title  = {Conditional Variational Autoencoder with Adversarial Learning for End-to-End Text-to-Speech (VITS)},
  year   = {2021},
  url    = {https://github.com/jaywalnut310/vits}
}