shi3z/VoxCPM2-q8

VoxCPM2 — INT8 (Q8) checkpoint for in-browser WebGPU inference

An INT8-quantized repack of openbmb/VoxCPM2, prepared so the model can be loaded and run entirely inside a web browser by shi3z/voxcpm2web (a WASM + WebGPU port of mii-nipah/voxcpm-rs).

Nothing about the model architecture was changed. This repository exists to solve two concrete browser constraints:

1. wasm32 has a 4 GB address space. The upstream checkpoint is 4.27 GiB of weights plus a 359 MB audiovae.pth.
2. Quantizing during load costs VRAM you may not have. Doing it ahead of time means the browser never materializes the full-precision tensors at all.

What is in here

File	Size	Notes
model.safetensors	2.45 GiB	the whole model, including the AudioVAE
config.json	4.3 KB	copied from upstream
tokenizer.json	3.6 MB	copied from upstream

There is no separate audiovae file: the AudioVAE weights are already merged in, so a browser needs exactly one large download instead of two.

Measured against the upstream checkpoint

Both rows produce applied=632 missing=0 unused=0 errors=0.

	upstream	this checkpoint
download	4.27 GiB + 359 MB	2.45 GiB
load, native (RTX 3090)	30–35 s	7.4 s
load, in browser	35.7 s	9.1 s
weights resident on the GPU	9.09 GB	2.90 GB
peak GPU memory, generation included	—	5.22 GiB
peak WASM heap	2.0 GB	1.4 GB

The 5.22 GiB figure is why this checkpoint exists: it is what makes an 8 GB GPU enough. Quantizing on the fly from the upstream checkpoint has to hold full-precision tensors while it works, which does not fit in 8 GB.

Format

Ordinary safetensors. Quantized tensors are stored as flat U8 blobs and are described by a voxcpm_quant entry in the file's __metadata__:

{
  "version": 1,
  "scheme": "...",
  "quantized": { "<tensor name>": [<shape>], "...": [] }
}

Each blob is Burn's QuantValue::Q8S packed representation — per-tensor scales appended after the packed values, the exact layout burn's q_from_data consumes. Weights that cannot be packed (anything whose last dimension is not a multiple of 4, and tensors under 65536 elements) stay in floating point, stored as F16.

Because the file is exported from an already-loaded model, the tensor names are the Rust module's final paths — after remapping, fusing and transposing. It is therefore not a drop-in replacement for the upstream checkpoint in the Python implementation; it is meant for voxcpm-rs, which applies it with no adapter.

Usage

In the browser this is the default weights source — "Hugging Face, INT8 (2.45 GiB, AudioVAE included)" — at https://shi3z.github.io/voxcpm2web/, or natively:

cargo run --release --features wgpu --example stream_load -- \
  /path/to/VoxCPM2-q8 "こんにちは。" out.wav

Caveats

• Nobody has evaluated the audio quality formally. Correlation against the F32 output has not been computed for this checkpoint, and no listening test has been run. INT8 weight quantization is usually benign for TTS, but treat that as an assumption, not a result.
• Only Q8 works. Q4 is blocked upstream: cubek-matmul's autotuner requests 98304 bytes of workgroup shared memory where WebGPU offers 49152, and panics rather than retrying with a smaller tile.

Licence

Apache-2.0, inherited from VoxCPM2. These are OpenBMB's weights, numerically transformed. See LICENSE. Original model: https://huggingface.co/openbmb/VoxCPM2.