mlx-community/fish-audio-drama-3-bf16

Drama 3 MLX bf16

Built with Fish Audio.

Fish Audio Drama 3 converted for mlx-audio on Apple Silicon, with nothing quantized. This is an unofficial community conversion; Fish Audio did not release or endorse it.

• routed experts: dequantized from the block-FP8 release to bfloat16
• every other weight: bfloat16, as released
• 67 GB on disk, about 74 GB peak memory while generating, so it needs a Mac with at least 96 GB

For smaller Macs, mlx-community/fish-audio-drama-3-4bit needs about 26 GB.

Usage

Drama 3 support was merged into mlx-audio in #1018, after the 0.5.9 release. Until the next release, install from main:

pip install "mlx-audio @ git+https://github.com/Blaizzy/mlx-audio"

from mlx_audio.tts.utils import load_model

model = load_model("mlx-community/fish-audio-drama-3-bf16")

for result in model.generate(
    text="[sighs] Fine. If we miss it, it's on you.",
    ref_audio="reference.wav",
    ref_text="Transcript of the reference clip.",
    stream=True,
    first_chunk_frames=4,
):
    print(result.audio.shape)  # 44.1 kHz chunks

python -m mlx_audio.tts.generate --model mlx-community/fish-audio-drama-3-bf16 \
    --text "[sighs] Fine. If we miss it, it's on you." \
    --ref_audio reference.wav --ref_text "Transcript of the reference clip." \
    --temperature 0.7 --top_p 0.7 --top_k 30

Speaker tags (<|speaker:0|> to <|speaker:15|>), multiple references and streaming are described in the model docs.

Speed

On an M3 Max with 128 GB:

• about 1.4x real time
• first streamed audio in 270 to 300 ms once a voice is cached
• loading takes about 12 s from a warm page cache, and the first call after loading about 3 s

Conversion

Made with mlx-audio's converter, with nothing quantized:

python -m mlx_audio.tts.models.fish_drama3.convert --src drama-3 --dst drama-3-mlx-bf16 \
    --expert-bits 0 --dense-bits 0 --tower-bits 0

NOTICE lists every change to the original weights.

License

The Fish Audio Community License applies, including its use restrictions and its terms for commercial use and hosted services.