ChristophSchuhmann's picture
Update TRAINING.md
b913ad1 verified
|
Raw History Blame Contribute Delete
5.74 kB

Reproducing the 24,000-group Chatterbox S3Gen VC follow-ups

Model and data lineage

The frozen upstream decoder is ResembleAI/chatterbox@5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18/s3gen.safetensors. The trainable starting adapters are the completed v1 step-4,000 checkpoints in laion/chatterbox-s3gen-vc-grow: continue for the 50/50 control and 40/40/20 speaker arm, merged_new for the remerged 50/50 arm. The v1 release explains the preceding 300-hour S6 and synthetic-sidecar supervised fine-tune, 99-axis score projector, packed audio preparation, model versions and checkpoint structure. These follow-ups do not start from a fresh Chatterbox model.

The frozen source/target condition table has 4,000 rows selected from that 300-hour prepared pool, with 100 top-ranked eligible sources for each of 40 emotion axes. The sources are 3–8 seconds; target references are 5–10 seconds and their CAMPPlus cosine to the respective source is below 0.30. There are 3,357 distinct source UIDs and 477 target UIDs. Each run makes six deterministic, differently shuffled passes over the same 4,000 pairs. The same pair table and random seeds are used across the three arms for a controlled comparison. The prepared pool.parquet, selection.parquet and four input_ears_*.parquet shards are required for exact reproduction but are not included in these model repositories; original record locators and licensed audio must be staged separately. Do not unpack samples into thousands of loose files.

Rewards and optimizer

All three arms generate 16 on-policy candidates with S3Gen's 10-step Euler flow sampler for a single source/target condition. Humaneness Ears Medium at revision 818506970d93c809a3295a0a02c40dee8ff2bfc3 predicts the audiobox:CE output (AudioBox Content Enjoyment) and 40 emotion scores on each generated output. The two strongest Ears emotion axes of the source audio define the emotion fidelity MSE. The third source emotion is omitted. The speaker-aware arm additionally runs the frozen S3Gen CAMPPlus encoder on each output and computes cosine to the target-reference CAMPPlus embedding.

Each raw component is independently standardized across all 16 candidates in the group. The control arms use 0.50*z(CE)+0.50*z(-top2_MSE); the speaker arm uses 0.40*z(CE)+0.40*z(-top2_MSE)+0.20*z(target_CAMP_cosine). The weighted score is clipped to [-2.5, 2.5] and centered to zero group mean. Signed advantages weight the flow-matching velocity loss on the generated mel portion, plus 0.025 times the squared velocity difference from the respective frozen v1 step-4,000 policy. This is GROW-inspired group-relative flow optimization, not token-level GRPO. Ears is a training reward model, so its scores are not an independent evaluation.

The only trainable parameters are the rank-128 q/k/v/out LoRA and the previously trained 99-score projection. All other Chatterbox modules remain frozen. Each arm restarts AdamW (weight_decay=0.01) at the v1 step-4,000 model, warms up over 1,200 of 24,000 groups, peaks at 2e-6, then follows one cosine decay to 2e-7. The 50/50 remerged arm merges the completed v1 merged_new linear LoRA into its effective base a second time, starts a new zero-output rank-128 LoRA, and retains the v1 score projector. Four GPUs process four candidates each, all-gather reward components, and average gradients before every synchronized optimizer update. The frozen anchor is the respective starting policy. Checkpoints every 1,000 groups include model trainables, full AdamW state, code/reward/schedule metadata and an atomic completion marker.

Jupiter execution and resume

The recorded Jupiter Slurm scripts request one exclusive Booster node, four GH200 GPUs, four tasks, one GPU per task, project reformo, and 11h50 walltime. An initial 20-group smoke checks identical initial trainables across ranks, finite rewards and gradients, and nonzero updates. Production must not begin without the matching smoke PASS marker. Logs, metrics, checkpoints and canary audio stay in Scratch; metrics use one JSONL and canaries one TAR per arm. This avoids large inode counts. If a job reaches the walltime, its dependent fallback can resume from the latest complete optimizer-bearing checkpoint. The actual run.sbatch paths are Jupiter specific; change account, storage, environment and paths before reuse.

The repositories contain code/grow_v2/train_ddp.py and code/grow_v3/train_speaker.py, plus their imported v1/base helpers, code/grow_followup/infer_followup.py, the Slurm scripts, requirements and the fixed 99-axis score order. For continuation, provide the same packed pool and selection order, set CHATTERBOX_RUN_ROOT, CHATTERBOX_GROW_TRAIN_ROOT, and the arm's CHATTERBOX_GROW_V2_ROOT or CHATTERBOX_GROW_V3_ROOT as appropriate. The new schedule and objective must match the checkpoint; changing either is a new experiment, not an exact resume.

Why three arms, and what is not yet known

The two 50/50 arms test whether continued training of the existing LoRA or merging it into the base and starting a fresh LoRA is better. The 40/40/20 arm tests whether an explicit speaker-identity reward prevents the slight speaker-similarity regression observed in the earlier CE-free v1 pilot. No default recommendation should be made from training reward alone. Held-out, matched-seed evaluation must report a speaker cosine, source emotion fidelity, an independent quality/naturalness metric, content accuracy (e.g. ASR/word error rate), and listening examples. The holdout should ideally be speaker-disjoint, which the previous UID-only split was not guaranteed to be.