# Reproducing the 24,000-group Chatterbox S3Gen VC follow-ups ## Model and data lineage The frozen upstream decoder is `ResembleAI/chatterbox@5bb1f6ee58e50c3b8d408bc82a6d3740c2db6e18/s3gen.safetensors`. The trainable starting adapters are the completed v1 step-4,000 checkpoints in [`laion/chatterbox-s3gen-vc-grow`](https://huggingface.co/laion/chatterbox-s3gen-vc-grow): `continue` for the 50/50 control and 40/40/20 speaker arm, `merged_new` for the remerged 50/50 arm. The v1 release explains the preceding 300-hour S6 and synthetic-sidecar supervised fine-tune, 99-axis score projector, packed audio preparation, model versions and checkpoint structure. These follow-ups **do not start from a fresh Chatterbox model**. The frozen source/target condition table has 4,000 rows selected from that 300-hour prepared pool, with 100 top-ranked eligible sources for each of 40 emotion axes. The sources are 3–8 seconds; target references are 5–10 seconds and their CAMPPlus cosine to the respective source is below 0.30. There are 3,357 distinct source UIDs and 477 target UIDs. Each run makes six deterministic, differently shuffled passes over the **same 4,000 pairs**. The same pair table and random seeds are used across the three arms for a controlled comparison. The prepared `pool.parquet`, `selection.parquet` and four `input_ears_*.parquet` shards are required for exact reproduction but are not included in these model repositories; original record locators and licensed audio must be staged separately. Do not unpack samples into thousands of loose files. ## Rewards and optimizer All three arms generate 16 on-policy candidates with S3Gen's 10-step Euler flow sampler for a single source/target condition. Humaneness Ears **Medium** at revision `818506970d93c809a3295a0a02c40dee8ff2bfc3` predicts the `audiobox:CE` output (AudioBox Content Enjoyment) and 40 emotion scores on each generated output. The two strongest Ears emotion axes of the **source audio** define the emotion fidelity MSE. The third source emotion is omitted. The speaker-aware arm additionally runs the frozen S3Gen CAMPPlus encoder on each output and computes cosine to the target-reference CAMPPlus embedding. Each raw component is independently standardized across all 16 candidates in the group. The control arms use `0.50*z(CE)+0.50*z(-top2_MSE)`; the speaker arm uses `0.40*z(CE)+0.40*z(-top2_MSE)+0.20*z(target_CAMP_cosine)`. The weighted score is clipped to `[-2.5, 2.5]` and centered to zero group mean. Signed advantages weight the flow-matching velocity loss on the generated mel portion, plus `0.025` times the squared velocity difference from the respective frozen v1 step-4,000 policy. This is GROW-inspired group-relative flow optimization, not token-level GRPO. Ears is a **training reward model**, so its scores are not an independent evaluation. The only trainable parameters are the rank-128 q/k/v/out LoRA and the previously trained 99-score projection. All other Chatterbox modules remain frozen. Each arm restarts AdamW (`weight_decay=0.01`) at the v1 step-4,000 model, warms up over 1,200 of 24,000 groups, peaks at `2e-6`, then follows one cosine decay to `2e-7`. The 50/50 remerged arm merges the completed v1 `merged_new` linear LoRA into its effective base a second time, starts a new zero-output rank-128 LoRA, and retains the v1 score projector. Four GPUs process four candidates each, all-gather reward components, and average gradients before every synchronized optimizer update. The frozen anchor is the respective starting policy. Checkpoints every 1,000 groups include model trainables, full AdamW state, code/reward/schedule metadata and an atomic completion marker. ## Jupiter execution and resume The recorded Jupiter Slurm scripts request one exclusive Booster node, four GH200 GPUs, four tasks, one GPU per task, project `reformo`, and 11h50 walltime. An initial 20-group smoke checks identical initial trainables across ranks, finite rewards and gradients, and nonzero updates. Production must not begin without the matching smoke PASS marker. Logs, metrics, checkpoints and canary audio stay in Scratch; metrics use one JSONL and canaries one TAR per arm. This avoids large inode counts. If a job reaches the walltime, its dependent fallback can resume from the latest complete optimizer-bearing checkpoint. The actual `run.sbatch` paths are Jupiter specific; change account, storage, environment and paths before reuse. The repositories contain `code/grow_v2/train_ddp.py` and `code/grow_v3/train_speaker.py`, plus their imported v1/base helpers, `code/grow_followup/infer_followup.py`, the Slurm scripts, requirements and the fixed 99-axis score order. For continuation, provide the same packed pool and selection order, set `CHATTERBOX_RUN_ROOT`, `CHATTERBOX_GROW_TRAIN_ROOT`, and the arm's `CHATTERBOX_GROW_V2_ROOT` or `CHATTERBOX_GROW_V3_ROOT` as appropriate. The new schedule and objective must match the checkpoint; changing either is a new experiment, not an exact resume. ## Why three arms, and what is not yet known The two 50/50 arms test whether continued training of the existing LoRA or merging it into the base and starting a fresh LoRA is better. The 40/40/20 arm tests whether an explicit speaker-identity reward prevents the slight speaker-similarity regression observed in the earlier CE-free v1 pilot. No default recommendation should be made from training reward alone. Held-out, matched-seed evaluation must report a speaker cosine, source emotion fidelity, an independent quality/naturalness metric, content accuracy (e.g. ASR/word error rate), and listening examples. The holdout should ideally be speaker-disjoint, which the previous UID-only split was not guaranteed to be.