verl-per_context_rb-ragged-c0-an-s2base-cfg3-168100-20261117

verl 0.6.1 GRPO-recipe training of ByteDance/Ouro-1.4B-Thinking on compression_dataset_3200.jsonl with the verl_trainer extension: algorithm per_context_rb, mode ragged, 100 rollout steps, Slurm job 168100, node orchard-flame-1. Training arguments --l1_mode none --length_penalty_alpha 0 --beta 0.001 --learning_rate 1e-6 --max_grad_norm 1.0 --cost_gradient analytic --cost_offset 0 --optimizer AdamW8bit --adam_betas 0.9 0.99 --weight_decay 0.1 --warmup_ratio 0.1 --ppo_micro_batch_size_per_gpu 1 --num_train_epochs 1 --loss_agg_mode seq-mean-token-sum-norm --no_gate_kl --gate_reward_baseline loo --gate_learning_rate 2e-4 --vllm_async_scheduling --ragged_kv_layout lanes; mode --exit_mode adaptive --cache_rule ragged --min_depth 2; engine --vllm_gpu_memory_utilization 0.45 --vllm_kv_cache_memory_bytes 68719476736.

Checkpoint format

Each global_step_N/ is a verl checkpoint: actor/model_world_size_W_rank_R.pt, optim_... and extra_state_... (FSDP shards, optimizer and RNG state), data.pt (dataloader state) and the completion marker _READY.json. actor/huggingface/ is the merged Hugging Face model written at the same step (safetensors, config, tokenizer, the Ouro modeling code, policy_config.json with the exit policy, READY), loadable without any merge:

from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("global_step_N/actor/huggingface", trust_remote_code=True)

Every file was verified at the archiving commit before the local copy was deleted; receipts are in records/receipts/. The training rollouts are the dataset hi-todayis-jh/ouro-1.4b-base-math-verl-per-context-rb-ragged-c0-an-s2base-cfg3-168100-20261117-rollouts.

Contents

  • run_manifest.json, run_config.json, verl_config.yaml: run identity, the trainer's resolved arguments and the complete verl configuration.
  • launch_code/: the launcher used for this run.
  • model_code/: the patched modeling_ouro.py, configuration_ouro.py and config.json the trainer loaded.
  • training_source.tar.gz: snapshot of verl_trainer/, cluster/, math_utils/, marginal_experiments/.
  • records/: training log, receipts and archive logs, uploaded when the run finished.

Training metrics: W&B project ouro-1.4b-grpo-rltt-math, run verl-per_context_rb-ragged-c0-an-s2base-cfg3-168100-20261117.

Downloads last month

-

Downloads are not tracked for this model. How to track
Video Preview
loading

Model tree for hi-todayis-jh/ouro-1.4b-base-math-verl-per-context-rb-ragged-c0-an-s2base-cfg3-168100-20261117

Finetuned
(36)
this model

Dataset used to train hi-todayis-jh/ouro-1.4b-base-math-verl-per-context-rb-ragged-c0-an-s2base-cfg3-168100-20261117