verl-per_context_rb-ragged-c0-an-s2base-cfg3-168100-20261117
verl 0.6.1 GRPO-recipe training of ByteDance/Ouro-1.4B-Thinking on compression_dataset_3200.jsonl with the
verl_trainer extension: algorithm per_context_rb, mode ragged, 100 rollout steps, Slurm job
168100, node orchard-flame-1. Training arguments --l1_mode none --length_penalty_alpha 0 --beta 0.001 --learning_rate 1e-6 --max_grad_norm 1.0 --cost_gradient analytic --cost_offset 0 --optimizer AdamW8bit --adam_betas 0.9 0.99 --weight_decay 0.1 --warmup_ratio 0.1 --ppo_micro_batch_size_per_gpu 1 --num_train_epochs 1 --loss_agg_mode seq-mean-token-sum-norm --no_gate_kl --gate_reward_baseline loo --gate_learning_rate 2e-4 --vllm_async_scheduling --ragged_kv_layout lanes; mode --exit_mode adaptive --cache_rule ragged --min_depth 2; engine --vllm_gpu_memory_utilization 0.45 --vllm_kv_cache_memory_bytes 68719476736.
Checkpoint format
Each global_step_N/ is a verl checkpoint: actor/model_world_size_W_rank_R.pt, optim_... and
extra_state_... (FSDP shards, optimizer and RNG state), data.pt (dataloader state) and the
completion marker _READY.json. actor/huggingface/ is the merged Hugging Face model written at the
same step (safetensors, config, tokenizer, the Ouro modeling code, policy_config.json with the exit
policy, READY), loadable without any merge:
from transformers import AutoModelForCausalLM
model = AutoModelForCausalLM.from_pretrained("global_step_N/actor/huggingface", trust_remote_code=True)
Every file was verified at the archiving commit before the local copy was deleted; receipts are in
records/receipts/. The training rollouts are the dataset hi-todayis-jh/ouro-1.4b-base-math-verl-per-context-rb-ragged-c0-an-s2base-cfg3-168100-20261117-rollouts.
Contents
run_manifest.json,run_config.json,verl_config.yaml: run identity, the trainer's resolved arguments and the complete verl configuration.launch_code/: the launcher used for this run.model_code/: the patchedmodeling_ouro.py,configuration_ouro.pyandconfig.jsonthe trainer loaded.training_source.tar.gz: snapshot ofverl_trainer/,cluster/,math_utils/,marginal_experiments/.records/: training log, receipts and archive logs, uploaded when the run finished.
Training metrics: W&B project ouro-1.4b-grpo-rltt-math, run verl-per_context_rb-ragged-c0-an-s2base-cfg3-168100-20261117.
Model tree for hi-todayis-jh/ouro-1.4b-base-math-verl-per-context-rb-ragged-c0-an-s2base-cfg3-168100-20261117
Base model
ByteDance/Ouro-1.4B-Thinking