CoPE on ALFWorld: Qwen2.5-7B-Instruct, seed 1, step 75

This is Qwen2.5-7B-Instruct trained on ALFWorld with CoPE (code: https://github.com/ulab-uiuc/CoPE), which here is GRPO plus the action-forecast auxiliary loss. It is the actor at training step 75 of 264 (2 epochs), seed 1.

The weights were merged from verl's FSDP checkpoint into a bf16 Hugging Face model. Only the weights are here: there is no optimizer or dataloader state, so this cannot continue that training run exactly. generation_config.json is the base model's.

Training score at this step

Score
Step 75 0.945
Mean of steps 66-75 0.916

The score is critic/score/mean: the training batch's ALFWorld success rate, over 16 tasks ร— 8 rollouts. It is not a held-out evaluation.

Training settings

  • Batch: 16 prompts ร— 8 rollouts.
  • Optimizer and losses: lr 1e-6, KL 0.001 (low_var_kl), entropy 0.002.
  • Rollouts: up to 20 rounds, 512 tokens per turn.
  • Action forecast: coefficient 0.01, K 3, gate wins, group norm on, skip invalid on.
  • Hardware: 4ร— H20.

Usage

from transformers import AutoModelForCausalLM, AutoTokenizer
repo = "skyyyyks/cope-alfworld-qwen2.5-7b-seed1-step75"
tok = AutoTokenizer.from_pretrained(repo)
model = AutoModelForCausalLM.from_pretrained(repo, torch_dtype="bfloat16", device_map="auto")
Downloads last month
26
Safetensors
Model size
8B params
Tensor type
BF16
ยท
Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for skyyyyks/cope-alfworld-qwen2.5-7b-seed1-step75

Base model

Qwen/Qwen2.5-7B
Finetuned
(3,274)
this model