You need to agree to share your Hugging Face information to access this model

This repository is publicly accessible, but you have to accept the conditions to access its files and content.

EditLens Llama-3.2-3B QLoRA reproduction

Reproduction of the Llama-3.2-3B QLoRA EditLens training config from https://github.com/pangramlabs/EditLens (commit 05a588f).

This is a LoRA adapter, not a standalone model. Load with:

from peft import PeftModel
from transformers import AutoModelForSequenceClassification
base = AutoModelForSequenceClassification.from_pretrained(
    "meta-llama/Llama-3.2-3B", num_labels=4, load_in_4bit=True)
model = PeftModel.from_pretrained(base, "Deepanika/editlens-llama3.2-3b-qlora-repro")

Requires access to gated meta-llama/Llama-3.2-3B.

Training

Data: pangram/editlens_iclr, score_col=cosine_score, 4 buckets, thresholds [0.03, 0.15]
lr=3e-5, effective batch=24 (per_device=3, grad_accum=8, replicating the repo's 8-GPU / per-device-3 setup on 1 GPU), 1 epoch, constant schedule
4-bit NF4 quantization, LoRA r=8 alpha=16, gradient checkpointing
Single A6000, 5h37m wall-clock, peak memory 5.7 GB
Trainable params: 12,175,360 / 3,224,943,616 (0.38%)
Deviations: num_proc 32→8; same TrainingArguments compat patch as RoBERTa run

Downloads last month: 30

Inference Providers NEW

This model isn't deployed by any Inference Provider. 🙋 Ask for provider support

Model tree for Deepanika/editlens-llama3.2-3b-qlora-repro

Base model

meta-llama/Llama-3.2-3B

Adapter

(440)

this model