satanicmangoes/nllb-600m-en-de-context-sft

This model is from the paper Balancing Global Quality and Pronoun-Specific Feedback for Context-Aware Machine Translation.

Context-aware SFT — English→German, with preceding context

NLLB-200-distilled-600M fine-tuned to translate a sentence given its preceding source sentence, from the ProNMT project (Balancing Global Quality and Pronoun-Specific Feedback for Context-Aware Machine Translation). Code and data: https://github.com/Harshit2807161/ProNMT

Input format

<context> PREVIOUS ENGLISH SENTENCE </context> CURRENT ENGLISH SENTENCE

The model outputs the German translation of the current sentence only.

Training

• Base model: facebook/nllb-200-distilled-600M.
• Data: Europarl En–De with the preceding sentence as context (42,750 train); model selection on the 750-sentence validation split by eval loss.
• Seq2Seq cross-entropy; AdamW, learning rate 1e-4, linear schedule without warmup, per-device batch 3, gradient accumulation 16, weight decay 0, fp16, seed 42.

Usage

from transformers import AutoModelForSeq2SeqLM, NllbTokenizerFast

repo = "satanicmangoes/nllb-600m-en-de-context-sft"
tokenizer = NllbTokenizerFast.from_pretrained(repo, src_lang="eng_Latn", tgt_lang="deu_Latn")
model = AutoModelForSeq2SeqLM.from_pretrained(repo)

src = '<context> I met the Commissioner yesterday. </context> She said the report would be ready soon.'
inputs = tokenizer(src, return_tensors="pt")
out = model.generate(**inputs, forced_bos_token_id=tokenizer.convert_tokens_to_ids("deu_Latn"))
print(tokenizer.decode(out[0], skip_special_tokens=True))

License

CC-BY-NC-4.0, inherited from NLLB-200.