jasonjimnz/tyness-boe-1.2b-pre-alpha-instruct

Tyness BOE 1.2B Instruct — Pre-Alpha (focused grounded SFT)

Third pre-alpha from Tyness Train. This one is fine-tuned from the Instruct variant and trained with focused contexts: instead of whole norm sections, each prompt carries only the paragraphs lexically relevant to the question — everything fits a 2048-token training window with zero truncation.

⚠️ Status: PRE-ALPHA. Best grounded results so far (mean judge score 0.35, 2/20 fully correct, 8/20 partially correct on the hardest eval set) but not production-grade. Verify against boe.es.

Model description

Base model	LiquidAI/LFM2.5-1.2B-Instruct
Task	Context-grounded Spanish law Q&A (BOE)
Method	QLoRA SFT (TRL, NF4 + double quant, fp32 compute)
Hardware	NVIDIA GTX 1060 6GB (Pascal, sm_61)
Training	3.1h · 216 steps · 3 epochs · 2.0M tokens · token accuracy 78.2%
Author	jasonjimnz

The experiment so far

Version	Approach	Mean judge score
pre-alpha 1	plain SFT (memorization attempt)	0.005
run2	more plain SFT	0.035
pre-alpha 2	RAG-style, whole sections (Base)	0.267
pre-alpha-instruct (this)	Instruct + focused paragraphs	0.3475

Key insight: SFT on 1.2B models teaches format, not facts — grounding must live in the prompt, and the Instruct prior beats extra epochs on the Base model.

Training data

10 BOE articles (state + autonomous scope), 1,148 grounded train items + 20 held-out:

BOE id	Norm	Train	Eval
BOE-A-2026-14659	Ley 8/2026, Transporte Marítimo de Cantabria	118	2
BOE-A-2020-7440	Decreto-ley 9/2020 (ayudas COVID, Extremadura)	142	2
BOE-A-2021-10163	Orden HAC/612/2021, modelo 179	118	2
BOE-A-2019-18746	Orden HAC/1270/2019	118	2
BOE-A-2016-9637	Orden INT/1676/2016	118	2
BOE-A-1995-19198	Acuerdo transporte internacional por carretera	119	2
BOE-A-1996-6400	RD 330/1996, traspaso de funciones	60	2
BOE-A-1975-9248	Ley 17/1975, Registro de Matrícula Industrial	118	2
BOE-A-2014-10908	Orden HAP/1949/2014, Punto General de Entrada	119	2
BOE-A-1993-10089	RD 562/1993	118	2

Methodology (focused contexts):

• Each entry cites precepts (articulos_citados); those sections are split into paragraphs and only the lexically relevant ones (term overlap with the question) enter the CONTEXTO block, plus the norm's frontmatter identifiers (grounds BOE-A-AAAA-NNNNN answers).
• System prompt: answer only from the context; «No consta en el contexto proporcionado» otherwise.
• Loss on assistant tokens only; provenance metadata never trained on.
• Evaluation: LLM referee (glm5.3-flash) on held-out items, judged against expected answers and key reasons — including the harder 9K-char-context eval set from pre-alpha 2 for comparability (this card's reported score).

Training configuration

Parameter	Value
LoRA r / alpha	16 / 32 (q,k,v,o_proj)
Learning rate / scheduler	2e-4 / cosine, warmup 6
Epochs / effective batch	3 / 1 × 16
Sequence length	2048 (all focused contexts fit)
Quantization / optimizer	NF4 + double quant, fp32 compute / paged_adamw_32bit

Evaluation

Metric	Value (hardest eval set, 9K-char contexts)
Correct	2/20
Partial	8/20 (40%)
Mean judge score	0.3475

On its native focused-context eval the model behaves consistently: grounded reading, correct norm identification, refusal calibration. Exact-structure outputs (JSON fichas) remain the weak spot.

How to run

llama.cpp

llama-cli -hf jasonjimnz/tyness-boe-1.2b-pre-alpha-instruct:Q4_K_M --jinja -c 16384

llama-server -m tyness-boe-1.2b-pre-alpha-instruct-q4_k_m.gguf -ngl 99 -c 16384 --port 8080 --jinja
curl http://127.0.0.1:8080/v1/chat/completions -d '{
  "messages": [
    {"role":"system","content":"Eres un jurista experto en el BOE. Responde ÚNICAMENTE con el CONTEXTO. Si un dato no consta: No consta en el contexto proporcionado."},
    {"role":"user","content":"CONTEXTO:\n<relevant norm paragraphs>\n\nPREGUNTA: ¿…?"}
  ],
  "temperature": 0
}'

transformers

from transformers import AutoModelForCausalLM, AutoTokenizer

model = AutoModelForCausalLM.from_pretrained("jasonjimnz/tyness-boe-1.2b-pre-alpha-instruct", dtype="auto")
tok = AutoTokenizer.from_pretrained("jasonjimnz/tyness-boe-1.2b-pre-alpha-instruct")
messages = [
    {"role": "system", "content": "Eres un jurista experto en el BOE. Responde ÚNICAMENTE con el CONTEXTO proporcionado; si un dato no consta, responde: No consta en el contexto proporcionado."},
    {"role": "user", "content": "CONTEXTO:\n...\n\nPREGUNTA: ..."},
]
inputs = tok.apply_chat_template(messages, add_generation_prompt=True, return_tensors="pt")

Files

File	Format	Size
model.safetensors	safetensors bf16	2.2GB
tyness-boe-1.2b-pre-alpha-instruct-f16.gguf	GGUF F16	2.2GB
tyness-boe-1.2b-pre-alpha-instruct-q8_0.gguf	GGUF Q8_0	1.2GB
tyness-boe-1.2b-pre-alpha-instruct-q4_k_m.gguf	GGUF Q4_K_M	698MB

Limitations

• 10-norm scope; 60% of hardest-eval items still fail or are only partially grounded.
• Structured/JSON outputs degrade; prose answers are stronger.
• 1.2B class: cite-only behavior is trained, but complex multi-precept reasoning is out of reach.
• Spanish only. Never use for real legal guidance.

Attribution

Base model: LiquidAI (LFM Open License v1.0). Data: Agencia Estatal Boletín Oficial del Estado, reused per its open-data conditions, via legalize-es.

Trained and published with Tyness Train v0.4.17 — Pre-Alpha, October 2026.