Turnike-JEV-0.8B — Open-Weight Decision Model

Turnike-JEV-0.8B: the 0.8B member of the Turnike family — an open, bilingual (English + Turkish) Jev-class decision model; text plus closed questions in, option probabilities out, fully offline.

Turnike-JEV-0.8B: Turnike ailesinin 0.8B üyesi — iki dilli (İngilizce + Türkçe), açık ağırlıklı, Jev sınıfı bir karar modeli; metin ve kapalı sorular girer, seçenek olasılıkları çıkar; tamamen çevrim dışı.

• The strongest sub-1B decision model we have measured. Macro accuracy 0.681 on the full JevBench test, +0.155 over the untrained Qwen3.5-0.8B (0.526). It is above every public leaderboard entry up to 7B except the fine-tuned 2B readout models (0.702–0.704): K2-Horizon-7B 0.623, Tier-1 Qwen3.5-2B 0.632, gemma-4-E2B 0.591, SmolLM3-3B 0.552. The leaderboard has no fine-tuned entry under 1B; our run is not submitted yet.
• Ahead of Kev-0.8B on all four shared tests — JevBench EN-300 0.676 vs 0.570 (18 of 22 configs, paired McNemar p = 2e-62) and 1,379 human-labelled Turkish decisions 0.699 vs 0.583 — and within 0.03 of Kev-4B, a model five times its size (0.706 / 0.708).
• 24 ms per decision, 1.4 GiB of GPU memory, a 43 MB adapter. civil_comments 0.921, level with the larger sizes (0.919 / 0.912).
• Same recipe as Turnike-JEV-4B. The same data, the same draw and the same hyperparameters; nothing was tuned for this size. The 4B card carries the full benchmark tables, the method figure and the honesty notes; this card gives the numbers for the 0.8B model.
• Full JevBench test (22 configs, 22,773 rows, official jevify-run report, raw probabilities): macro accuracy 0.681, macro ECE 0.078, six held-out sources 0.694, other 16 0.676; Turnike-JEV-4B 0.774. Single run.
• English: JevBench EN-300 macro accuracy 0.676 (six held-out sources 0.682, other 16 0.674); Turnike-JEV-4B: 0.769. Other models on the same 6,600 rows: Microsoft-Decision-1 (9B, closed, API) 0.775, Kev-4B 0.706, internlm/Intern-Decision-4B 0.687, Kev-0.8B 0.570.
• Turkish: tr_insan, 1,379 human-labelled Turkish decisions: 0.699; Turnike-JEV-4B: 0.748. Others on the same items: Jev 1.13.0 (API) 0.727, Microsoft-Decision-1 (9B, closed, API) 0.714, Kev-4B 0.708, internlm/Intern-Decision-4B 0.675.
• Also: practical Turkish questions (2,966) 0.840; LegalBench rule application (885 items, tasks not trained on) 0.724; machine-translated JevBench TR-50 (supplementary, 50 rows per config) 0.560.
• Runs on your hardware, offline. A LoRA adapter of about 43 MB on Qwen/Qwen3.5-0.8B; the text never leaves the machine. Up to 16 questions about one text are answered in a single forward pass. No merged repo for this size: the adapter is 43 MB and the base model loads in seconds.
• Latency. bf16: ~24 ms per decision (1 question) and ~32 ms for 8 questions on one text, 1.4 GiB in steady state; measured on an RTX 5060 Ti 16 GB through karar.py, p50, CUDA-synchronised, 2 warm-up calls dropped (1 question: a 7-option topic question over 60 tr_insan texts; 8 questions: 15 support-desk messages). The 4B under the same protocol: ~108 / ~156 ms in bf16, ~82 / ~127 ms in FP8.

"JevBench" on this card means the Praveenrajus/jev-bench dataset (22 configs) scored with the jevify-run report of uspraveen/Jevify. It is not the JevBench board by Benchmark Heaven, a different benchmark with the same name, which we have not entered yet.

Declarations / Beyanlar

• Trained on public Turkish and English datasets, LLM-labelled public texts and synthetic data; the datasets are not released. / Herkese açık Türkçe ve İngilizce veri setleri, LLM ile etiketlenmiş açık metinler ve sentetik veriyle eğitildi; veri setleri yayınlanmıyor.
• Not distilled from any commercial decision API. / Herhangi bir ticari karar API'sinden damıtılmadı.
• AI assistance: the card and code were written with Claude coding agents under the author's direction (details). / Kart ve kod, yazarın yönlendirmesiyle Claude kodlama ajanlarıyla yazıldı.
• No web/browser data in this release; the browser version will be published separately. / Bu sürümde web verisi yok; tarayıcı sürümü ayrı yayınlanacak.

Turnike-JEV-0.8B is a LoRA adapter (r=16, 10.8M trainable parameters, about 43 MB) on Qwen/Qwen3.5-0.8B. It does not generate text. The text and the questions go into one chat prompt, and the assistant turn is a JSON skeleton with one placeholder per question. Every option of a question is mapped to a single-token symbol (A, B, C, ...). At the token just before each placeholder, the model's final hidden state is multiplied with the output rows of that question's symbols only. A softmax over those symbols gives the option probabilities.

• Author: Berkay Hasret (independent researcher)
• Languages: English and Turkish
• Base model: Qwen/Qwen3.5-0.8B (Apache-2.0, Copyright 2026 Alibaba Cloud), frozen during training; only the adapter is trained
• License of the adapter, karar.py and this card: Apache-2.0 (see License)
• Trained on: RTX 5060 Ti 16 GB, batch 4 × 8; 2,831 steps, effective batch 32, max 1,536 tokens, seed 3

Model family

Three sizes, one recipe: the same data, the same draw (seed 3) and the same hyperparameters (LoRA r=16, α=32, lr 1e-4, 2,831 steps, effective batch 32) for every size. Nothing was tuned per size, so the smallest and the largest model are probably somewhat under-tuned. Each size is its own repo; the row in bold is the model of this card.

model	base	trainable params	full JevBench (22,773)	JevBench EN-300 (EN)	held-6 / other-16	tr_insan, 1,379 human-labelled (TR)	practical TR, 2,966	LegalBench rules, 885 (EN)	weights	trained on
Turnike-JEV-0.8B (this card)	Qwen/Qwen3.5-0.8B	10.8M	0.681	0.676	0.682 / 0.674	0.699	0.840	0.724	adapter (43 MB)	RTX 5060 Ti 16 GB, batch 4 × 8
Turnike-JEV-4B	Qwen/Qwen3.5-4B	32.5M	0.774	0.769	0.826 / 0.748	0.748	0.891	0.832	adapter (130 MB) · merged bf16 · FP8 (accuracy check pending)	RTX 5060 Ti 16 GB, batch 4 × 8
Turnike-JEV-9B	Qwen/Qwen3.5-9B	43.3M	0.783	0.780	0.839 / 0.758	0.763	0.889	0.890	adapter (173 MB) · merged bf16 · FP8	Colab RTX PRO 6000 96 GB, batch 16 × 2

_{Same protocol as the tables below: single runs, the same items for every model, no calibration recipe; full JevBench = official jevify-run report on all 22,773 test rows. Differences below about 0.01 are within run-to-run noise. The machine-translated TR-50 is in Table 5.}

Quick start

pip install torch "transformers>=5.17" peft accelerate
hf download berkay123001/turnike-jev-0.8b karar.py requirements.txt --local-dir .
# optional: pip install torchao (FP8), causal-conv1d and flash-linear-attention (faster linear-attention layers)

from karar import Karar
k = Karar("berkay123001/turnike-jev-0.8b")   # the base model (Qwen/Qwen3.5-0.8B) is read from adapter_config.json and downloaded automatically

# TR
k.sor("Kulaklık iki günde bozuldu, paramı geri istiyorum.",
      {"Yazar para iadesi istiyor mu?": {"type": "noul"},
       "Mesajın tonu nedir?": ["olumlu", "olumsuz", "nötr"]})
# -> {"Yazar para iadesi istiyor mu?": {"false": p, "true": p}, "Mesajın tonu nedir?": {"olumlu": p, "olumsuz": p, "nötr": p}}

# EN
k.sor("The production database has been down since 9am. Please escalate now.",
      {"Does this message require immediate action?": {"type": "noul"},
       "Which team should handle it?": {"billing": "payments, refunds", "infra": "servers, outages", "sales": "pricing"}})

karar.py is the same file for every Turnike size; the question formats (noul yes/no, choice with a list or a dict of option descriptions, score) and the command line (python karar.py --help) are documented on the Turnike-JEV-4B card. Karar(..., fp8=True) enables FP8 on CUDA.

Results

All numbers are our own measurements with the same scripts and the same items as on the 4B card; single runs, raw probabilities, no calibration recipe. Turnike rows are in bold.

model	JevBench EN-300 (EN)	LegalBench rules, 885 (EN)	tr_insan, 1,379 human-labelled (TR)	practical TR, 2,966 (TR)
Turnike-JEV-4B	0.769	0.832	0.748	0.891
Turnike-JEV-9B	0.780	0.890	0.763	0.889
Turnike-JEV-0.8B	0.676	0.724	0.699	0.840
Microsoft-Decision-1 (9B, closed, API)	0.775	–	0.714	0.857
Jev 1.13.0 (API)	–	0.924 †	0.727	0.872
Kev-4B	0.706	0.786	0.708	0.797
jevify Qwen3.5-4B readout LoRA + coherence	–	–	0.642	0.847
internlm/Intern-Decision-4B	0.687	0.656	0.675	0.843
OpenJev 4B (qwen3.5-4b-nli-v5)	–	–	0.645	0.738
Qwen3.5-4B, zero-shot (our prompt)	–	–	0.612	0.745
Kev-0.8B	0.570	0.496	0.583	0.697

† Reported by the jevify author on the same 885 items (jevify README, finding 14); not our run. – = not measured. Run-to-run noise: retraining an earlier run of the same recipe with another seed moved tr_insan by +0.001 and EN-300 by +0.002. The machine-translated JevBench TR-50 is a supplementary set (Table 5); the human-labelled tr_insan is the primary Turkish measure.

The detailed tables (per-config JevBench against Jev 1.13.0, paired McNemar tests, the "none of the above" and planted-instruction behaviour tests, KVKK, latency) are on the Turnike-JEV-4B card. Turnike-JEV-4B is the reference model of the family; this size was measured afterwards with the same code.

Method and training

One call, one forward pass: the text and up to 16 questions are packed into a single chat prompt whose assistant turn is a JSON skeleton with one placeholder per question; each option is mapped to a single-token symbol and the option probabilities are read from a softmax over those symbols only (see the Method and Prompt format sections of the 4B card, with the algorithm and the figure).

Training: KL divergence between the target distribution and the model's option distribution per question; LoRA r=16, α=32, dropout 0.05 on all attention, MLP and linear-attention projections of the language model (the vision tower is untouched and unused); lr 1e-4, warm-up 3%, effective batch 32, 2,831 steps, max length 1,536 tokens, the same 80,000 packed items (seed 3) as the 4B. Two training-data shortcuts found during the 4B work ("none of the above" read from a sibling question, and from the option count) were fixed before this run; see Training on the 4B card.

Limitations

• No per-size tuning. The learning rate, the number of steps and the LoRA rank were chosen for the 4B; this size may be under- or over-trained. Single run; differences below about 0.01 (0.02 on TR-50) are within run-to-run noise.
• Test sets were used for selection of the recipe (on the 4B); none of the numbers is a fully untouched estimate. The train splits of 16 JevBench configs and of the five tr_insan sources are in training (test texts excluded).
• Mostly LLM labels. Much of the training data is LLM-labelled, so the model inherits those judgements. tr_insan is the only human-labelled Turkish test.
• Calibration: no temperature or recipe is applied. Fit a temperature on your own validation data if you need calibrated probabilities, and do not threshold yes/no probabilities near 0.5 without a validation set.
• Not trained on source code. On a small internal security code-review test (32 snippets with access-control bugs, deliberately fuzzy options, 128 decisions) this size scores 0.352, against 0.805 for Jev 1.13.0 and 0.695 for Microsoft-Decision-1; the other local 4B decision models we ran (Qwen3.5-4B zero-shot, openjev, Intern-Decision-4B, jevify readout) land between 0.414 and 0.547. Treat code review as out of scope for this release.
• FP8 (Karar(..., fp8=True)) was validated on the 4B only. On the 0.8B one of 13 test decisions changed its argmax under FP8 (max |Δp| 0.24); use bf16.
• Long prompts are truncated (text only; maks_istem_token in karar.py); training used 1,536 tokens. Questions with more than 62 options use an elimination step.

License

The base model Qwen/Qwen3.5-0.8B is licensed under Apache-2.0 (Copyright 2026 Alibaba Cloud). The adapter weights, karar.py, the tests and this card are released under Apache-2.0 (LICENSE).

AI assistance

This model card, karar.py, the tests and most of the training and evaluation code were written with Claude coding agents (Claude Code, Anthropic) under the author's direction. The author is responsible for the content and verified the reported numbers.

Citation

@misc{turnike_jev_0_8b_2026,
  title        = "Turnike-JEV-0.8B: a bilingual (English + Turkish) decision model on Qwen/Qwen3.5-0.8B",
  author       = "Hasret, Berkay",
  year         = "2026",
  howpublished = "https://huggingface.co/berkay123001/turnike-jev-0.8b"
}

Türkçe özet

Turnike-JEV-0.8B, Turnike ailesinin 0.8B üyesidir: Qwen/Qwen3.5-0.8B üzerine yaklaşık 43 MB'lık bir LoRA adaptörü. Metin üretmez; bir metin ve bir ya da birkaç kapalı soru verildiğinde her seçeneğe bir olasılık döndürür, bir metnin 16 sorusuna kadar tek ileri geçişte cevap verir. RTX 5060 Ti 16 GB üzerinde tek karar bf16 ≈ 24 ms, 8 soru ≈ 32 ms sürer (1,4 GiB bellek). Eğitim verisi, çekiliş ve bütün ayarlar Turnike-JEV-4B ile birebir aynıdır; bu boyut için ayrı ayar yapılmadı.

Sonuçlar (aynı betikler, aynı maddeler, tek koşu): tam JevBench (22.773 satır; Praveenrajus/jev-bench veri seti, Benchmark Heaven'ın aynı adlı JevBench tablosu değil) makro 0.681 (Turnike-JEV-4B: 0.774), JevBench EN-300 makro 0.676 (Turnike-JEV-4B: 0.769), insan etiketli Türkçe test tr_insan 0.699 (Turnike-JEV-4B: 0.748), pratik Türkçe sorular 0.840, LegalBench kural uygulama 0.724, çeviri JevBench TR-50 0.560. Ayrıntılı tablolar, davranış testleri ve dürüstlük notları Turnike-JEV-4B kartındadır.

Herkese açık Türkçe ve İngilizce veri setleri, LLM ile etiketlenmiş açık metinler ve sentetik veriyle eğitildi; veri setleri yayınlanmıyor. Herhangi bir ticari karar API'sinden damıtılmadı. Eğitimde kaynak kodu yoktur; kod incelemesi bu sürümün kapsamı dışındadır. Lisans: taban model ve adaptör Apache-2.0. Kullanım için karar.py içindeki Karar("berkay123001/turnike-jev-0.8b").sor(metin, sorular) yeterlidir. Bu kart ve çıkarım kodu, yazarın yönlendirmesiyle Claude (Anthropic) kodlama ajanları tarafından yazıldı; sayılar yazar tarafından doğrulandı.