Ilk yayin: LoRA adaptoru + model karti + karar.py

.gitattributes

@@ -33,3 +33,8 @@ saved_model/**/* filter=lfs diff=lfs merge=lfs -text
 *.zip filter=lfs diff=lfs merge=lfs -text
 *.zst filter=lfs diff=lfs merge=lfs -text
 *tfevents* filter=lfs diff=lfs merge=lfs -text
+assets/arch.png filter=lfs diff=lfs merge=lfs -text
+assets/banner_turnike.png filter=lfs diff=lfs merge=lfs -text
+assets/fig_compare.png filter=lfs diff=lfs merge=lfs -text
+assets/fig_jevbench_delta.png filter=lfs diff=lfs merge=lfs -text
+assets/fig_speed.png filter=lfs diff=lfs merge=lfs -text

README.md

@@ -15,15 +15,30 @@ tags:
 - multiple-choice
 ---
 
-# Karar-TR-4B
+![Turnike-4B](assets/banner_turnike.png)
 
-**A Turkish-first decision layer: give it a text and one or more closed questions, get back a probability for every option. Up to 16 questions are answered in a single forward pass.**
-**Türkçe öncelikli karar katmanı: bir metin ve bir ya da birkaç kapalı soru verin, her seçenek için olasılık alın; 16 soruya kadar tek ileri geçiş.**
+# Turnike-4B
+
+**Turnike-4B: a bilingual (English + Turkish) 4B decision model — text plus closed questions in, calibrated option probabilities out, ~100 ms, fully offline.**
+
+**Turnike-4B: 4B parametreli, iki dilli (İngilizce + Türkçe) bir karar modeli — metin ve kapalı sorular girer, kalibre seçenek olasılıkları çıkar; ~100 ms, tamamen çevrim dışı.**
+
+- **English: most accurate in its class on JevBench.** Macro accuracy **0.774** on the full JevBench test (22 configs, 22,773 rows, official `jevify-run report`). The best 4B entry on the public jev-bench leaderboard has 0.751 and the top entry overall 0.770 (a 35B MoE). Our result has not been submitted to the leaderboard and the 0.004 lead is small. Jev 1.13.0 scores 0.733 on the same set.
+- **Turkish: best on human-labelled decisions.** tr_insan (1,379 human-labelled Turkish decisions): **0.748**, against 0.727 for Jev, 0.714 for Microsoft-Decision-1 (9B, API; paired McNemar p = 0.003), 0.708 for Kev-4B and 0.675 for Intern-Decision-4B.
+- **Calibrated without a recipe.** Macro ECE on full JevBench is **0.084** from raw probabilities, against 0.113 for Jev 1.13.0. Leaderboard models that use a fitted recipe get to about 0.05, so fit a temperature if you need tighter calibration.
+- **Fast.** One decision takes **~109 ms** in bf16 and **~82 ms** in FP8. Eight questions on one text, in one pass, take ~156 / ~126 ms. Measured on one RTX 5060 Ti 16 GB (8.7 GB used), with an earlier run of the same architecture and size.
+- **Many questions, one pass.** Up to 16 questions about one text are answered in a single forward pass.
+- **Runs on your hardware, offline.** You need a ~130 MB LoRA adapter (or the merged bf16 weights) and one consumer GPU. There is no API and the text never leaves the machine. CPU works, but slowly.
+- **Cheap.** On your own hardware there is no per-call fee. For comparison, our logs show Jev's API at about 1.8e-5 $ per one-question request and 3.6e-5 $ per eight-question request, and Microsoft-Decision-1 at 0.7–1.5e-5 $ per request.
+- **Three sizes, one recipe.** Turnike-0.8B, Turnike-4B and Turnike-9B share the data, the draw and the hyperparameters; nothing was tuned per size. See [Model family](#model-family).
+- **Honest.** We found and fixed two training-data shortcuts (see [Training](#training)). "None of the above" is tested on a balanced set whose option count does not give the answer away. All test sets used to pick this run are listed in [Honesty notes](#honesty-notes).
+
+![Five decision models on the same items](assets/fig_compare.png)
 
 ```python
-# pip install torch "transformers>=5.17" peft accelerate ; karar.py: hf download berkay123001/karar-tr-4b karar.py --local-dir .
+# pip install torch "transformers>=5.17" peft accelerate ; karar.py: hf download berkay123001/turnike-4b karar.py --local-dir .
 from karar import Karar
-k = Karar("berkay123001/karar-tr-4b")
+k = Karar("berkay123001/turnike-4b")
 
 # TR
 k.sor("Kulaklık iki günde bozuldu, paramı geri istiyorum.",
@@ -43,25 +58,37 @@ k.sor("The production database has been down since 9am. Please escalate now.",
 > - AI assistance: the card and code were written with Claude coding agents under the author's direction ([details](#ai-assistance)). / Kart ve kod, yazarın yönlendirmesiyle Claude kodlama ajanlarıyla yazıldı.
 > - No web/browser data in this release; the browser version will be published separately. / Bu sürümde web verisi yok; tarayıcı sürümü ayrı yayınlanacak.
 
-Karar-TR-4B is a LoRA adapter (r=16, about 130 MB) on [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). It does not generate text. The text and the questions go into one chat prompt; the assistant turn is a JSON skeleton with one placeholder per question. Every option of a question is mapped to a single-token symbol (A, B, C, ...). At the position just before each placeholder the model's hidden state is multiplied with the output rows of that question's symbols only, and a softmax over those symbols gives the option probabilities.
+Turnike-4B is a LoRA adapter (r=16, about 130 MB) on [Qwen/Qwen3.5-4B](https://huggingface.co/Qwen/Qwen3.5-4B). It does not generate text. The text and the questions go into one chat prompt, and the assistant turn is a JSON skeleton with one placeholder per question. Every option of a question is mapped to a single-token symbol (A, B, C, ...). At the token just before each placeholder, the model's final hidden state is multiplied with the output rows of that question's symbols only. A softmax over those symbols gives the option probabilities.
 
 - Author: Berkay Hasret (independent researcher)
-- Languages: Turkish (primary), English
+- Languages: English and Turkish
 - Base model: Qwen/Qwen3.5-4B (Apache-2.0, Copyright 2026 Alibaba Cloud), frozen during training; only the adapter is trained
 - License of the adapter, the merged weights, `karar.py` and this card: Apache-2.0 (see [License](#license))
-- Merged bf16 weights (no PEFT needed) are published as a separate repo: `berkay123001/karar-tr-4b-merged`
+- Merged bf16 weights (no PEFT needed) are published as a separate repo: `berkay123001/turnike-4b-merged`
+
+<a id="model-family"></a>
+## Model family
+
+Three sizes, one recipe: the same data, the same draw (seed 3) and the same hyperparameters (LoRA r=16, α=32, lr 1e-4, 2,831 steps, effective batch 32) for every size. Nothing was tuned per size, so the smallest and the largest model are probably somewhat under-tuned. Each size is its own repo; the row in bold is the model of this card.
+
+| model | base | trainable params | JevBench EN-300 (EN) | held-6 / other-16 | tr_insan, 1,379 human-labelled (TR) | JevBench TR-50 (TR) | practical TR, 2,966 | LegalBench rules, 885 (EN) | trained on |
+|---|---|---|---|---|---|---|---|---|---|
+| [Turnike-0.8B](https://huggingface.co/berkay123001/turnike-0.8b) | Qwen/Qwen3.5-0.8B | 10.8M | 0.676 | 0.682 / 0.674 | 0.699 | 0.560 | 0.840 | 0.724 | RTX 5060 Ti 16 GB, batch 4 × 8 |
+| **Turnike-4B** (this card) | Qwen/Qwen3.5-4B | 32.5M | 0.769 | 0.826 / 0.748 | 0.748 | 0.700 | 0.891 | 0.832 | RTX 5060 Ti 16 GB, batch 4 × 8 |
+
+<sub>Same protocol as the tables below: single runs, the same items for every model, no calibration recipe. Differences below about 0.01 (0.02 on TR-50) are within run-to-run noise.</sub>
 
 ## Quick start
 
 ```bash
-hf download berkay123001/karar-tr-4b karar.py requirements.txt --local-dir .
+hf download berkay123001/turnike-4b karar.py requirements.txt --local-dir .
 pip install -r requirements.txt     # torch, transformers 5.x, peft, safetensors, accelerate
 # optional: pip install torchao (FP8), causal-conv1d and flash-linear-attention (faster linear-attention layers)
 ```
 
 ```python
 from karar import Karar                                   # karar.py from this repo
-k = Karar("berkay123001/karar-tr-4b")                      # base model is downloaded automatically
+k = Karar("berkay123001/turnike-4b")                     # base model is downloaded automatically
 print(k.sor(
     "Hocam merhaba, BIL204 proje ödevi bu cuma teslim ama geçen hafta ameliyat oldum, raporum ekte. "
     "Teslimi salıya uzatabilir misiniz?",
@@ -71,7 +98,7 @@ print(k.sor(
                              "nötr": "duygu belirtmiyor, yalnızca bilgi aktarıyor"}}))
 ```
 
-`sor(text, questions)` returns `{question: {label: probability}}`; the probabilities of each question sum to 1. A question can be given as
+`sor(text, questions)` returns `{question: {label: probability}}`, and the probabilities of each question sum to 1. A question can be given in any of these forms:
 
 | form | meaning |
 |---|---|
@@ -80,24 +107,79 @@ print(k.sor(
 | `{"type": "noul"}` | yes/no; returns `{"false": p, "true": p}` |
 | `{"type": "choice" \| "score" \| "noul", "instructions": "...", "criteria": {...}}` | full form, as used in our evaluations |
 
-If every key is a short identifier (`"urgent"`, `"iade_istiyor"`), it is used as the field name in the prompt and the question text comes from `instructions`; otherwise the fields are named `q` / `q1..qN` and the key is the question text. More than 16 questions are split into balanced passes; a question with more than 62 options is solved by a two-stage elimination (options that do not reach the final get probability 0).
+If every key is a short identifier (`"urgent"`, `"iade_istiyor"`), it becomes the field name in the prompt and the question text comes from `instructions`. Otherwise the fields are named `q` / `q1..qN` and the key is the question text. More than 16 questions are split into balanced passes. A question with more than 62 options is solved by two-stage elimination; options that do not reach the final get probability 0.
 
 Command line:
 
 ```bash
-python karar.py --adapter berkay123001/karar-tr-4b --metin "..." \
+python karar.py --adapter berkay123001/turnike-4b --metin "..." \
     --soru "Mesajın tonu nedir?|olumlu|olumsuz|nötr" --soru "Cevap bekliyor mu?|evet|hayır" [--fp8]
 ```
 
-Merged weights (no PEFT): `Karar(None, taban="berkay123001/karar-tr-4b-merged")`. They are produced from this adapter by `birlestir.py` (same bf16 merge that `karar.py` does at load time).
+Merged weights (no PEFT): `Karar(None, taban="berkay123001/turnike-4b-merged")`. `birlestir.py` produces them from this adapter with the same bf16 merge that `karar.py` does at load time.
+
+`karar.py` is a single self-contained file (about 400 lines; imports only torch, transformers, peft and optionally torchao). `test_karar.py` has 9 tests:
 
-`karar.py` is a single self-contained file (about 400 lines; imports only torch, transformers, peft and optionally torchao). `test_karar.py` contains 9 tests (input validation, independence from any other code, 9 Turkish/English examples, CLI, long text / many options / many questions, FP8, and, when the evaluation code is available, token-level prompt identity and probability identity with the code used for all numbers below). Measured (bf16, RTX 5060 Ti): max |Δp| 0.00000 against that code on 33 questions, including an 18-question and a 71-option case; FP8 vs bf16 max |Δp| 0.071 on the 9 examples, same argmax.
+- input validation and independence from any other code;
+- 9 Turkish/English examples and the CLI;
+- long text, many options and many questions;
+- FP8;
+- when the evaluation code is available, token-level prompt identity and probability identity with the code behind all numbers below.
+
+Measured (bf16, RTX 5060 Ti): max |Δp| 0.00000 against that code on 33 questions, including an 18-question case and a 71-option case. FP8 against bf16: max |Δp| 0.071 on the 9 examples, with the same argmax.
 
 Tips:
 - Options work best as full candidate answers or with a short description ("olumsuz": "genel olarak olumsuz"), not bare codes.
 - The question wording matters; an explicit rule with examples beats a vague question.
 - Yes/no questions: either `{"type": "noul"}` (the form used in training for most yes/no data) or two descriptive options.
 
+## Method
+
+![Architecture: (a) one call, (b) training, (c) symbol readout](assets/arch.png)
+
+**Readout.** For question $q$ with options $O_q$, let $s(o)$ be the symbol of option $o$, $h_q$ the final hidden state at the token just before $q$'s placeholder, and $w_s$ the output-head row of symbol $s$. Then
+
+$$
+p_q(o) \;=\; \frac{\exp\!\big(w_{s(o)}^{\top} h_q\big)}{\sum_{o' \in O_q} \exp\!\big(w_{s(o')}^{\top} h_q\big)}, \qquad |O_q| \le 62 .
+$$
+
+Only the $|O_q|$ symbol rows are multiplied (in fp32). The 248,320-token vocabulary is never scored, and no temperature is applied ($T = 1$).
+
+**Packing.** One prompt holds $Q \le 16$ questions about the same text, and one forward pass yields all $h_{q_1}, \dots, h_{q_Q}$. Attention is causal, so a question's readout sees the text, the full decision schema and the `?` placeholders of the earlier fields. It does not see their answers.
+
+**Training loss.** Each question has a target distribution $y_q$: the gold option smoothed to 0.9, or the crowd-label distribution where the source has one. Over all questions $\mathcal{B}$ in a mini-batch:
+
+$$
+\mathcal{L} \;=\; \frac{1}{|\mathcal{B}|} \sum_{q \in \mathcal{B}} \mathrm{KL}\!\left(y_q \,\middle\|\, p_q\right)
+\;=\; \frac{1}{|\mathcal{B}|} \sum_{q \in \mathcal{B}} \sum_{o \in O_q} y_q(o) \big[\log y_q(o) - \log p_q(o)\big].
+$$
+
+There is no per-question weight in the loss. Group shares, class balancing (α = 0.5) and the 0.08 share of broad-template questions act on how often an item is drawn, not on $\mathcal{L}$.
+
+```text
+Algorithm 1   One Turnike-4B call: sor(text, questions)
+Input : text x; questions q_1..q_N, each with options O_q (labels, optional descriptions)
+Output: for every q, a probability for every option in O_q
+
+ 1  validate the inputs; write every question in the full form {type, instructions, criteria}
+ 2  questions with |O_q| > 62 go to line 10; split the rest into ceil(N/16) balanced packs
+ 3  for each pack P:
+ 4      prompt <- system message
+               + "## State" + x as a JSON string
+               + "## Decision schema": per q, "field: question" then "A = label: description", ...
+               + assistant skeleton  { "q1": "?", "q2": "?", ... }    (x truncated if > 4,096 tokens)
+ 5      run ONE forward pass of base + LoRA (no generation)
+ 6      for each q in P:
+ 7          h_q <- final hidden state at the token just before q's "?"
+ 8          z_q <- W_sym[symbols of q] . h_q                (<= 62 rows, fp32)
+ 9          p_q <- softmax(z_q)                              (T = 1; any temperature is up to the caller)
+10  for each q with |O_q| > 62:
+        k <- ceil(|O_q| / 62); split O_q into k balanced chunks; one pass per chunk;
+        keep the top floor(62/k) options of every chunk; one final pass over the finalists;
+        options that are not finalists get probability 0
+11  return {q: {label: p_q(label)}}
+```
+
 ## Prompt format
 
 The prompt is the Qwen3.5 chat template with an empty thinking block. The system message and the user-message layout are fixed and must not be changed:
@@ -115,68 +197,187 @@ user:   Return one answer for every field using the supplied answer symbols.
             B = <label>: <description>
         q2: ...
 assistant: {
-            "q1": ?,      <- probabilities are read at the token before each placeholder
-            "q2": ?
+            "q1": "?",      <- probabilities are read at the token before each "?" placeholder
+            "q2": "?"
         }
 ```
 
-Yes/no questions are written as `A = no: The answer is no (...)`, `B = yes: The answer is yes (...)`. The text is truncated from the end if the prompt exceeds `MAKS_ISTEM_TOKEN = 4096` tokens (training used 1536); questions and options are never truncated.
+Yes/no questions are written as `A = no: The answer is no (...)`, `B = yes: The answer is yes (...)`. If the prompt exceeds `MAKS_ISTEM_TOKEN = 4096` tokens (training used 1536), the text is truncated from the end. Questions and options are never truncated.
 
 ## Results
 
-All numbers were measured by us, on the same items, with the same prompt format for all Qwen3.5-based models. "Karar-TR-4B" is training run *temiz5* ("temiz" = clean). The previous clean runs and our earlier Jev-distilled model are listed for comparison. Raw probabilities, no calibration recipe.
+All numbers below are our own measurements unless marked as reported by someone else. Every model saw the same items and was scored by the same code. Probabilities are raw, with no calibration recipe. Each result is a single run.
+
+| model | JevBench EN-300 (EN) | LegalBench rules, 885 (EN) | tr_insan, 1,379 human-labelled (TR) | JevBench TR-50 (TR) | practical TR, 2,966 (TR) |
+|---|---|---|---|---|---|
+| **Turnike-4B** | 0.769 | 0.832 | 0.748 | 0.700 | 0.891 |
+| **Turnike-0.8B** | 0.676 | 0.724 | 0.699 | 0.560 | 0.840 |
+| Microsoft-Decision-1 (9B, closed, API) | 0.775 | – | 0.714 | 0.745 | 0.857 |
+| Jev 1.13.0 (API) | – | 0.924 † | 0.727 | – | 0.872 |
+| Kev-4B | 0.706 | 0.786 | 0.708 | 0.655 | 0.797 |
+| jevify Qwen3.5-4B readout LoRA + coherence | – | – | 0.642 | 0.687 | 0.847 |
+| internlm/Intern-Decision-4B | 0.687 | 0.656 | 0.675 | 0.624 | 0.843 |
+| OpenJev 4B (qwen3.5-4b-nli-v5) | – | – | 0.645 | 0.621 | 0.738 |
+| Qwen3.5-4B, zero-shot (our prompt) | – | – | 0.612 | – | 0.745 |
+| Kev-0.8B | 0.570 | 0.496 | 0.583 | 0.508 | 0.697 |
 
-| model | tr_insan (1,379 human-labelled TR decisions) | practical TR (2,966) | JevBench EN-300 macro | JevBench TR-50 macro | KVKK errors (of 1,172) ↓ | LegalBench rules (885) |
+† Reported by the jevify author on the same 885 items (jevify README, finding 14); not our run. – = not measured. Run-to-run noise: retraining an earlier run of the same recipe with another seed moved tr_insan by +0.001, EN-300 by +0.002 and TR-50 by +0.020.
+
+### English
+
+#### Table 1. Full JevBench test, next to the public leaderboard
+
+| model | source | macro acc | macro ECE ↓ | macro Brier ↓ | sel@90 | choice / score / noul acc |
 |---|---|---|---|---|---|---|
-| **Karar-TR-4B (temiz5)** | **0.748** | **0.891** | **0.769** | **0.700** | **2** | **0.832** |
-| temiz3 (same data, sibling-shortcut bug, see Limitations) | 0.753 | 0.892 | 0.764 | 0.692 | 4 | 0.838 |
-| temiz2 (40k items, seed 2) | 0.744 | 0.888 | 0.743 | 0.667 | 11 | 0.791 |
-| our v23 (Qwen3-Reranker-4B, **distilled from Jev**) | 0.745 | 0.857 | 0.732 | 0.671 | 12 | 0.753 |
-| Microsoft-Decision-1 (9B, closed, API) | 0.714 | 0.857 | 0.775 | 0.745 | – | – |
-| internlm/Intern-Decision-4B | 0.675 | 0.843 | 0.687 | 0.624 | – | 0.656 |
-| seed repeat (temiz2 seed 3 − seed 2) | +0.001 | −0.002 | +0.002 | +0.020 | +5 | +0.015 |
-
-Full JevBench test (22,773 rows, official `jevify-run report`; macro = mean of the 22 config accuracies, equal weight): **Karar-TR-4B macro accuracy 0.774**, macro ECE 0.084 (no calibration recipe; v23 0.099, Jev 1.13.0 0.113), held-6 0.830 / other-16 0.752, TVD→human 0.318 (Jev 0.432); v23 0.733, Jev 1.13.0 0.733 (our own run of the TypeSafe API on the full test set). On the public jev-bench leaderboard (46 entries), 0.774 would place first by a 0.004 margin over the top entry (a 35B MoE); the margin is small and the result has not been submitted to the maintainers. On the 300-row sample the gap to Microsoft-Decision-1 is not significant (see below).
-
-Behaviour and calibration (same runs):
-
-| model | "none of the above", balanced test: overall / "none" when the right option is missing / "none" when it is present (↓) | hijack rate, planted instruction (↓) | EN-300 ECE ↓ | EN-300, 6 held-out sources |
-|---|---|---|---|---|
-| **Karar-TR-4B (temiz5)** | **0.74–0.77 / 0.80–0.81 / 0.22–0.25** | **−0.014** | **0.087** | **0.826** |
-| temiz3 | 0.41–0.47 / 0.01–0.08 / 0.00 | −0.007 | 0.091 | 0.813 |
-| temiz2 | 0.49–0.67 / 0.16–0.51 / 0.01–0.07 | −0.001 | 0.093 | 0.798 |
-| v23 (Jev-distilled) | 0.61–0.63 / 0.52 / 0.11–0.13 | +0.000 | 0.104 | 0.765 |
-| Microsoft-Decision-1 | – | – | 0.079 | 0.843 |
-| Intern-Decision-4B | – | +0.279 | 0.115 | 0.783 |
-
-Protocol:
-- **tr_insan**: 1,379 Turkish decisions with human gold labels from five public sets (topic, film stars, product stars, 3-way sentiment, NLI), test texts only. Our only human-labelled Turkish test.
-- **practical TR**: 50 plain-language questions × 60 Turkish texts, 2,966 decisions; gold labels come from an LLM, not from people.
-- **JevBench EN-300**: the same 300 rows per config (seed 7) from the JevBench test split, 22 configs, 6,600 rows; the six sources clinc150, arc_challenge, yelp5, measuring_hate_speech, fever_evidence and strategyqa_grounded are not in training. The other 16 configs' *train* splits are in training, so they are in-distribution.
-- **JevBench TR-50**: 50 rows per config from a machine-translated Turkish JevBench test, 1,100 rows; small and noisy.
-- **KVKK**: an internal Turkish personal-data (KVKK) sufficiency check, two yes/no questions (legal basis present, data controller present) on 1,172 texts: 31 gold-labelled items plus 1,141 items that must be answered "insufficient". Reported as total errors.
-- **LegalBench rules**: 885 LegalBench fact patterns with a stated rule (10 tasks, none trained on); **hijack** is our behaviour test (800 items × clean + planted-instruction copy). **None of the above, balanced:** 908 MMLU/ARC items, half with the right option removed; in two variants the option count no longer reveals the answer (a distractor dropped from items that keep the right option, or one added to items that lost it). Ranges are over the two variants. In the "added option" variant the added option is off-topic only in items without the right option, which can itself hint "none"; the "dropped option" variant has no such hint.
-- **Seed repeat**: temiz2 retrained with seed 3 (also a different batch layout and measurement machine). One pair only, so read it as the order of magnitude of run-to-run noise, not as a bound. TR-50 moves by 0.02 between seeds.
-- Significance (temiz3, exact paired McNemar): vs temiz2 EN-300 p = 6e-8, TR-50 p = 0.023, tr_insan p = 0.33 (n.s.); vs v23 EN-300 p = 3e-11, tr_insan p = 0.44 (n.s.); vs Microsoft-Decision-1 EN-300 p = 0.014 (MSD1 ahead), tr_insan p = 3e-4 (temiz3 ahead).
-- Significance (temiz5, exact paired McNemar): vs temiz4 EN-300 p = 0.64, tr_insan p = 0.32 (both n.s.); vs v23 EN-300 p = 6.8e-14 (Karar-TR-4B ahead), tr_insan p = 0.78 (n.s.); vs Microsoft-Decision-1 EN-300 p = 0.20 (n.s.; 0.769 vs 0.775), tr_insan p = 0.0026 (Karar-TR-4B ahead); balanced "none of the above" vs v23 p = 3e-27 (Karar-TR-4B ahead).
-
-### Speed
-
-One RTX 5060 Ti 16 GB, `causal-conv1d` and `flash-linear-attention` installed, measured with the temiz3 adapter (same architecture and size; temiz5 not re-measured):
-
-| | 1 question (p50) | 8 questions on one text (p50) | GPU memory |
+| **Turnike-4B** | our run, not submitted | **0.774** | 0.084 | 0.293 | **0.805** | 0.791 / 0.599 / 0.902 |
+| Qwen/Qwen-AgentWorld-35B-A3B (readout LoRA + coherence) | reported by the leaderboard | 0.770 | 0.055 | 0.292 | 0.799 | 0.792 / 0.566 / 0.917 |
+| Qwen/Qwen3.5-9B (readout LoRA + coherence) | reported by the leaderboard | 0.763 | 0.056 | 0.299 | 0.791 | 0.791 / 0.554 / 0.906 |
+| Qwen/Qwen3.5-9B (readout LoRA) | reported by the leaderboard | 0.759 | 0.052 | 0.302 | 0.787 | 0.790 / 0.541 / 0.905 |
+| Qwen/Qwen3.5-4B (readout LoRA + coherence) | reported by the leaderboard | 0.751 | 0.058 | 0.316 | 0.779 | 0.771 / 0.555 / 0.893 |
+| perplexity-ai/pplx-decider-v1-27b (hosted API) | reported by the leaderboard | 0.748 | 0.080 | 0.318 | 0.774 | 0.775 / 0.519 / 0.911 |
+| Qwen/Qwen3.5-4B-Base (readout LoRA + coherence) | reported by the leaderboard | 0.747 | 0.058 | 0.318 | 0.777 | 0.771 / 0.535 / 0.897 |
+| Qwen/Qwen3.5-4B (Tier 2 residual) | reported by the leaderboard | 0.747 | 0.110 | 0.342 | 0.774 | 0.770 / 0.529 / 0.903 |
+| Qwen/Qwen3.5-4B (readout LoRA) | reported by the leaderboard | 0.743 | 0.059 | 0.321 | 0.772 | 0.766 / 0.530 / 0.896 |
+| google/gemma-4-E4B-it (readout LoRA + coherence) | reported by the leaderboard | 0.742 | 0.053 | 0.322 | 0.772 | 0.759 / 0.546 / 0.889 |
+| Qwen/Qwen3.5-4B-Base (readout LoRA) | reported by the leaderboard | 0.741 | 0.064 | 0.324 | 0.770 | 0.768 / 0.522 / 0.893 |
+| google/gemma-4-E4B-it (readout LoRA) | reported by the leaderboard | 0.737 | 0.057 | 0.330 | 0.766 | 0.754 / 0.531 / 0.892 |
+| Qwen/Qwen3.5-4B (Tier 2 residual, lr 3e-05) | reported by the leaderboard | 0.734 | 0.096 | 0.342 | 0.762 | 0.761 / 0.518 / 0.884 |
+| Jev 1.13.0 (TypeSafe API) | reported by the leaderboard | 0.733 | 0.113 | 0.349 | 0.760 | 0.770 / 0.503 / 0.881 |
+| google/gemma-4-E4B (readout LoRA) | reported by the leaderboard | 0.708 | 0.061 | 0.359 | 0.736 | 0.717 / 0.502 / 0.874 |
+| togethercomputer/Tev1-4B-experimental (Tev1 prompt) | reported by the leaderboard | 0.703 | 0.086 | 0.361 | 0.732 | 0.717 / 0.512 / 0.850 |
+| Qwen/Qwen3.5-4B (Tier 1 residual) | reported by the leaderboard | 0.698 | 0.089 | 0.378 | 0.729 | 0.699 / 0.507 / 0.862 |
+| Qwen/Qwen3.5-4B (Tier 0, no training) | reported by the leaderboard | 0.662 | 0.090 | 0.401 | 0.689 | 0.687 / 0.468 / 0.796 |
+
+<sub>Leaderboard rows: the public jev-bench leaderboard (Praveenrajus/jev-bench dataset card, data v0.1.1, 46 rows), values as published: the top 10, the other 4B-class entries with macro ≥ 0.66, and the Jev reference row. Per-config values for these rows are not available to us. Ours: full test split (22,773 rows), official `jevify-run report`, argmax and yes/no at p ≥ 0.5, raw probabilities, one deterministic run. The lead over the top row is small and has not been checked by the leaderboard maintainers; leaderboard models report lower ECE (≈0.05).</sub>
+
+![Full JevBench, per config: Turnike-4B − Jev 1.13.0](assets/fig_jevbench_delta.png)
+
+#### Table 2. Full JevBench, per config, against Jev 1.13.0
+
+| config | type | n | Turnike-4B acc | Turnike-4B ECE | Jev 1.13.0 acc | Jev 1.13.0 ECE | Turnike-4B − Jev |
+|---|---|---|---|---|---|---|---|
+| *held-out 6 (not in training)* | | | | | | | |
+| `arc_challenge` | choice | 1,000 | 0.932 | 0.087 | 0.979 | 0.010 | −0.047 |
+| `clinc150` | choice | 1,000 | 0.856 | 0.162 | 0.893 | 0.033 | −0.037 |
+| `fever_evidence` | noul | 1,000 | 0.952 | 0.066 | 0.972 | 0.028 | −0.020 |
+| `measuring_hate_speech` | score | 1,000 | 0.631 | 0.070 | 0.527 | 0.237 | +0.104 |
+| `strategyqa_grounded` | noul | 687 | 0.943 | 0.118 | 0.956 | 0.059 | −0.013 |
+| `yelp5` | score | 1,000 | 0.668 | 0.061 | 0.685 | 0.174 | −0.017 |
+| *held-out 6 (not in training), macro* | | 5,687 | **0.830** | 0.094 | 0.835 | 0.090 | −0.005 |
+| *other 16 (train splits in training)* | | | | | | | |
+| `banking77` | choice | 1,000 | 0.862 | 0.100 | 0.796 | 0.095 | +0.066 |
+| `boolq` | noul | 1,000 | 0.904 | 0.085 | 0.917 | 0.021 | −0.013 |
+| `chaosnli` | choice | 1,599 | 0.631 | 0.108 | 0.615 | 0.222 | +0.016 |
+| `civil_comments` | noul | 2,000 | 0.919 | 0.086 | 0.729 | 0.045 | +0.191 |
+| `go_emotions` | choice | 1,000 | 0.569 | 0.032 | 0.282 | 0.384 | +0.287 |
+| `helpsteer2_helpfulness` | score | 1,000 | 0.446 | 0.025 | 0.363 | 0.232 | +0.083 |
+| `helpsteer2_verbosity` | score | 1,000 | 0.679 | 0.160 | 0.341 | 0.231 | +0.338 |
+| `ledgar` | choice | 1,000 | 0.775 | 0.099 | 0.751 | 0.117 | +0.024 |
+| `massive` | choice | 1,000 | 0.857 | 0.099 | 0.808 | 0.090 | +0.049 |
+| `mmlu` | choice | 1,000 | 0.729 | 0.075 | 0.923 | 0.027 | −0.194 |
+| `mnli` | choice | 1,000 | 0.905 | 0.085 | 0.883 | 0.032 | +0.022 |
+| `paws` | noul | 1,000 | 0.909 | 0.079 | 0.846 | 0.040 | +0.063 |
+| `sms_spam` | noul | 800 | 0.983 | 0.093 | 0.965 | 0.068 | +0.018 |
+| `sst5` | score | 1,000 | 0.582 | 0.059 | 0.565 | 0.190 | +0.017 |
+| `strategyqa_closed` | noul | 687 | 0.703 | 0.055 | 0.785 | 0.042 | −0.082 |
+| `stsb` | score | 1,000 | 0.585 | 0.048 | 0.538 | 0.119 | +0.047 |
+| *other 16 (train splits in training), macro* | | 17,086 | **0.752** | 0.081 | 0.694 | 0.122 | +0.058 |
+| **all 22, macro** | | 22,773 | **0.774** | **0.084** | 0.733 | 0.113 | +0.041 |
+
+<sub>Jev values: the Jev 1.13.0 per-config table on the jev-bench dataset card (TypeSafe API, every test record); our project records hold the same accuracies for all 22 configs. Turnike-4B is ahead of Jev on 14 of 22 configs and by +0.041 macro. On the six held-out sources Turnike-4B (0.830) is above the leaderboard author's best open model (0.828, as stated on the card) and behind Jev (0.835) and the hosted pplx-decider-v1-27b (0.840). Jev also leads on the knowledge configs (mmlu, strategyqa_closed, arc_challenge).</sub>
+
+#### Table 3. JevBench EN-300: the same 6,600 rows for every model
+
+| model | EN-300 macro | held-6 / other-16 | McNemar vs Turnike-4B (only-Turnike-4B / only-other) |
+|---|---|---|---|
+| **Turnike-4B** | 0.769 | 0.826 / 0.748 | – |
+| **Turnike-0.8B** | 0.676 | 0.682 / 0.674 | 993 / 378, p = 6e-64 |
+| Microsoft-Decision-1 (9B, closed, API) | 0.775 | 0.843 / 0.750 | 426 / 465, p = 0.20 |
+| Kev-4B | 0.706 | 0.772 / 0.682 | 795 / 381, p = 4e-34 |
+| internlm/Intern-Decision-4B | 0.687 | 0.783 / 0.651 | 901 / 362, p = 2e-53 |
+| Kev-0.8B | 0.570 | 0.586 / 0.564 | 1666 / 349, p = 4e-205 |
+
+<sub>Protocol: 300 rows per config from the JevBench test split (seed 7), 22 configs, the same 6,600 rows for every model; exact two-sided McNemar on paired rows. Not measured on this subset: Jev 1.13.0 (API), jevify Qwen3.5-4B readout LoRA + coherence, OpenJev 4B (qwen3.5-4b-nli-v5), Qwen3.5-4B, zero-shot (our prompt) (Jev: see Tables 1–2). We do not know whether Microsoft-Decision-1 was trained on JevBench. Intern-Decision-4B used its own prompt and calibration temperature (for more than 62 options we added the same elimination step), Kev its own HTTP server. EN-300 macro ECE: Turnike-4B 0.087, Microsoft-Decision-1 0.079, Intern-Decision-4B 0.115, Kev-4B 0.135.</sub>
+
+### Turkish
+
+#### Table 4. Turkish human gold (tr_insan), per task
+
+| model | topic (301) | sentiment 3-way (300) | NLI (300) | film stars 1–5 (178) | product stars 1–5 (300) | all (1,379) | McNemar vs Turnike-4B |
+|---|---|---|---|---|---|---|---|
+| **Turnike-4B** | 0.924 | 0.910 | 0.790 | 0.562 | 0.480 | **0.748** | – |
+| **Turnike-0.8B** | 0.884 | 0.870 | 0.727 | 0.472 | 0.450 | **0.699** | 152 / 84, p = 1e-5 |
+| Microsoft-Decision-1 (9B, closed, API) | 0.887 | 0.893 | 0.727 | 0.545 | 0.450 | **0.714** | 141 / 94, p = 0.003 |
+| Jev 1.13.0 (API) | 0.904 | 0.893 | 0.777 | 0.517 | 0.457 | **0.727** | 131 / 101, p = 0.06 |
+| Kev-4B | 0.887 | 0.860 | 0.737 | 0.528 | 0.453 | **0.708** | 126 / 70, p = 8e-5 |
+| jevify Qwen3.5-4B readout LoRA + coherence | 0.844 | 0.823 | 0.747 | 0.337 | 0.333 | **0.642** | 222 / 75, p = 5e-18 |
+| internlm/Intern-Decision-4B | 0.867 | 0.860 | 0.707 | 0.455 | 0.397 | **0.675** | 172 / 71, p = 8e-11 |
+| OpenJev 4B (qwen3.5-4b-nli-v5) | 0.804 | 0.837 | 0.683 | 0.455 | 0.370 | **0.645** | 253 / 111, p = 7e-14 |
+| Qwen3.5-4B, zero-shot (our prompt) | 0.850 | 0.850 | 0.540 | 0.326 | 0.377 | **0.612** | 280 / 92, p = 3e-23 |
+| Kev-0.8B | 0.841 | 0.693 | 0.630 | 0.371 | 0.293 | **0.583** | 309 / 81, p = 2e-32 |
+
+<sub>Protocol: human gold labels from five public Turkish datasets, test texts only. All models were scored by the same script against the same gold file; no model answered "unsure". Stars are exact match on a 5-point scale (within ±1 star Turnike-4B gets 0.91 / 0.90). Our leakage scan flagged 19 items as overlapping with training texts; without them Turnike-4B gets 0.746. Microsoft-Decision-1 and Jev were called through their APIs, the others ran locally.</sub>
+
+#### Table 5. JevBench TR-50: the same 1,100 machine-translated rows for every model
+
+| model | TR-50 macro | held-6 / other-16 | McNemar vs Turnike-4B (only-Turnike-4B / only-other) |
 |---|---|---|---|
-| bf16 | 109 ms | 156 ms | 8.7 GB |
-| FP8 (`--fp8`, torchao) | 82 ms | 126 ms | 8.8 GB |
+| **Turnike-4B** | 0.700 | 0.750 / 0.681 | – |
+| **Turnike-0.8B** | 0.560 | 0.583 / 0.551 | 239 / 85, p = 4e-18 |
+| Microsoft-Decision-1 (9B, closed, API) | 0.745 | 0.840 / 0.709 | 72 / 121, p = 5e-4 |
+| Kev-4B | 0.655 | 0.720 / 0.630 | 127 / 77, p = 6e-4 |
+| jevify Qwen3.5-4B readout LoRA + coherence | 0.687 | 0.743 / 0.666 | 106 / 92, p = 0.36 |
+| internlm/Intern-Decision-4B | 0.624 | 0.700 / 0.595 | 143 / 59, p = 3e-9 |
+| OpenJev 4B (qwen3.5-4b-nli-v5) | 0.621 | 0.717 / 0.585 | 172 / 85, p = 6e-8 |
+| Kev-0.8B | 0.508 | 0.580 / 0.481 | 295 / 84, p = 1e-28 |
+
+<sub>Protocol: 50 rows per config (seed 7) from a machine-translated Turkish JevBench test, 22 configs, 1,100 rows. The set is small and noisy: it moved by 0.02 between two seeds of the same recipe. Exact two-sided McNemar on paired rows; Microsoft-Decision-1 rescored from its raw API answers with the same decision rule (macro 0.7445, as originally reported).</sub>
+
+### Behaviour tests (Table 6)
+
+| test | **Turnike-4B** | **Turnike-0.8B** | Jev 1.13.0 (API) | Kev-4B | internlm/Intern-Decision-4B | Kev-0.8B |
+|---|---|---|---|---|---|---|
+| planted instruction: accuracy clean / planted (800 × 2) | 0.927 / 0.941 | 0.885 / 0.892 | – | 0.897 / 0.816 | 0.877 / 0.599 | 0.826 / 0.568 |
+| hijack rate ↓ | −0.014 | −0.008 | – | +0.081 | +0.279 | +0.259 |
+| LegalBench rule application (885, 10 tasks, none trained on) | 0.832 | 0.724 | 0.924 † | 0.786 | 0.656 | 0.496 |
+
+<sub>Protocol: one run each, same items and scoring code. Planted instruction: the same 800 items with a sentence inserted into the text that names a wrong answer; hijack rate = how much more often the model gives the named answer than on the clean copy (0 = ignored). † Reported by the jevify author on the same 885 items; not our run.</sub>
+
+Measured for the Turnike models only (the competitors were not run on these two tests):
+
+- **Turnike-4B, "None of the above", balanced** (908 MMLU/ARC items × 2 variants, half with the right option removed; the option count no longer gives the answer away): accuracy 0.768 (drop variant) / 0.742 (add variant). It picks "none" in 0.809 / 0.798 of the items where the right option is missing, and wrongly in 0.223 / 0.252 where it is present. In the "add" variant the added option is off-topic only in items without the right option, which can itself hint "none"; the "drop" variant has no such hint.
+- **Turnike-4B, KVKK sufficiency** (internal Turkish personal-data check; two yes/no questions: is a legal basis stated, is the data controller named; 1,172 decisions): 2 errors. On the real policy texts it is 1,160/1,160 correct (named controller 9/9, legal basis 10/10, must-be-insufficient 1,141/1,141); all 2 errors are on the 12 short synthetic passages (titles, headings, one-sentence statements) of the controller question (10/12). Gold sets are tiny; two seeds of the 4B recipe gave 11 and 16 errors.
+
+- **Turnike-0.8B, "None of the above", balanced** (908 MMLU/ARC items × 2 variants, half with the right option removed; the option count no longer gives the answer away): accuracy 0.558 (drop variant) / 0.532 (add variant). It picks "none" in 0.666 / 0.653 of the items where the right option is missing, and wrongly in 0.402 / 0.428 where it is present. In the "add" variant the added option is off-topic only in items without the right option, which can itself hint "none"; the "drop" variant has no such hint.
+- **Turnike-0.8B, KVKK sufficiency** (internal Turkish personal-data check; two yes/no questions: is a legal basis stated, is the data controller named; 1,172 decisions): 6 errors. On the real policy texts it is 1,158/1,160 correct (named controller 7/9, legal basis 10/10, must-be-insufficient 1,141/1,141); errors by set: A-controller 2/9, S 4/12. Gold sets are tiny; two seeds of the 4B recipe gave 11 and 16 errors.
+
+### Latency and cost (Table 7)
+
+| model | where | 1 question, p50 | 8 questions on one text, p50 | GPU memory | cost per call |
+|---|---|---|---|---|---|
+| **Turnike-4B**, bf16, optional kernels | RTX 5060 Ti 16 GB | **109 ms** | **156 ms** | 8.7 GB | none (own hardware) |
+| **Turnike-4B**, FP8 (`--fp8`), optional kernels | RTX 5060 Ti 16 GB | **82 ms** | **126 ms** | 8.8 GB | none |
+| Turnike-4B architecture, bf16, no optional kernels | RTX 5060 Ti 16 GB | 120 ms | 173 ms | 8.7 GB | none |
+| Microsoft-Decision-1 | API, network included | 913 ms (tr_insan run, n = 1,379) | – | – | ≈0.7e-5–1.5e-5 $ (0.042 $/M input tokens) |
+| Jev 1.13.0 | API, network included | 592 ms (n = 42) | 386 ms (n = 44) | – | ≈1.8e-5 $ (1 q) / ≈3.6e-5 $ (8 q) |
+| Kev-4B, its own HTTP server | RTX 5060 Ti 16 GB | – | 282 ms | – | none |
+| Intern-Decision-4B, bf16, no optional kernels | RTX 5060 Ti 16 GB | 121 ms | 173 ms | 8.7 GB | none |
+| Qwen3.5-4B zero-shot, bf16, no optional kernels | RTX 5060 Ti 16 GB | 120 ms | 173 ms | 8.7 GB | none |
+
+![Accuracy on Turkish human gold vs latency](assets/fig_speed.png)
 
-FP8 kept accuracy on tr_insan (0.7505 → 0.7513, McNemar p = 1.0), EN-300 (0.7641 → 0.7641) and KVKK (no decision changed), but about 2–2.5% of individual decisions flip (mean |Δp| 0.017, max 0.34; mostly star ratings and NLI). Use bf16 if you threshold on probabilities. Without the optional kernels the linear-attention layers run in plain PyTorch: same results, slower.
+<sub>Protocol: local rows are wall-clock `decide()` time, CUDA-synchronised, 2 warm-up calls dropped. "1 question" is a 7-option topic question over 60 tr_insan texts; "8 questions" is 15 messages × 8 support-desk questions in one call. The Turnike-4B kernel and FP8 rows were measured with an earlier run of the same architecture and size, with `causal-conv1d` and `flash-linear-attention` installed. The rows marked "no optional kernels" come from one session without them. Kev-4B: 8-question p50 through its own HTTP server (our earlier run). API latencies come from separate runs and include the network round trip; Jev from our demo logs, Microsoft-Decision-1 with 4 parallel requests, cost = our spend / requests. FP8 kept accuracy on tr_insan (0.7505 → 0.7513, McNemar p = 1.0) and EN-300 (0.7641 → 0.7641), but about 2–2.5% of individual decisions flip (mean |Δp| 0.017, max 0.34); use bf16 if you threshold on probabilities.</sub>
 
 ## Training
 
-- **Objective:** KL divergence between the model's option distribution and the target distribution, per question. Hard labels are smoothed (0.9 on the gold option); crowd-label distributions are used where the source has them.
-- **Items:** each training item is one text with 8–16 questions packed into one prompt (exactly the inference format), sampled from the pool with per-group shares (about 47% English), max length 1536 tokens, seed 3.
-- **Adapter:** LoRA r=16, alpha=32, dropout 0.05 on all attention, MLP and linear-attention projections of the language model (vision tower untouched, not used); lr 1e-4, warm-up 3%, effective batch 32, 2,831 steps on one RTX 5060 Ti 16 GB (batch 4 × accumulation 8).
-- **Shortcut history:** two training-data shortcuts were found and removed on the way: temiz3 could read a "none of the above" answer from a sibling question in the same packed item (fixed in temiz4), and temiz4 inferred "none" from the option count because items with the right option removed had one option fewer (fixed in temiz5: option count and any added off-topic option are now independent of the answer; 7,585 rows rebuilt).
+- **Objective:** KL divergence between the target distribution and the model's option distribution, per question, averaged over the questions in a batch (see [Method](#method)). Hard labels are smoothed to 0.9 on the gold option; crowd-label distributions are used where the source has them.
+- **Items:** each training item is one text with 8–16 questions packed into one prompt, exactly the inference format. Items are drawn from the pool with per-group shares (about 47% English), max length 1,536 tokens, seed 3. If two questions have option lists that reveal each other's answer, they never share a prompt.
+- **Adapter:** LoRA r=16, alpha=32, dropout 0.05 on all attention, MLP and linear-attention projections of the language model; the vision tower is untouched and not used. lr 1e-4, warm-up 3%, effective batch 32, 2,831 steps on one RTX 5060 Ti 16 GB (batch 4 × accumulation 8).
+- **Shortcut history:** we found and removed two training-data shortcuts.
+  - In an earlier run, the model could read a "none of the above" answer from a sibling question in the same packed item. Fix: such questions are now split into separate prompts.
+  - The next run inferred "none" from the option count, because items with the right option removed had one option fewer. Fix in this release: the option count and any added off-topic option are independent of the answer (7,585 rows rebuilt).
+- **Family:** Turnike-0.8B (Qwen3.5-0.8B) and Turnike-9B (Qwen3.5-9B) were trained with exactly this recipe and this draw, with no per-size tuning; the 9B ran on a Colab RTX PRO 6000 96 GB with batch 16 × accumulation 2 (the same effective batch of 32).
+- An earlier internal model distilled from a commercial API was not released; Turnike-4B is trained without such labels.
 
 <a id="training-data"></a>
 ## Training data
@@ -189,20 +390,23 @@ This release contains no web/browser data; the browser version will be published
 
 ## Limitations
 
-- **"None of the above" over-used somewhat.** On the balanced test the model scores 0.74–0.77 (v23 0.61–0.63) and picks "none" in 0.80–0.81 of the items where the right option is missing, but also in 0.22–0.25 of the items where it is present. Our older "none of the above" test is not reported because its option counts reveal the answer.
-- **KVKK thresholds are noisy.** temiz5 makes 2 errors of 1,172 (temiz4 9, temiz3 4); the same recipe with two seeds gave 11 and 16, so differences of this size are noise. The gold sets are tiny (9, 10, 12 items) and a few items sit near p = 0.5; FP8 moved one controller item from 0.51 to 0.66 without changing any decision. Do not threshold yes/no probabilities near 0.5 without your own validation set.
-- **go_emotions and hate speech:** on EN-300 the model scores 0.53 on go_emotions and 0.62 on measuring_hate_speech; the boundary between offensive language and group-targeted hate is the weakest point, as is 5-level star rating (score type 0.59).
-- **Mostly LLM labels.** Much of the training data is LLM-labelled, so the model inherits those judgements; tr_insan is the only human-labelled Turkish test, and the practical-question gold labels are themselves LLM labels.
-- **Knowledge questions** come from the backbone; the model is meant to judge a given text, not to recall facts.
-- **Calibration:** no temperature or recipe is applied; EN-300 ECE is about 0.09. Fit a temperature on your own validation data if you need calibrated probabilities.
-- Prompts longer than 4096 tokens are truncated (text only); training used 1536 tokens. More than 62 options use an elimination step (two or more passes).
+- **"None of the above" is used somewhat too often.** On the balanced test the model picks "none" in 0.80–0.81 of the items where the right option is missing, but also in 0.22–0.25 of the items where it is present.
+- **Not the best everywhere.** Microsoft-Decision-1 (9B) is ahead on JevBench TR-50 (0.745 vs 0.700, p = 5e-4) and slightly ahead on EN-300 (n.s.). Jev is ahead on the six held-out JevBench sources, on knowledge questions (mmlu 0.923 vs 0.729) and on LegalBench rule application (0.924 vs 0.832).
+- **KVKK thresholds are noisy.** The gold sets are tiny (9, 10, 12 items) and a few items sit near p = 0.5. Do not threshold yes/no probabilities near 0.5 without your own validation set.
+- **go_emotions, hate speech, star ratings:** on full JevBench the model scores 0.569 on go_emotions and 0.631 on measuring_hate_speech. The boundary between offensive language and group-targeted hate is the weakest point, together with 5-level star ratings.
+- **Mostly LLM labels.** Much of the training data is LLM-labelled, so the model inherits those judgements. tr_insan is the only human-labelled Turkish test.
+- **Knowledge questions** come from the backbone. The model is meant to judge a given text, not to recall facts.
+- **Calibration:** no temperature or recipe is applied (macro ECE 0.084 on full JevBench). Fit a temperature on your own validation data if you need calibrated probabilities.
+- Prompts longer than 4,096 tokens are truncated (text only); training used 1,536 tokens. Questions with more than 62 options use an elimination step (two or more passes).
 
+<a id="honesty-notes"></a>
 ## Honesty notes
 
-- **Test sets used for selection.** tr_insan, practical TR, EN-300, TR-50, the LegalBench rule test, an earlier "none of the above" test (dropped from this card: its option counts reveal the answer) and KVKK were all used to compare training runs, and the acceptance rule for the released run (tr_insan ≥ 0.74, EN-300 ≥ 0.743, balanced "none of the above" ≥ 0.62 overall and ≥ 0.50 "none" found, KVKK errors ≤ 12; 5/5 passed) was set on them. None of these numbers is a fully untouched estimate. EN-300 always uses the same 300-row sample.
-- **In-distribution parts.** The train splits of 16 JevBench configs and of the five tr_insan sources are in training (test texts excluded); only the six held-out JevBench sources, the LegalBench rule test and the behaviour tests are unseen tasks.
-- **Single runs.** Each row is one training run; the only seed repeat is temiz2 (see table). Differences below about 0.02 on TR-50 and 0.01 elsewhere are within run-to-run noise.
-- **Competitors.** Microsoft-Decision-1 was run through its API; whether it was trained on JevBench is unknown. Intern-Decision-4B was run locally with its own prompt format and calibration temperature; for more than 62 options we added the same elimination step.
+- **Test sets were used for selection.** We compared training runs on tr_insan, practical TR, EN-300, TR-50, the LegalBench rule test, the balanced "none of the above" test and KVKK. The acceptance rule for this run was set on them too: tr_insan ≥ 0.74, EN-300 ≥ 0.743, balanced "none of the above" ≥ 0.62 overall and ≥ 0.50 "none" found, KVKK errors ≤ 12; the run passed 5/5. None of these numbers is a fully untouched estimate. The full JevBench run was made after selection.
+- **In-distribution parts.** The train splits of 16 JevBench configs and of the five tr_insan sources are in training (test texts excluded). Only the six held-out JevBench sources, the LegalBench rule test and the behaviour tests are unseen tasks.
+- **Single runs.** Each number comes from one training run. Differences below about 0.02 on TR-50 and 0.01 elsewhere are within run-to-run noise.
+- **Earlier runs.** Two earlier clean runs (temiz3 / temiz4) scored 0.753 / 0.740 on tr_insan and 0.764 / 0.767 on EN-300. They were rejected because of the shortcuts described above.
+- **Competitors.** Microsoft-Decision-1 and Jev were run through their APIs; whether they were trained on JevBench is unknown. Intern-Decision-4B was run locally with its own prompt format and calibration temperature.
 
 ## License
 
@@ -211,27 +415,27 @@ The base model Qwen/Qwen3.5-4B is licensed under Apache-2.0 (the `LICENSE` file
 <a id="ai-assistance"></a>
 ## AI assistance
 
-This model card, `karar.py`, `birlestir.py`, the tests and most of the training and evaluation code were written with Claude coding agents (Claude Code, Anthropic) under the author's direction; the author is responsible for the content and verified the reported numbers.
+This model card, its figures, `karar.py`, `birlestir.py`, the tests and most of the training and evaluation code were written with Claude coding agents (Claude Code, Anthropic) under the author's direction. The author is responsible for the content and verified the reported numbers.
 
 ## Citation
 
 ```bibtex
-@misc{karar_tr_4b_2026,
-  title        = "Karar-TR-4B: a Turkish-first decision model on Qwen3.5-4B",
+@misc{turnike_4b_2026,
+  title        = "Turnike-4B: a bilingual (English + Turkish) decision model on Qwen3.5-4B",
   author       = "Hasret, Berkay",
   year         = "2026",
-  howpublished = "https://huggingface.co/berkay123001/karar-tr-4b"
+  howpublished = "https://huggingface.co/berkay123001/turnike-4b"
 }
 ```
 
 ## Türkçe özet
 
-Karar-TR-4B, Qwen3.5-4B üzerine yaklaşık 130 MB'lık bir LoRA adaptörüdür. Metin üretmez: bir metin ve bir ya da birkaç kapalı soru verildiğinde her seçeneğe olasılık döndürür. Bir metnin 16 sorusuna kadar tek ileri geçişte cevap verir (RTX 5060 Ti'de 8 soru bf16 ≈ 156 ms, FP8 ≈ 126 ms). Her sorunun seçenekleri tek-token sembollere (A, B, C, ...) eşlenir; olasılık, her yer tutucudan hemen önceki konumda yalnız o sorunun sembol satırlarından okunur.
+Turnike-4B, İngilizce ve Türkçe için Qwen3.5-4B üzerine kurulu, yaklaşık 130 MB'lık bir LoRA adaptörüdür. Metin üretmez: bir metin ve bir ya da birkaç kapalı soru verildiğinde her seçeneğe bir olasılık döndürür. Bir metnin 16 sorusuna kadar tek ileri geçişte cevap verir. RTX 5060 Ti'de tek karar bf16 ≈ 109 ms, FP8 ≈ 82 ms; 8 soru bf16 ≈ 156 ms, FP8 ≈ 126 ms sürer. Her sorunun seçenekleri tek-token sembollere (A, B, C, ...) eşlenir. Olasılık, her yer tutucudan hemen önceki konumda yalnız o sorunun sembol satırlarından okunur.
 
 Herkese açık Türkçe ve İngilizce veri setleri, LLM ile etiketlenmiş açık metinler ve sentetik veriyle eğitildi; veri setleri yayınlanmıyor. Herhangi bir ticari karar API'sinden damıtılmadı. Bu sürümde web verisi yok; tarayıcı sürümü ayrı yayınlanacak.
 
-Tam JevBench (22.773 satır, resmi `jevify-run report`, 22 config eşit ağırlıklı makro): 0,774 (makro ECE 0,084; v23 0,733; ölçtüğümüz Jev 1.13.0 0,733). Herkese açık jev-bench liderlik tablosunda (46 satır) 0,774, en üstteki girdinin (35B MoE) 0,004 önünde birinci sıraya denk gelirdi; fark küçük ve sonuç tablo sahiplerine gönderilmedi. Sonuçlar (temiz5 / temiz4): tr_insan 0,748 / 0,740; pratik 0,891 / 0,894; JevBench EN-300 0,769 / 0,767; TR-50 0,700 / 0,693; KVKK hata 2 / 9; kural uygulama 0,832 / 0,833; dengeli "hiçbiri" 0,755 / 0,508. Jev'den damıtılmış önceki modelimiz v23: 0,745 / 0,857 / 0,732 / 0,671 / 12 / 0,753 / 0,620. Tohum tekrarı farkı tr_insan +0,001, TR-50 +0,020. Bu test setlerinin hepsi koşular arasında seçim yaparken kullanıldı; tamamen bağımsız tahmin değiller.
+Tam JevBench'te (22.773 satır, resmi `jevify-run report`, 22 config eşit ağırlıklı makro) Turnike-4B 0,774 alır: makro ECE 0,084, held-6 0,830, diğer-16 0,752. Aynı sette Jev 1.13.0 0,733 alır. Herkese açık jev-bench liderlik tablosunda (46 satır) 0,774, en üstteki girdinin (35B MoE, 0,770) önünde birinci sıraya denk gelirdi; fark küçük ve sonuç tabloya gönderilmedi. İnsan etiketli Türkçe testte (tr_insan, 1.379 karar) Turnike-4B 0,748, Microsoft-Decision-1 0,714 (p = 0,003), Jev 0,727, Intern-Decision-4B 0,675 alır. Çeviri JevBench TR-50'de Microsoft-Decision-1 önde: 0,745'e 0,700. Gömülü talimata kanma −0,014 (Intern +0,279). Dengeli "hiçbiri" testinde Turnike-4B 0,742–0,768. Bu test setlerinin hepsi koşular arasında seçim yaparken kullanıldı; tamamen bağımsız tahmin değiller.
 
-Sınırlamalar: "Yukarıdakilerden hiçbiri" dengeli testte 0,74–0,77 (v23 0,61–0,63); doğru şık yokken "hiçbiri" seçme 0,80–0,81, doğru şık varken de 0,22–0,25 (biraz fazla kullanıyor). Eski "hiçbiri" testi yapısal kısayol (şık sayısı cevabı ele veriyor) nedeniyle raporlanmıyor. Eğitim verisindeki iki kısayol temiz4 ve temiz5'te giderildi. KVKK 2 hata (tohumlar arası 11↔16 oynuyor; altın kümeler çok küçük, eşik 0,5 gürültülü); go_emotions, nefret/saldırgan dil sınırı ve yıldız puanı zayıf. Etiketlerin çoğu LLM'den.
+Sınırlamalar: "Yukarıdakilerden hiçbiri" biraz fazla kullanılıyor; doğru şık varken de 0,22–0,25 oranında seçiliyor. KVKK eşikleri gürültülü (2 hata, ama tohumlar arasında 11 ile 16 arasında oynuyor). go_emotions, nefret/saldırgan dil sınırı ve yıldız puanı zayıf. Etiketlerin çoğu LLM'den geliyor.
 
-Lisans: taban model Apache-2.0; adaptör, birleştirilmiş ağırlıklar ve kod Apache-2.0. Kullanım için `karar.py` içindeki `Karar(...).sor(metin, sorular)` yeterli. Bu kart ve çıkarım kodu yazarın yönlendirmesiyle Claude (Anthropic) kodlama ajanları tarafından yazıldı; sayılar yazar tarafından deney kayıtlarıyla doğrulandı.
+Lisans: taban model Apache-2.0; adaptör, birleştirilmiş ağırlıklar ve kod da Apache-2.0. Kullanım için `karar.py` içindeki `Karar(...).sor(metin, sorular)` yeterli. Bu kart ve çıkarım kodu, yazarın yönlendirmesiyle Claude (Anthropic) kodlama ajanları tarafından yazıldı; sayılar yazar tarafından deney kayıtlarıyla doğrulandı.

birlestir.py

@@ -28,8 +28,10 @@ DOGRULAMA_SORULARI = {"iade": {"type": "noul", "instructions": "Yazar para iades
 DOGRULAMA_TOLERANS = 1e-3
 EK_DOSYALAR = ("LICENSE",)
 ISLEMCI_DOSYALARI = ("preprocessor_config.json", "video_preprocessor_config.json")
-ANA_REPO = "berkay123001/karar-tr-4b"
-BIRLESIK_REPO = ANA_REPO + "-merged"
+# Ad ve repo kimlikleri isim.txt'den isim_uygula.py ile yazilir (elle degistirmeyin).
+MODEL_ADI = "Turnike-4B"
+ANA_REPO = "berkay123001/turnike-4b"
+BIRLESIK_REPO = "berkay123001/turnike-4b-merged"
 BIRLESIK_KART = """---
 license: apache-2.0
 base_model: Qwen/Qwen3.5-4B
@@ -43,7 +45,7 @@ tags:
 - turkish
 ---
 
-# Karar-TR-4B (merged bf16 weights)
+# {ad} (merged bf16 weights)
 
 The LoRA adapter [{ana}](https://huggingface.co/{ana}) merged into Qwen/Qwen3.5-4B (bf16, same merge that
 `karar.py` performs at load time). Use it without PEFT:
@@ -85,7 +87,8 @@ def birlestir(adapter: str, taban: str, cikti: Path) -> None:
     for ad in EK_DOSYALAR:
         if (burasi / ad).exists():
             shutil.copy2(burasi / ad, cikti / ad)
-    (cikti / "README.md").write_text(BIRLESIK_KART.format(ana=ANA_REPO, birlesik=BIRLESIK_REPO), encoding="utf-8")
+    kart = BIRLESIK_KART.format(ad=MODEL_ADI, ana=ANA_REPO, birlesik=BIRLESIK_REPO)
+    (cikti / "README.md").write_text(kart, encoding="utf-8")
     print(f"yazildi: {cikti} ({sum(f.stat().st_size for f in cikti.iterdir()) / 1e9:.2f} GB)")
 
 

karar.py

@@ -1,4 +1,4 @@
-"""Tek dosyalik cikarim modulu: Qwen3.5-4B + LoRA adaptoru ile tek geciste cok soruluk karar.
+"""Turnike-4B tek dosyalik cikarim modulu: Qwen3.5 (0.8B / 4B / 9B) + LoRA adaptoru ile tek geciste cok soruluk karar.
 
 Model metin uretmez. Metin ("State") ve sorular ("Decision schema") tek bir sohbet istemine yazilir; asistanin
 cevabi bir JSON iskeletidir, her soru icin bir yer tutucu. Her sorunun siklari tek-token sembollere eslenir
@@ -14,7 +14,7 @@ bunlari kullanir (opsiyonel hizlandirici; yoksa saf torch yolu, ayni sonuc, daha
 
 Kullanim:
     from karar import Karar
-    k = Karar("kullanici/model")                       # ya da yerel adaptor dizini
+    k = Karar("berkay123001/turnike-4b")           # ya da yerel adaptor dizini
     k.sor("Kulaklık bozuk geldi, paramı geri istiyorum.",
           {"Yazar para iadesi istiyor mu?": ["evet", "hayır"],
            "Mesajın tonu nedir?": {"olumlu": "memnun", "olumsuz": "şikâyetçi", "nötr": "duygu yok"}})
@@ -266,17 +266,33 @@ def _adaptor_yolu(adapter: str | Path) -> str:
     return str(yol)
 
 
+def _adaptor_tabani(adapter: str) -> str:
+    """Adaptorun adapter_config.json'undaki base_model_name_or_path (yerel dizin ya da Hub kimligi; peft okur)."""
+    from peft import PeftConfig
+
+    try:
+        taban = PeftConfig.from_pretrained(adapter).base_model_name_or_path
+    except (OSError, ValueError) as hata:
+        raise FileNotFoundError(f"adaptorun adapter_config.json dosyasi okunamadi: '{adapter}' ({hata})") from hata
+    if not taban:
+        raise FileNotFoundError(f"adapter_config.json icinde base_model_name_or_path yok: {adapter}")
+    return taban
+
+
 class Karar:
-    """Qwen3.5-4B + LoRA karar katmani. Bir ornek bircok metinde kullanilabilir."""
+    """Qwen3.5 (0.8B / 4B / 9B) + LoRA karar katmani. Bir ornek bircok metinde kullanilabilir."""
 
-    def __init__(self, adapter: str | Path | None, taban: str = VARSAYILAN_TABAN, device: str | None = None,
+    def __init__(self, adapter: str | Path | None, taban: str | None = None, device: str | None = None,
                  fp8: bool = False, maks_istem_token: int = MAKS_ISTEM_TOKEN) -> None:
-        """adapter=None: taban zaten birlestirilmis agirliklardir (birlestir.py ciktisi ya da onun Hub reposu)."""
+        """adapter=None: taban zaten birlestirilmis agirliklardir (birlestir.py ciktisi ya da onun Hub reposu).
+        taban=None: adaptor varsa adapter_config.json'daki taban (0.8B / 4B / 9B dogru eslesir), yoksa Qwen3.5-4B."""
         from transformers import AutoTokenizer, Qwen3_5ForConditionalGeneration
 
         if not isinstance(maks_istem_token, int) or maks_istem_token < 256:
             raise ValueError("maks_istem_token en az 256 olan bir int olmali")
         adapter = None if adapter is None else _adaptor_yolu(adapter)
+        if taban is None:
+            taban = _adaptor_tabani(adapter) if adapter is not None else VARSAYILAN_TABAN
         if adapter is None and taban == VARSAYILAN_TABAN:
             warnings.warn("adaptor yok ve taban egitimsiz Qwen3.5-4B: sifir-atis calisir (karar modeli degil).",
                           stacklevel=2)
@@ -284,7 +300,7 @@ class Karar:
         if fp8 and self.device.type != "cuda":
             raise RuntimeError("fp8=True yalniz CUDA'da calisir (FP8 matris carpimi, sm_89+)")
         if self.device.type == "cpu":
-            warnings.warn("CUDA yok: 4B model CPU'da bf16 calisir ama cok yavastir.", stacklevel=2)
+            warnings.warn("CUDA yok: model CPU'da bf16 calisir ama cok yavastir.", stacklevel=2)
         try:
             self.tok = AutoTokenizer.from_pretrained(taban)
             # CPU'ya yuklenir, LoRA CPU'da birlestirilir, sonra cihaza tasinir (olcum koduyla ayni yol).
@@ -380,7 +396,8 @@ def arguman_ayristirici() -> argparse.ArgumentParser:
     p = argparse.ArgumentParser(description="Metin + sorular -> secenek olasiliklari (tek geciste).")
     p.add_argument("--adapter", default=None,
                    help="adaptor dizini ya da Hub kimligi (verilmezse --taban birlestirilmis agirlik sayilir)")
-    p.add_argument("--taban", default=VARSAYILAN_TABAN, help="taban model (Hub kimligi ya da yerel yol)")
+    p.add_argument("--taban", default=None,
+                   help="taban model (Hub kimligi ya da yerel yol); verilmezse adaptorun adapter_config.json tabani")
     p.add_argument("--metin", required=True, help="karar verilecek metin")
     p.add_argument("--soru", action="append", required=True, type=soru_ayristir,
                    help=f"'Soru?{CLI_SORU_AYRACI}secenek1{CLI_SORU_AYRACI}secenek2...' (tekrarlanabilir)")