---
license: apache-2.0
library_name: pytorch
pipeline_tag: text-classification
tags:
- decision-model
- classification
- system-one
- s1mb
- typed-decisions
- darwin
- ztc
- custom-code
---
# Darwin-27B-ZTC-v9
-gold?style=for-the-badge)
Darwin-27B-ZTC-v9 is a decision model. It does not generate text. Given a state and a typed question, it returns a calibrated probability for every option in one forward pass: yes or no (`noul`), one of several options (`choice`), or a level on an ordered scale (`score`). Requests use the System One shape (`state` plus `questions`), so clients written for that format work unchanged.
It is the next version of [Darwin-27B-ZTC-v5](https://huggingface.co/FINAL-Bench/Darwin-27B-ZTC-v5). The main change is much stronger multiple-choice decisions (intent, entity type, relation, part of speech, rule reasoning, product relevance) while keeping the yes/no and score behaviour of v5.
## Results
### S1MB (System One Mosaic Benchmark), english-v1
All 137 benchmarks complete, measured with the S1MB evaluator and its unchanged `autojev` adapter, zero failed decisions. Merged by the maintainer on 2026-10-19 ([PR #75](https://huggingface.co/datasets/hotchpotch/s1mb-result/discussions/75)). The ranking below follows the leaderboard's own Borda method (`viewer/src/lib/borda.ts`), recomputed on the published results of 2026-10-19 (195 complete models) with this model added; the [live leaderboard](https://huggingface.co/spaces/hotchpotch/S1MB-leaderboard) is authoritative. At submission (results of 2026-10-17, 171 models) it also ranked first, with 91.29.
| Rank | Model | Borda score | Mean baseline-adjusted score |
|---|---|---|---|
| **1** | **Darwin-27B-ZTC-v9** | **90.38** | **72.20** |
| 2 | Sqwish kettei-fat v1.0 (API) | 90.14 | |
| 3 | Punky Tiger Labs Superion S1 | 89.53 | |
| 4 | D3 (27B) | 89.35 | |
| 5 | FINAL-Bench/Darwin-27B-ZTC-v5 | 88.52 | |
### Largest gains over v5 (baseline-adjusted score)
| Benchmark | v5 | v9 |
|---|---|---|
| boardgameqa-choice | 0.500 | 0.953 |
| ud-ewt-choice (part of speech) | 0.433 | 0.833 |
| gretel-pii-choice (entity type) | 0.653 | 0.981 |
| wiqa-choice | 0.365 | 0.667 |
| fol-nli-choice | 0.279 | 0.574 |
| snli-choice | 0.790 | 0.968 |
| esci-choice | 0.315 | 0.466 |
| argument-quality-choice | 0.857 | 0.959 |
## Training and data
Full-weight fine-tuning in the Darwin-27B-ZTC line on public **train** splits of decision datasets under licenses that allow commercial use. For v9 we added the original upstream train splits of choice-task sources (for example DBpedia-14, SNLI, MASSIVE, HWU64, BANKING77, CLINC150, MTOP, Few-NERD, UD English-EWT, Gretel PII masking, ESCI, WIQA, FOL-NLI, BoardgameQA, IBM argument quality), converted to the System One format.
- No test split of S1MB or of any benchmark was used for training.
- Every training row was screened against all S1MB cases (exact match of the normalised state, rare long text fields and duplicate states); matches were removed. Converters were checked by reproducing the S1MB test cases from the upstream test splits, and those test splits were used only for that check.
- Inference is one forward pass per question with the checkpoint temperature; no ensembling.
## Usage
Python 3.12, a CUDA GPU with about 60 GB of free memory (BF16 weights), and the code shipped in this repository (`autojev/`).
```python
import sys
from pathlib import Path
from huggingface_hub import snapshot_download
path = Path(snapshot_download("FINAL-Bench/Darwin-27B-ZTC-v9"))
sys.path.insert(0, str(path))
from autojev.model import DecisionModel, answer
model = DecisionModel(checkpoint=path).eval()
question = {
"type": "choice",
"instructions": "Identify the banking intent expressed by the utterance.",
"criteria": {"card_arrival": "Card delivery timing", "lost_or_stolen_card": "Card lost or stolen", "top_up_failed": "Top-up failed"},
}
row = {"state": {"utterance": "How long does a new card take to arrive?"}, "question": question}
print(answer(question, model.predict([row])[0]))
```
`ztc_server.py` serves `POST /v1/systemone` with batching.
```bash
curl http://localhost:8000/v1/systemone -H "Content-Type: application/json" -d '{"model": "FINAL-Bench/Darwin-27B-ZTC-v9",
"state": "I was charged twice for my order and want a refund.",
"questions": {
"refund": {"type": "noul", "instructions": "Is a refund requested?"},
"intent": {"type": "choice", "instructions": "What does the customer want?",
"criteria": {"refund": "wants money back", "cancel": "wants to cancel an order", "track": "wants to track an order"}}}}'
```
## License
Apache-2.0. The decision code in `autojev/` is MIT (upstream autojev by Denis Yarats) with changes for text-only backbones. Training data keep their own licenses; several sources are CC BY-SA (DBpedia-14, SNLI, UD English-EWT, MTOP, Few-NERD) and are credited here.