StandardThinking/StandardOne-3B-SH-GGUF

Standard One 3B SH — GGUF

Updated weights (v3, 2026-10-13). This repository now holds GGUF builds of Standard One 3B SH v3, in thirteen quant levels. The previous builds stay available under the tag v1.

Version: v3 (2026-10-13)

GGUF builds of the backbone of Standard One 3B SH v3 (mistral3 architecture, text model) for use with llama.cpp, together with the schema head and a small runner that serves the same POST /v1/systemone contract on CPU, Apple silicon or any other llama.cpp backend.

The schema head does not read text output. It reads two hidden states for every token — the final layer (after the last norm) and the output of decoder layer 19 — so stock llama-server cannot run it on its own. The runner here takes both states from llama.cpp through its evaluation callback (runner/sh_states.cpp) and runs the head in PyTorch.

Files

File	Quant	Size
StandardOne-3B-SH-BF16.gguf	BF16 (no quantization)	6.9 GB
StandardOne-3B-SH-Q8_0.gguf	Q8_0	3.7 GB
StandardOne-3B-SH-Q6_K.gguf	Q6_K	2.8 GB
StandardOne-3B-SH-Q5_K_M.gguf	Q5_K_M	2.5 GB
StandardOne-3B-SH-Q5_K_S.gguf	Q5_K_S	2.4 GB
StandardOne-3B-SH-Q4_K_M.gguf	Q4_K_M	2.1 GB
StandardOne-3B-SH-Q4_K_S.gguf	Q4_K_S	2.1 GB
StandardOne-3B-SH-Q4_0.gguf	Q4_0	2.0 GB
StandardOne-3B-SH-IQ4_XS.gguf	IQ4_XS (importance matrix)	2.0 GB
StandardOne-3B-SH-Q3_K_M.gguf	Q3_K_M (importance matrix)	1.8 GB
StandardOne-3B-SH-IQ3_M.gguf	IQ3_M (importance matrix)	1.7 GB
StandardOne-3B-SH-Q2_K.gguf	Q2_K (importance matrix)	1.5 GB
StandardOne-3B-SH-IQ2_M.gguf	IQ2_M (importance matrix)	1.3 GB
imatrix.gguf	importance matrix used for IQ4_XS, Q3_K_M, IQ3_M, Q2_K and IQ2_M	3.0 MB
head/	schema head, float32 (69.5M parameters) and its config	278 MB
tokenizer/	tokenizer and chat template used to render requests (empty default system message)	34 MB
runner/	sh_states.cpp (state extractor), serve_head_gguf.py (server) and the shared request/head code	—

BF16 down to Q4_0 are plain llama-quantize builds; IQ4_XS, Q3_K_M, IQ3_M, Q2_K and IQ2_M use an importance matrix (imatrix.gguf, llama-imatrix -c 2048 over our calibration corpus of decision requests, the same recipe as the StandardOne GGUF repos). SHA256 checksums: SHA256SUMS. Source and conversion details: release-manifest.json.

Quick start with llama.cpp

The SH model requires the supplied state extractor and Python schema-head runner. A stock llama-server -hf command loads the backbone but does not execute the SH head. Use this workflow for SH decisions. For the earlier letter-logit model that runs directly in stock llama-server, see StandardOne-3B-GGUF.

Prerequisites: Git, CMake, a C++17 compiler, and Python with pip in an activated virtual environment. The commands below use Bash/Zsh on Linux or macOS. Use a fresh working directory. Q8_0 is the recommended starting build; model-file size excludes the head, tokenizer and runtime buffers.

python -m pip install --upgrade huggingface_hub
python -m pip install torch transformers safetensors numpy tokenizers

# Download the backbone AND the required head, tokenizer and runner.
hf download StandardThinking/StandardOne-3B-SH-GGUF \
  --include "StandardOne-3B-SH-Q8_0.gguf" "head/*" "tokenizer/*" "runner/*" \
  --local-dir sh-gguf

# Build the version used for the published validation.
git clone https://github.com/ggml-org/llama.cpp
cd llama.cpp
git checkout 53ed051
cmake -B build
cmake --build build -j

# Link the state extractor against the matching llama.cpp libraries.
g++ -O2 -std=c++17 -Iinclude -Iggml/include ../sh-gguf/runner/sh_states.cpp \
  -Lbuild/bin -lllama -lggml -lggml-base \
  -Wl,-rpath,"$PWD/build/bin" -o ../sh-gguf/runner/sh-states
cd ../sh-gguf

# Keep this process running; send requests from another terminal.
python runner/serve_head_gguf.py \
  --gguf StandardOne-3B-SH-Q8_0.gguf --head head --tokenizer tokenizer \
  --sh-states runner/sh-states --port 30171 --threads 16

When the server reports that it is listening, send the requests in Practical examples. Stop it with Ctrl+C. To use another quantization, change the filename in both the download and server commands; keep the head and tokenizer from this same repository. For reproducibility, add --revision <full-repository-commit> to the download command.

The runner serves one request at a time; --context sets the longest request in tokens (default 34,000). The request/response contract is shared with StandardOne-3B-SH. Backend availability depends on the llama.cpp build; see the validation section for measured hardware.

For a CUDA build, configure with cmake -B build -DGGML_CUDA=ON before compiling. Set GGML_CUDA_DISABLE_GRAPHS=1 when launching the Python server: the published B300 check observed a hang with CUDA graphs enabled.

Practical examples

The following invented records illustrate four integrations: receiving inventory, diagnosing an import failure, evaluating an inspection note, and checking backup evidence. They are example inputs, not benchmark results. Start either the reference server or the SH GGUF runner on port 30171 before sending them.

Warehouse receiving: reconcile a delivery record

An inventory application can ask several questions about one receiving note. Here, the model compares the recorded quantities and checks whether the paperwork supports closing the receipt.

curl --fail-with-body http://127.0.0.1:30171/v1/systemone \
  -H "Content-Type: application/json" --data-binary @- <<'JSON'
{
  "state": "Purchase order R-417 lists 48 cable reels. The receiving scan recorded 46 reels. Two pallet tags cannot be read, and the receiver has not signed the count sheet.",
  "questions": {
    "receipt_action": {
      "type": "choice",
      "instructions": "Select the next inventory step supported by the receiving note.",
      "criteria": {
        "close_receipt": "The delivered quantity is reconciled and the count sheet is signed.",
        "reconcile_count": "The recorded quantity differs from the order or pallet identification is unresolved.",
        "await_delivery": "The shipment has not reached the receiving area."
      }
    },
    "quantities_match": {
      "type": "noul",
      "instructions": "Do the receiving scan and purchase order report the same number of reels?"
    },
    "record_completeness": {
      "type": "score",
      "instructions": "Evaluate how complete the receiving evidence is.",
      "criteria": [
        "No quantity or identification evidence is available.",
        "Some evidence is recorded, but discrepancies or missing sign-off remain.",
        "Quantities and identifiers are reconciled and sign-off is recorded."
      ]
    }
  }
}
JSON

This record describes an unresolved count, unequal quantities and partial paperwork. Use those facts to inspect the returned answers. A choice question compares the descriptions you provide, a noul question returns a yes/no probability, and a score question returns an expected position on its ordered, zero-based scale. The score can lie between two levels.

Python client

Use this small standard-library client for the remaining examples. It displays the response as received so that you can examine the selected values and probability estimates.

import json
from urllib.request import Request, urlopen

def evaluate_record(record, questions):
    payload = json.dumps({"state": record, "questions": questions}).encode("utf-8")
    request = Request(
        "http://127.0.0.1:30171/v1/systemone",
        data=payload,
        headers={"Content-Type": "application/json"},
        method="POST",
    )
    with urlopen(request, timeout=300) as response:
        print(json.dumps(json.load(response), indent=2))

Data import: identify a column mismatch

An import monitor can classify a diagnostic record before selecting a repair playbook. This input contains rows, but the expected timestamp column is absent.

evaluate_record(
    "Import job J-82 read 1,200 CSV rows. Required column event_time was not found. The file contains a column named timestamp_ms. UTF-8 decoding succeeded.",
    {"failure_kind": {
        "type": "choice",
        "instructions": "Identify the import failure described in this diagnostic record.",
        "criteria": {
            "column_mismatch": "The available column names differ from the required schema.",
            "empty_input": "The file contains no data rows.",
            "text_encoding": "The importer cannot decode the file contents.",
            "unresolved": "The record does not establish a failure category."
        }
    }}
)

Equipment inspection: evaluate a filter reading

A maintenance log can map a written observation onto a defined condition scale. The criterion here is the gauge marking recorded by the inspector.

evaluate_record(
    "Inspection of ventilation unit V-12: the filter pressure gauge is above the marked replacement line. The fan guard is secured and the housing has no visible cracks.",
    {"filter_condition": {
        "type": "score",
        "instructions": "Assess the filter condition using the reported gauge position.",
        "criteria": [
            "The gauge is inside the normal operating band.",
            "The gauge is above normal but below the replacement line.",
            "The gauge has reached or exceeded the replacement line."
        ]
    }}
)

Backup verification: compare evidence with an acceptance rule

A backup catalog can check whether a verification record satisfies a supplied rule. This example requires both a matching checksum and a successful restore check.

evaluate_record(
    "Archive B-206 was written to storage. Verification reported a checksum mismatch. A restore check has not been attempted.",
    {"backup_verified": {
        "type": "noul",
        "instructions": "Does this record satisfy the rule that a verified backup must have a matching checksum AND a successful restore check?"
    }}
)

These calls only request decisions; the surrounding application decides what to do with them. Measure accuracy and choose probability thresholds using records from your own workflow. All examples send text, which is the validated input path for these releases.

Validation

Measured 2026-10-13 through the runner with a ROCm (HIP) build of llama.cpp 53ed051 on one AMD Instinct MI350X, against the official Decision Index 0.3 run of the BF16 release (StandardOne-3B-SH v3) on the same requests. Same sample as the FP8 check: 150 random requests from each of six benchmarks where the model is strong (Home appliance simulator, ContractNLI, HoVer, CLINC150, BANKING77, POP909-CL) and six where it is weak (MuSR, WinoGrande, GSM8K, CRUXEval, ChessBench, MMLU-Pro); ContractNLI has 123 in total.

BF16 reference (the official Decision Index run): 92.68 % on the strong benchmarks, 46.42 % on the weak ones.

Build	Accuracy, strong	Accuracy, weak	Same answer, strong	Same answer, weak	Same answer, all (5,019 questions)
Q8_0	92.68 %	47.78 %	99.26 %	89.38 %	97.67 %
Q6_K	92.68 %	47.78 %	99.22 %	88.64 %	97.51 %
Q5_K_M	92.71 %	47.53 %	99.07 %	87.53 %	97.21 %
Q5_K_S	92.59 %	48.02 %	99.14 %	87.41 %	97.25 %
Q4_K_M	92.80 %	46.17 %	98.88 %	83.70 %	96.43 %
Q4_K_S	92.63 %	46.42 %	98.79 %	84.07 %	96.41 %
Q4_0	92.61 %	46.67 %	98.62 %	79.38 %	95.52 %
IQ4_XS	92.78 %	46.79 %	98.86 %	83.46 %	96.37 %
Q3_K_M	92.75 %	45.19 %	98.03 %	79.75 %	95.08 %
IQ3_M	92.40 %	47.04 %	97.98 %	78.64 %	94.86 %
Q2_K	90.71 %	43.09 %	95.44 %	63.95 %	90.36 %
IQ2_M	90.62 %	42.47 %	94.75 %	64.94 %	89.94 %

Q8_0 is the recommended build. Down to Q4_0 (and IQ4_XS), accuracy moves by at most 1.6 points from BF16, in both directions; the answers that change are mostly ones the model was unsure about. Q3_K_M and IQ3_M move by at most 1.2 points. Q2_K and IQ2_M lose up to 4.0 points and change a quarter or more of the uncertain answers; use them only when memory is the limit.

Limitations

• Text input only (no vision projector in this repository).
• CPU prefill is slow for long requests: prefill runs on the CPU, so expect several seconds per thousand tokens on a 16-core server.
• The runner is a reference implementation, not an inference engine: one request at a time, no batching.

License

Apache-2.0, the same terms as StandardOne-3B-SH and Standard One 3B, built on Ministral 3 3B (Apache-2.0).