Instructions to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with libraries, inference providers, notebooks, and local apps. Follow these links to get started.

Notebooks
Google Colab
Kaggle
Local Apps Settings

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with llama.cpp:

Install (macOS, Linux)

curl -LsSf https://llama.app/install.sh | sh
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
# Run inference directly in the terminal:
llama cli -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Install from WinGet (Windows)

winget install llama.cpp
# Start a local OpenAI-compatible server with a web UI:
llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
# Run inference directly in the terminal:
llama cli -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Build from source code

git clone https://github.com/ggml-org/llama.cpp.git
cd llama.cpp
cmake -B build
cmake --build build -j --target llama-app
# Start a local OpenAI-compatible server with a web UI:
./build/bin/llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
# Run inference directly in the terminal:
./build/bin/llama cli -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Use Docker

docker model run hf.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Llama new
LM Studio
Jan

vLLM

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with vLLM:

Install from pip and serve model

# Install vLLM from pip:
pip install vllm
# Start the vLLM server:
vllm serve "jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4"
# Call the server using curl (OpenAI-compatible API):
curl -X POST "http://localhost:8000/v1/chat/completions" \
	-H "Content-Type: application/json" \
	--data '{
		"model": "jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4",
		"messages": [
			{
				"role": "user",
				"content": [
					{
						"type": "text",
						"text": "Describe this image in one sentence."
					},
					{
						"type": "image_url",
						"image_url": {
							"url": "https://cdn.britannica.com/61/93061-050-99147DCE/Statue-of-Liberty-Island-New-York-Bay.jpg"
						}
					}
				]
			}
		]
	}'

Use Docker

docker model run hf.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Ollama
How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with Ollama:
```
ollama run hf.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
```
Unsloth Desktop

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with Pi:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Configure the model in Pi

# Install Pi:
npm install -g @earendil-works/pi-coding-agent
# Add to ~/.pi/agent/models.json:
{
  "providers": {
    "llama-cpp": {
      "baseUrl": "http://localhost:8080/v1",
      "api": "openai-completions",
      "apiKey": "none",
      "models": [
        {
          "id": "jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0"
        }
      ]
    }
  }
}

Run Pi

# Start Pi in your project directory:
pi

Docker Model Runner
How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with Docker Model Runner:
```
docker model run hf.co/jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0
```

Lemonade

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with Lemonade:

Pull the model

# Download Lemonade from https://lemonade-server.ai/
lemonade pull jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Run and chat with the model

lemonade run user.Qwen3.8-Flash-CIRU-STRIX-IU4-Q8_0

List all available models

lemonade list

Hermes Agent

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with Hermes Agent:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Configure Hermes

# Install Hermes:
curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash
hermes setup
# Point Hermes at the local server:
hermes config set model.provider custom
hermes config set model.base_url http://127.0.0.1:8080/v1
hermes config set model.default jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Run Hermes

hermes

Atomic Chat

OpenClaw

How to use jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4 with OpenClaw:

Start the llama.cpp server

# Install llama.cpp:
brew install llama.cpp
# Start a local OpenAI-compatible server:
llama serve -hf jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0

Configure OpenClaw

# Install OpenClaw:
npm install -g openclaw@latest
# Register the local server and set it as the default model:
openclaw onboard --non-interactive --mode local \
  --auth-choice custom-api-key \
  --custom-base-url http://127.0.0.1:8080/v1 \
  --custom-model-id "jcbtc/Qwen3.8-Flash-CIRU-STRIX-IU4:Q8_0" \
  --custom-provider-id llama-cpp \
  --custom-compatibility openai \
  --custom-text-input \
  --accept-risk \
  --skip-health

Run OpenClaw

openclaw agent --local --agent main --message "Hello from Hugging Face"

Qwen3.8-Flash-CIRU-STRIX-IU4 / HISTORY.md

jcbtc

Release v5.0 with benchmarks, source and installation guide

dff8c4d verified about 1 month ago

preview code

Raw History Blame Contribute Delete

15 kB

Historical Flash release benchmarks

The current v5.0 comparison is in BENCHMARKS.md. This history preserves earlier reported results and their limitations. It is not a rerun of old tests on v5.0. The retained model weights and versioned runtime files remain available. For the old installation instructions, see the immutable v4.4.1 model card.

v4.4.1

v4.4.1 corrected image input with MTP. It required updating the complete runtime; model weights stayed unchanged. This patch had no new broad quality or speed suite in its model-card section. See the v4.4.1 source/build instructions. The new matched v4.4.1 versus v5.0 suite is reported separately.

v4.4

v4.4 combined speculative decoding, attention/cache, GPU launch, projection fusion, prompt-branch reuse and recurrent-layer improvements. MTP depth 3 remained the default.

Measured IU4 workload	Earlier matched control	v4.4	Change
12960-token replay decode	37.79 tok/s	44.53 tok/s	+17.84%
245760-token cold decode	7.62 tok/s	21.14 tok/s	+177.48%
Cached deep append, whole request	69.73 s	25.84 s	−62.95%
Branch revisit, first streamed piece	5.653 s	0.222 s	−96.08%

The matched control was an intermediate v4.3 build with additional runtime changes, not the plain published v4.3 release. There was one final block per arm with matched requests/settings.

On a separate HumanEval 0–9 speed panel, default MTP3 measured 60.35 TG; the optional speculative boost mode measured 64.07 TG (+6.16%, peak task 70.95). These were short prompts with 256K allocated capacity, not full-context generation or coding pass scores. The optional boost mode was not used for HermesAgent-20.

The Hermes run had 19/20 full-score tasks, arithmetic mean 98.5/100 and weighted score 99/100. Its remaining artifact-task failure was retained. Different engines/seeds prevent attributing historical score changes to a single kernel. A separate target-only 512K prefill measured 604.65 PP; no 512K decode or quality claim followed. The v4.4 image/MTP failure was subsequently fixed in v4.4.1.

Release notes · Detailed qualification

v4.3

v4.3 selected MTP depth 3 for Flash IU4 and depth 4 for Orca, and enabled sparse draft attention for one-token batches. It retained corrected v4.2 inference binaries and model files.

Model	Hermes mean score /100, passes 1 / 2	Full-score tasks, passes 1 / 2	Prompt tok/s	Generation tok/s
IU4	95.0 / 98.5	17/20 / 19/20	548.5	34.74
Orca	92.5 / 98.5	17/20 / 19/20	551.3	34.44

These HermesAgent-20 runs used two passes per model, 80 scenario attempts and 449 model requests, a 32-turn allowance, production sampling and high thinking effort. Speeds pool the two passes and include reasoning tokens. The models ran on two 128-GB Strix Halo systems with NixOS/ROCm 10. They were observed configurations, not a matched speedup over v4.2. The report retains grading limits and Orca memory-persistence behavior.

Hermes results and grading notes

v4.2

v4.2 corrected a fallback attention path that ignored selected keys in v4.0/v4.1. It passed 48/48 CPU-reference checks; Flash and Orca completed the reproduced incident prompt with valid tool calls and no detected repeated long line. This was targeted regression evidence.

One matched Flash workload slowed from 25.54–25.55 to 21.20–21.27 TG. The two mirrored differences were −17.04% and −16.71%; prefill remained within a 5% parity band. Draft acceptance also fell. This is a repeatable cost on that one workload, not a universal slowdown or an Orca speed measurement. The earlier attention computation was incorrect.

Attention correction and measurements

v4.1

Increasing SSD embedding-table I/O workers from 16 to 32 improved a matched Flash 16K prefill by 4.44%. A short matched MTP6 check found no generation loss. Weights were unchanged. Versioned release

v4.0 performance and limits

These observations predate the v4.2 attention correction and used diagnostic tracing. They are not clean repeated comparisons with later builds. Hardware was Ryzen AI Max+ 395 / Radeon 8060S, 128-GB unified memory, NixOS and ROCm 10. Each row is one observation except the five-prompt range.

V4 request	Input tokens	Prompt tok/s	Generation tok/s	First streamed piece (s)	Whole request (s)
Fixed token input, 128 output	4,096	631.04	14.61	6.49	15.19
Cold key recall	7,951	973.71	26.23	8.25	8.82
Five cold coding prompts	30,847–30,886	991.76–1,000.81	35.13–40.80	30.87–31.12	32.26–37.18
Cold key recall	65,295	947.89	32.56	69.00	69.77
Fixed token input, 128 output	16,384	1,003.33	14.43	16.45	25.25
Capacity, fixed input, 128 output	261,888	748.38	6.22	349.97	370.39

Three short coding requests measured 51.21–52.93 TG. Serving completed 15/15 requests, recall 5/5 and cache/replay 3/3, with at least 6.82 GiB available RAM. The full-capacity request left 7.70 GiB available RAM and measured capacity, not accuracy throughout the context. Numerical checks passed 22/22 GPU cases.

The eight-task nonthinking coding screen scored 7/8 as standalone modules. One response omitted a typing import present in its prompt. A separate, uniform rescore that prepended each prompt's original imports scored 8/8 on the same answers. No answers were regenerated. Five prompts were about 30.8K tokens; three were short. Image/MTP and multiple-slot MTP had not been qualified at this point.

Full v4.0 qualification · Structured result

Earlier full and sampled quality suites

The score/count table and early perplexity diagnostic are retained in BENCHMARKS.md. These earlier suites used the retained model artifacts, not new v5.0 weights. The 29 August depth-1 MTP evaluation covered these elapsed times, with one request at a time and natural end-of-sequence generation:

Dataset	Items	Wall time
ARC-Challenge	1,172	24m 42s
GPQA-Diamond	50	1h 33m 21s
MMLU-Pro	70	36m 37s
GSM8K	100	20m 19s
IFEval	100	22m 50s
Quality suite total	1,492	3h 17m 50s

Setup and earlier failed attempts are excluded. Including a separate performance probe gave 3h 20m 46s. No equivalent full EvalScope wall-time measurement was available for v3 or v4. Historical methodology

v3 serving, quality and capacity

One Strix Halo system, identical input token IDs, cold cache, 128 requested output tokens, one slot and 262,144-token capacity. Nonthinking sampling used temperature 0.7, top-p 0.8, top-k 20, presence penalty 1.5 and seed 123; EOS was honored.

Input tokens	Profile	Prompt tok/s	Generation tok/s	First streamed piece (s)	Whole request (s)
4,096	Previous CIRU	392.00	22.52	10.70	16.34
4,096	CIRU v3	455.65	24.60	9.25	14.41
4,096	Halo	381.49	35.30	11.09	14.69
65,536	Previous CIRU	284.49	13.33	230.46	239.99
65,536	CIRU v3	369.81	24.22	177.32	182.57
65,536	Halo	263.42	23.28	248.91	254.37

“Previous CIRU” means v2.0.1/MTP6 with batch 2,048 and microbatch 512; v3 used the same weights with MTP6 and batch/microbatch 1,024. The earlier Halo comparison used Vulkan revision 5f851647fe5ed795dfd6c0a3fba543114879e874, Unsloth UD-Q4_K_XL and a compatible EasiiX Q8 MTP head. It differs from the new HaloBox HIP comparison in BENCHMARKS.md. Two incompatible draft-head loads failed before the compatible head was selected; no source/target weights were changed. These are package comparisons.

Halo's short-context MTP screen measured 35.37 TG at depth 3, 31.4 at depth 2, 30.04 at depth 4, 25.91 at depth 6 and 29.31 with adaptive 6. Three v3 loads, two selected Halo loads and one older CIRU load are bounded observations, not statistical confidence intervals.

Optional v3 MTP depth 2

Input tokens	Optional v3 MTP 2 prompt tok/s	Generation tok/s	Whole request (s)
4,096	453.09	29.39	13.62
65,536	373.08	24.88	180.87

Depth 2 helped those lower-acceptance requests, while depth 6 remained faster on the short coding panel. Full-vocabulary target verification was retained at either depth.

Profile	HumanEval base	EvalPlus extended tests	Recall at about 8K and 64K
Previous CIRU	20/20	20/20	Both keys and exact cached replay
CIRU v3	20/20	20/20	Both keys and exact cached replay
Halo	20/20	20/20	Both keys and exact cached replay

The coding panel used HumanEval 0–19 and EvalPlus 0.1.10, first samples, a 4,096-token cap and truncations counted as failures. Code executed in a filesystem/network sandbox. v3 separately completed 261,888 input plus 128 output tokens at 257.44 PP / 18.00 TG in 1,024.44 seconds. This was a CIRU-only capacity check, not a full-depth quality/comparator test.

v3 passed 69 attention-state checks, 33 ROCm operator-reference cases and 30 allocator tests. A separate four-prefix diagnostic matched 15,892,480 FP32 logits byte-for-byte; general bitwise equivalence was not claimed because threshold ties and selection order could vary.

Comparison and memory report · Structured results

9 September mixed task panel

These times include agent/tool work, scoring and harness overhead. The models ran sequentially with one slot: older CIRU and v3 used MTP6; the unchanged Halo Vulkan comparator used MTP3.

Benchmark stage	Previous CIRU	CIRU v3	Halo MTP3
Complete mixed panel, after readiness	29m 17.11s	25m 18.32s	24m 34.43s
Additional model load	31.17s	31.15s	33.62s
Short scored stage	5m 20.33s	5m 20.85s	4m 35.60s
Hermes, two rounds combined	12m 28.01s	11m 37.57s	10m 08.12s
Shared 63K-token history seeding	3m 26.33s	2m 42.80s	3m 21.00s
Long hard stage, history already loaded	4m 43.92s	2m 41.90s	3m 22.96s
Total generated tokens	33,294	33,451	31,058

Profile	Short IFEval strict	Short GSM8K	Short HumanEval	Long IFEval strict	Long GSM8K	Long HumanEval
Previous CIRU	5/8	8/8	5/6	2/2	2/2	2/4
CIRU v3	5/8	8/8	5/6	2/2	2/2	2/4
Halo MTP3	6/8	8/8	5/6	2/2	2/2	3/4

Profile	Hermes native full passes	Native mean points / 100	Reviewed end states
Previous CIRU	7/12	80.83	11/12
CIRU v3	11/12	95.83	12/12
Halo MTP3	11/12	95.83	12/12

This panel comprised 22 short academic tasks, six Hermes scenarios twice, eight hard tasks returning to shared long history, and 10 short/eight long coding checks. Tasks were selected from earlier disagreements and failures; they are not a representative random sample. Output lengths differed. v3 took 13.59% less complete time than older CIRU; Halo finished 43.89 seconds sooner than v3.

Native scores were retained. Reviewed end states separately counted grader artifacts such as memory wording mismatches. All arms passed the 10 short and eight long coding checks on base and extended tests. Two Hermes repetitions do not establish a reliable failure rate.

Short-task TG was 38.44 / 39.09 / 43.97 for older CIRU / v3 / Halo; long-stage TG was 21.75 / 35.93 / 31.25. Short tasks used temperature 0, seed 15035, thinking off and a 32,768-token allowance. Hermes used thinking, temperature 0.6, top-p 0.95 and top-k 20. History seeding was timed separately. Interrupted/instrumented captures were excluded.

Short coding and MTP depth

All four profiles passed 20/20 base and extended tests. API request times exclude grading.

Profile	Generated tokens	Prompt tok/s	Generation tok/s	Sum of request times
Previous CIRU MTP6	3,179	148.53	53.33	75.49s
CIRU v3 MTP6	3,179	219.51	53.24	70.67s
CIRU v3 MTP2	3,212	226.25	39.63	91.57s
Halo MTP3	3,241	176.10	49.48	79.15s

Older CIRU and v3 MTP6 produced identical token IDs on all 20 tasks. v3 reduced request time 6.39%, mainly through prefill; decode was effectively unchanged. This supported that release's default, not a universal MTP optimum. The historical v3 Q4_1 matrix path expanded values into IU8 arithmetic; its name alone did not establish native IU4 acceleration.

Task panel and caveats · Results JSON · Wall-time CSV · Earlier v2 model card

The earlier v2 card's 42.3 TG short greedy probe used a different workload. It is retained as history and was not rerun as a full suite for later versions.