--- license: apache-2.0 base_model: HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF base_model_relation: quantized library_name: ninfer pipeline_tag: image-text-to-text tags: - ninfer - qwen3.8 - groupwise-int - dflash2 - speculative-decoding - uncensored - artifact - rtx-3090 - rtx-4090 - rtx-5090 --- # Qwen3.8-27B Uncensored (HauhauCS Aggressive) + DFlash2, NInfer v3 artifact A single-file [NInfer](https://github.com/Neroued/ninfer) **v3** artifact carrying the community uncensored tune [HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF](https://huggingface.co/HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF) as the target model and the official [z-lab/Qwen3.8-27B-DFlash2](https://huggingface.co/z-lab/Qwen3.8-27B-DFlash2) draft model as the DFlash2 speculative companion (recipe `qwen3_8_27b-v2`, the first NInfer recipe with the DFlash2 payload). The native NextN/MTP head of the tune is preserved as well, so both `--spec mtp` and `--spec dflash2` work. It runs on **NInfer-all**, the `master` branch of [iamwavecut/ninfer-all](https://github.com/iamwavecut/ninfer-all), which serves the RTX 3090, RTX 4090, RTX 5090 and RTX PRO 6000 Blackwell. Its first (v2) release ran on stock NInfer, which builds only for the RTX 5090; this v3 file was verified on NInfer-all. | | | |---|---| | Artifact identity | `qwen3.8-27b / groupwise-int`, container v3 | | File | `Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-DFlash2-ninfer-v3.ninfer`, 20,437,571,568 bytes | | SHA-256 | `cc96e6b0ecc3644a878c4045d4ee73dce217f22fe8e3e6d52ee11c2ddd225ffe` | | Contents | text, Vision, MTP and DFlash2 components, the 131,072-row proposal head for `--lm-head-draft`; 1190 objects (1184 tensors, 6 frontend resources), 1513 bindings; formats bf16 627, fp32 96, int32 1, q4_g64_fp16 183, q5_g64_fp16 246, q6_g64_fp16 1, q8_g32_fp16 30 | | Built with | ninfer `tools.convert.qwen3_8_27b.convert`, recipe `qwen3_8_27b-v2`, converter revision `487f89773f07cb18a2fb841fe0971ec9634d409b`; container upgraded to v3 with `tools/upgrade_ninfer_v2_to_v3.py` (weights byte for byte) | | Target source | HauhauCS Aggressive **Q8_K_P** GGUF (`Q8_0` matrices, BF16 attention q/k/v and lm_head, F32 control tensors) + the release's BF16 `mmproj` | | Draft source | `z-lab/Qwen3.8-27B-DFlash2` revision `50307d4c4cde6860d4eee73e2547cd786fe8e8a4` (byte-identical to `nerkyor/.../BF16/DFlash2-FP8/model.safetensors`, sha256 `67fc76d6…`) | | Verified | greedy smoke on NInfer-all `master` (RTX 5090): coherent answers without speculation, with MTP (3 drafts) and with DFlash2 (7 drafts), and a chart image described correctly with `--vision` | | First release (v2) | smoke-run on stock NInfer `487f8977` (CUDA 13.1, RTX PRO 4500 Blackwell (32 GB, sm_120)): plain: decode 43.8 tok/s; dflash2: decode 59.8 tok/s, acceptance 27.2% (2.90 tok/round); dflash2_think: decode 112.1 tok/s, acceptance 64.3% (5.50 tok/round) | ## Download & run ```bash hf download WaveCut/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-DFlash2-NInfer-v3 \ Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-DFlash2-ninfer-v3.ninfer --local-dir models # OpenAI/Anthropic-compatible server from the Docker image, on http://localhost:8080/v1: docker run --rm --gpus all -p 8080:8080 --ulimit memlock=-1 \ -v "$PWD/models:/models" -v ninfer-cache:/cache \ ghcr.io/iamwavecut/ninfer-all serve \ /models/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-DFlash2-ninfer-v3.ninfer \ --max-context 32768 --kv-capacity auto --spec dflash2 --draft-tokens 7 --lm-head-draft # From a build of NInfer-all, DFlash2 speculative decoding (7 drafts, optimized head), CLI: ./build/apps/ninfer models/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-DFlash2-ninfer-v3.ninfer --prompt "Summarize the difference between prefill and decode." \ --max-context 16384 --spec dflash2 --draft-tokens 7 --lm-head-draft # OpenAI/Anthropic-compatible server: ./build/apps/ninfer-serve models/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-DFlash2-ninfer-v3.ninfer --max-context 32768 --kv-capacity auto \ --spec dflash2 --draft-tokens 7 --lm-head-draft ``` `--spec mtp --draft-tokens 3 --lm-head-draft` selects the tune's own NextN head instead. Vision (`--vision`) is the official tower: the release's BF16 projector reproduces it bit-for-bit through the FP16 export path, so the artifact carries the official BF16 vision words. **Hardware.** Upstream NInfer targets one RTX 5090 (`sm_120a`, 32 GB). NInfer-all serves the same `qwen3.8-27b / groupwise-int` identity with DFlash2 on the RTX 3090, 4090, 5090 and RTX PRO 6000: its [Docker image](https://github.com/iamwavecut/ninfer-all#docker) needs an NVIDIA driver of the CUDA 13 branch (580 or newer) and the NVIDIA Container Toolkit, and a build ([its README](https://github.com/iamwavecut/ninfer-all#building)) sets `CMAKE_CUDA_ARCHITECTURES` to `86`, `89` or `120a`. The speed of the official artifact of this identity on each card is in its [reference measurements](https://github.com/iamwavecut/ninfer-all/blob/master/docs/performance/reference-2026-09.md), this tune itself was not measured there. On 24 GB cards the official artifact of the same size fits DFlash2 with five drafts beside 258,048 tokens of `rk4v4` KV on the 3090 and 245,760 on the 4090 (`--kv-dtype rk4v4 --gdn-state-fp16 --spec dflash2 --draft-tokens 5`). ## How it was built The tune is published only as GGUF quants, so the target weights were rebuilt into the official `Qwen/Qwen3.8-27B` safetensors layout first, then converted with the unmodified stock converter: 1. **GGUF → HF BF16 (`gguf_to_hf.py`, included).** Every llama.cpp `qwen35` export transform was inverted exactly: the Gated-DeltaNet V-head grouped→tiled reorder (qkv/z/a/b rows, `A_log`, `dt_bias`, conv1d channels, `out_proj` columns), `-exp(A_log)`, the `+1` on zero-centered RMSNorm weights, the conv1d squeeze, and the `mtp.* → blk.64` NextN rename; the `mmproj` inverts the Conv3D temporal split and merger names. `Q8_0` blocks are dequantized exactly like ggml (`f16 scale × int8`, RNE to BF16). 2. **Exactness against the official checkpoint.** This release was exported from an FP16 copy of the weights, so llama.cpp computed `w+1` and `-exp(A_log)` in FP16 before storing F32 and the BF16/F32 words in the GGUF are lossy relative to the original BF16 checkpoint; where re-applying that forward transform to the official `Qwen/Qwen3.8-27B` tensor reproduces the GGUF words bit-for-bit, the official BF16 word was used. For `Q8_0` matrices the official tensor was re-quantized with ggml's rule and compared byte-for-byte: identical codes ⇒ the tune did not touch that matrix ⇒ exact official BF16 words; any difference ⇒ the dequantized tune values. Result: **49 of 1199 tensors carry the tune** (all in text layers 19–53); every other tensor reproduced the GGUF words bit-for-bit from the official checkpoint through the FP16 export path and therefore carries the exact official BF16 words (1150 tensors, 404 of them Q8_0 matrices whose only differences were ±1 rounding flips at quantizer ties with identical block scales). The modified matrices by kind: - `model.language_model.layers.N.linear_attn.out_proj.weight` × 13 - `model.language_model.layers.N.mlp.down_proj.weight` × 16 - `model.language_model.layers.N.mlp.gate_proj.weight` × 7 - `model.language_model.layers.N.mlp.up_proj.weight` × 7 - `model.language_model.layers.N.self_attn.o_proj.weight` × 6 | reconstruction status | tensors | |---|---:| | `official-exact(f16-path)` | 746 | | `official-exact(rounding-noise)` | 404 | | `gguf-dequant(modified)` | 49 | 3. **Vision:** the release's BF16 `mmproj` reproduces the official vision tower bit-for-bit through FP16 (333/333 tensors) — the tune did not touch vision. 4. **Frontend resources** (tokenizer, preprocessor configs, `generation_config.json`) are the official `Qwen/Qwen3.8-27B` files pinned by SHA-256 in the converter. 5. **Converter:** `python3 -m tools.convert.qwen3_8_27b.convert --model --dflash2-model --out qwen3_8_27b_hauhaucs_aggressive_dflash2.ninfer` (92 s on the RTX PRO 4500). Its own report is `reports/v2/qwen3_8_27b_hauhaucs_aggressive_dflash2.ninfer.conversion.json`; the GGUF→HF decisions are in `gguf_to_hf.report.json`. 6. **v3:** `python3 tools/upgrade_ninfer_v2_to_v3.py qwen3_8_27b_hauhaucs_aggressive_dflash2.ninfer Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-DFlash2-ninfer-v3.ninfer` rewrote the container of that v2 file: every weight byte is unchanged and the maintained Qwen3.8 chat template is installed. ## Caveats - **Double quantization on the modified tensors.** Matrices the tune changed exist only as `Q8_0` and are re-quantized into NInfer's groupwise profile (Q4/Q5/Q6/W8 groupwise + BF16). Unmodified matrices carry the exact official BF16 words. - The engine binds this file to the registered `qwen3.8-27b / groupwise-int` profile: architecture, tokenizer and chat template are the official ones; only the weights carry the tune. The tune's own chat template in the GGUF is not embedded. - DFlash2 lossless verification: the target model authorizes every emitted token, so the uncensored behaviour of the tune is unchanged by the draft model; DFlash2 only affects speed/acceptance. - Set the sampler explicitly (`temperature=1.0, top_p=0.95, top_k=20`) if you rely on the tune's intended behaviour. - Greedy decoding with `--spec dflash2` is lossless in distribution but not bit-parity with plain greedy: the verify pass runs a batched floating-point route, so long greedy outputs can diverge by a token (in the smoke run: one word of the haiku after ~80 identical tokens). ## Lineage ``` Qwen/Qwen3.8-27B (1d4bf0f2) ├─ HauhauCS/Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-MTP-GGUF (Q8_K_P + BF16 mmproj, Apache-2.0) │ └─ exact GGUF→HF inversion + official-consistency reconstruction (gguf_to_hf.py) └─ z-lab/Qwen3.8-27B-DFlash2 (50307d4c, MIT) ── DFlash2 companion └─ qwen3_8_27b_hauhaucs_aggressive_dflash2.ninfer (v2, recipe v2) └─ Qwen3.8-27B-Uncensored-HauhauCS-Aggressive-DFlash2-ninfer-v3.ninfer (this repo) ``` Licensed Apache-2.0 for the target weights (both upstreams), MIT for the DFlash2 draft weights.