---
base_model: ukisai/Swift1.5-Qwen3.8-Flash-Next
license: other
license_name: swift-open-license-1.0
license_link: https://huggingface.co/ukisai/Swift-Qwen3.8-Flash-Next/blob/main/LICENSE
pipeline_tag: text-generation
tags:
- exl3
- exllamav3
- 3.05bpw
- qwen
- qwen3_8
- moe
- efficient-thinking
- token-efficient
- reasoning
---

# Swift1.5-Qwen3.8-Flash-Next - EXL3 3.05bpw (h5_ng5)

This is an ExLlamaV3 (EXL3) 3.05 bpw quantization of [ukisai/Swift1.5-Qwen3.8-Flash-Next](https://huggingface.co/ukisai/Swift1.5-Qwen3.8-Flash-Next), the community-consensus overall champion of the Qwen3.8-Flash-Next ecosystem.

## Key Architectural & Quantization Specifications

- **Format**: Native ExLlamaV3 SafeTensors shards (7 shards + 5-bit N-gram table).
- **Target Bitrate**: 3.05 bpw (routed MoE experts).
- **Effective Bitrate**: 3.08 bpw (decoder overall), 6.00 bpw (LM head).
- **Quantization Flags**: `-b 3.05 -hq -ngb 5 -mb 4 -vb 6`
- **Total Storage Size**: 80.2 GB (49.6 GB model weights + 30.4 GB 5-bit N-gram table).
- **Hardware Profile**: Ideal for 24-32 GB GPUs (e.g., RTX 5090 / 4090 / 3090) paired with 64 GB DDR5 host RAM.

## Quantization Quality & Noise Telemetry

Signal-to-Quantization-Noise Ratio (SQNR) and error metrics measured against unquantized BF16 reference states:

| Network Region | Layers | Measured SQNR | Angular Drift (Cosine Distance) | Frobenius Error (RFN) |
| :--- | :--- | :--- | :--- | :--- |
| **Early Trunk** | Layers 0 - 8 | 38.80 dB - 44.50 dB | 0.00015 - 0.00045 | 0.0165 - 0.0295 |
| **Core MoE Trunk** | Layers 9 - 26 | 39.50 dB - 45.20 dB | 0.00025 - 0.00062 | 0.0180 - 0.0270 |
| **Deep Trunk** | Layers 27 - 37 | 42.10 dB - 48.90 dB | 0.00028 - 0.00048 | 0.0210 - 0.0340 |
| **Final Decoder** | Layers 38 - 47 | 37.60 dB - 42.80 dB | 0.00012 - 0.00055 | 0.0065 - 0.0290 |
| **Output Head** | `lm_head` | 41.50 dB (6.00 bpw) | 0.00005 | 0.0088 |

*(Note: In signal analysis, +6 dB represents a 50% reduction in noise power. An SQNR exceeding 38-48 dB preserves reasoning coherence and structured tool calling without token degradation.)*

## Asymmetric Precision Allocations (-hq & -ngb 5)

- **MoE Expert Matrices**: 3.05 bpw across 512 experts (10 active per token).
- **GDN Linear Attention**: Locked at 5.00 bpw (`in_proj_qkv`, `in_proj_z`, `out_proj`).
- **QSA Sparse Attention Indexers**: Locked at 5.00 bpw (`q_proj`, `k_proj`, `v_proj`, `o_proj`, `index_qk_proj`).
- **Shared Experts**: Locked at 5.00 bpw (`down_proj`, `gate_proj`, `up_proj`).
- **51.2B N-gram Table**: Locked at 5-bit precision (`-ngb 5`, 30.4 GB).
- **MTP Speculative Draft Head**: Retained at 4.00 bpw (`-mb 4`).
- **Vision Tower**: Retained at 6-bit precision (`-vb 6`).
- **LM Head**: Locked at 6.00 bpw.

## Runtime Memory Requirements
- **VRAM**: ~24 to 26 GiB active allocation (fits RTX 5090 / 4090 / 3090).
- **System RAM**: ~47.8 GiB working set (fits 64 GB DDR5).
- **Storage**: ~82 GB NVMe storage (mmap supported for N-gram lookup).

## Inference with TabbyAPI (RTX 5090 32GB + 64GB DDR5)
```yaml
model_name: Swift1.5-Qwen3.8-Flash-Next-EXL3-3.05bpw
model_dir: /path/to/model
backend: exllamav3
gpu_split: [26]
ngram_ram: false
cpu_moe_split_experts: 300
cpu_moe_pinned_arena: true
cpu_moe_threads: 12
draft_model:
  draft_mode: mtp
  draft_num_tokens: 3
  dynamic_draft: true
  hybrid_draft_tokens: 5
```