How to use from
Docker Model Runner
docker model run hf.co/jashepp/Qwable-v1-35B-A3B-MXFP4_MOE_Hybrid-Imatrix-GGUF:MXFP4_MOE_Q
Quick Links

๐Ÿ’Ž Qwable-v1-35B-A3B - Custom Mixed Precision GGUFs with Imatrix

Qwen + Fable ยท An open-weights agentic coding model. 35B Mixture-of-Experts (3B active), built by layering Claude Fable-5 agentic tool-use behavior on top of a Claude Opus 4.7 reasoning distill of Qwen3.6-35B-A3B.

Base model Base model License

This repository contains custom, highly optimized, multi-tier mixed precision GGUF weights for lordx64/Qwable-v1.

โ„น๏ธ For advanced agentic and programming tasks, I highly recommend upgrading to Ornith-1.0-35B for significantly better performance.

Qwable-v1 is a 35B Mixture-of-Experts (3B active) hybrid architecture alternating between standard Attention and Mamba State-Space (SSM) blocks.
Distilled heavily from Claude 4.7 Opus reasoning and Claude Fable-5 agentic traces, on top of Qwen3.6-35B-A3B.

These quants were generated using an external importance matrix bartowski/lordx64_Qwable-v1-GGUF/lordx64_Qwable-v1-imatrix.gguf and manual layer targeting to maximize quality while shrinking the massive VRAM footprint of the Mixture of Experts layers.


๐Ÿ“„ GGUF Files

In order of quality:

Filename Size Quants
Qwable-v1-35B-A3B-MXFP4_MOE_Q8_0_F16-Imatrix.gguf 21.3 GB MXFP4_MOE + Q8_0 + F16
Qwable-v1-35B-A3B-MXFP4_MOE_Q8_0-Imatrix.gguf 20.3 GB MXFP4_MOE + Q8_0

Update 2026-06-27:

  • MXFP4_MOE_Q8_0_F16 - ssm_alpha & ssm_beta is now F16, to lessen looping at high context.
  • MXFP4_MOE_Q8_0_F16 - attn_output is now Q8_0, as it was previously F16 by accident due to incorrect --tensor-type regex.

๐Ÿ” Precision Matrix & Flavor Variations

Standard global quantization presets (like stock MXFP4_MOE) compress the backbone layers uniformly, which degrades the delicate reasoning capabilities of advanced agent models.
This repository provides two distinct manual configuration layouts to balance precision and memory constraints:

1. The Tri-Quant Hybrid Flavor (MXFP4 + Q8_0 + F16)

Qwable-v1-35B-A3B-MXFP4_MOE_Q8_0_F16-Imatrix.gguf - Designed for maximum quality preservation, this layout implements a strict 3-Tier Precision Matrix:

  • Tier 1 (Core & Mamba Gating - F16 Precision):
    • token_embd.weight, output.weight - Protects the critical input/output vocabulary mappings. Adds ~1GB to the file size but dramatically prevents text degradation.
    • ssm_alpha, ssm_beta - Protects the integrity of the Mamba state-space calculations across long-range context tokens.
  • Tier 2 (Backbone & Shared - Q8_0 Precision): ssm_out, *._shexp - Keeps the attention mechanics, and all trailing shared experts at high quality, to protect the logical research loops.
  • Tier 3 (Routed Experts - MXFP4 Precision): ffn_down_exps, ffn_gate_exps, ffn_up_exps - Shrink the massive background expert parameters directly to MXFP4.

2. The Dual-Quant Hybrid Flavor (MXFP4 + Q8_0)

Qwable-v1-35B-A3B-MXFP4_MOE_Q8_0-Imatrix.gguf - Designed for a slightly leaner memory profile, this layout utilizes 2-Tier Precision:

  • Tier 1 (Backbone - Q8_0 Precision): All attention blocks, Mamba structures, vocabulary embeddings, and internal routers use the universal Q8_0 format.
  • Tier 2 (Experts - MXFP4 Precision): The heavy sparse expert blocks are target-quantized directly to MXFP4.

๐Ÿ“ Exact Conversion Details

Because Qwable-v1 utilizes trailing shared-expert vectors at the boundary edge of its alternating architecture (Layer 40 boundary), standard conversion requires boundary bypass instructions using .*_shexp\.weight=Q8_0. These files were converted via llama-quantize utilizing the following manual recipe parameters:

Convert SafeTensors to GGUF:

# Requires python3.12, with `pip install --upgrade transformers`
python convert_hf_to_gguf.py "Qwable-v1/" --outtype f16 --outfile "Qwable-v1_F16.gguf"

Generate Tri-Quant MXFP4_MOE + Q8_0 + F16:

llama-quantize \
  --tensor-type ".*_shexp\.weight=Q8_0" \
  --tensor-type "token_embd\.weight=F16" \
  --tensor-type "^output\.weight=F16" \
  --tensor-type "blk\..*\.(ssm_alpha|ssm_beta)\.weight=F16" \
  --tensor-type "blk\..*\.(ffn_down_exps|ffn_gate_exps|ffn_up_exps)\.weight=MXFP4" \
  --imatrix "bartowski-lordx64_Qwable-v1-imatrix.gguf" \
  "Qwable-v1_F16.gguf" \
  "Qwable-v1-35B-A3B-MXFP4_MOE_Q8_0_F16-Imatrix.gguf" \
  Q8_0

Generate Dual-Quant MXFP4_MOE + Q8_0:

llama-quantize \
  --tensor-type ".*_shexp\.weight=Q8_0" \
  --tensor-type "blk\..*\.(ffn_down_exps|ffn_gate_exps|ffn_up_exps)\.weight=MXFP4" \
  --imatrix "bartowski-lordx64_Qwable-v1-imatrix.gguf" \
  "Qwable-v1_F16.gguf" \
  "Qwable-v1-35B-A3B-MXFP4_MOE_Q8_0-Imatrix.gguf" \
  Q8_0

๐Ÿ“ Local Deployment & llama-server Configuration (config.ini)

To maintain the rock-solid reasoning loop depth of the Fable+Opus distillation and prevent agents from falling into repetitive tool-calling deadlocks, use the following server parameter recommendations.

# --- Samplers (Dynamic & Expressive) ---
temperature = 0.65
top-k = 40
top-p = 0.90
min-p = 0.08
# --- Penalties (Prevent Syntax & Reasoner Corruption) ---
repeat-penalty = 1.00
presence-penalty = 0.00
# --- DRY Sampler (Protects Indentation & Structural Boilerplate) ---
dry-multiplier = 0.8
dry-base = 1.75
dry-allowed-length = 8
dry-penalty-last-n = 1024
dry-sequence-breaker = ["\n", ":", " ", "\t", "\"", ","]
# --- Enforced Execution Graph ---
samplers = temp;top_k;top_p;min_p;dry

Keep reasoning off for this model.

reasoning = off
reasoning-budget = 0
reasoning-format = none

This works well with 256k context window.

Note: If using a larger context window, ensure you use the updated MXFP4_MOE_Q8_0_F16 flavor (updated 2026-06-27) to maintain pristine Mamba state-space precision and lessen looping at high context.


โ„น๏ธ Misc Details

I'm doing this as a side hobby, with my AMD 5900X, 64GB DDR4, RTX 3060 12GB & RTX 5060 Ti 16GB.

In addition to the above configuration, I also use:

slots = 1
parallel = 1
no-warmup = true

flash-attn = on
mlock = false
no-mmap = false
no-context-shift = true

batch-size = 2048
ubatch-size = 256

fit = on
fit-target = 768
main-gpu = 0
split-mode = layer
n-gpu-layers = 999
n-cpu-moe = 0
tensor-split = 16,12
override-tensor = (token_embd)=CUDA0,(vision|vpm|nextn)=CPU

cache-type-k = q8_0
cache-type-v = q8_0

jinja = true
chat-template = jinja
chat-template-file = chat_template.jinja

For further quality and better ssm behaviour, this configuration can help:

context-shift = false
cache-type-k = f16
cache-type-v = f16

๐Ÿค Support the Journey

As a passionate developer, I'm always programming, automating, or experimenting with new ideas.
I love building open-source tools, trying out new web tech, and creating things that don't yet exist, including local AI & quantizing models.

I love sharing these creations to give back to the community.
If my projects have saved you time or helped you out, consider supporting my work below!

๐Ÿ‘‰ Support me on Ko-fi


โœจ Acknowledgments

  • lordx64 for the exceptional Qwable-v1 base model.
  • bartowski for the high-quality importance matrix data anchors.

๐Ÿ“œ License

See lordx64/Qwable-v1#license--terms.

๐Ÿ”— Citation

@misc{lordx64_qwable_v1_2026,
  title  = {Qwable-v1: Agentic coding distillation from Claude Fable-5 onto Qwen3.6-35B-A3B},
  author = {lordx64},
  year   = {2026},
  howpublished = {\url{https://huggingface.co/lordx64/Qwable-v1}},
}
Downloads last month
311
GGUF
Model size
36B params
Architecture
qwen35moe
Hardware compatibility
Log In to add your hardware

4-bit

16-bit

Inference Providers NEW
This model isn't deployed by any Inference Provider. ๐Ÿ™‹ Ask for provider support

Model tree for jashepp/Qwable-v1-35B-A3B-MXFP4_MOE_Hybrid-Imatrix-GGUF

Collection including jashepp/Qwable-v1-35B-A3B-MXFP4_MOE_Hybrid-Imatrix-GGUF