---
license: other
license_name: lfm1.0
base_model: LiquidAI/LFM2-8B-A1B
base_model_relation: quantized
tags: [gguf, llama.cpp, rocm, gfx1151, strix-halo, amd, ryzen-ai-max-395, rocmfp4, rocmfpx, lfm2, liquid, moe]
language: [en]
pipeline_tag: text-generation
---

> ### ⚠️ STOCK `llama.cpp` WILL NOT LOAD THESE
>
> The `ROCmFP4` / `ROCmFPX` quant types exist only in
> [`charlie12345/ROCmFPX`](https://github.com/charlie12345/ROCmFPX) — stock llama.cpp reports
> `invalid ggml type`. Ignore the auto-generated "Use this model" commands above.

# LFM2-8B-A1B — ROCmFP4 / ROCmFPX GGUF

**The first ROCmFP4 quantizations of LFM2-8B-A1B.** Built for **AMD gfx1151
(Ryzen AI MAX+ 395 / Strix Halo)**. 24 layers · hidden 2048 · **32 experts**, ~1B active.

Quantized from **BF16 GGUF** — a lossless source, not a requantization of a lower-bit build.

## All four quants — pick by size, not by hoping

| file | ftype | size | bpw | decode (median of 3) | runs | best for |
|---|---|---|---|---|---|---|
| `Q4_0_ROCMFP4_COHERENT` | 102 | 4.41 GiB | 4.54 | **146.61** | 150.64 / 146.32 / 146.61 | smallest — best fit for a 128 GB Strix Halo |
| `Q6_0_ROCMFPX_AGENT` | 114 | 7.14 GiB | 7.35 | **104.09** | 104.09 / 104.34 / 103.95 | agent/tool-call routing at 6-bit |
| `Q8_0_ROCMFPX` | 111 | 8.02 GiB | 8.26 | **102.97** | 103.77 / 102.91 / 102.97 | highest fidelity, plain routing |
| `Q8_0_ROCMFPX_AGENT` | 115 | 8.13 GiB | 8.37 | **104.31** | 104.31 / 103.87 / 104.78 | highest fidelity + agent routing |

Measured on a Ryzen AI MAX+ 395 (gfx1151, ROCm), `-ngl 999 -c 4096 -fa on -fit off`,
median of 3 with warm-up discarded on an otherwise-idle box.

**Correctness:** 3/3 on every quant — 17x23 => `391` - capital of Japan => `Tokyo` - days in 2024 => `366` (max_tokens 1024)

**Speculative decoding:** **None.** No MTP / EAGLE / nextn tensors, so there is no speculative speedup to enable and no acceptance length to quote. Source: `unsloth/LFM2-8B-A1B-GGUF` BF16.

> ℹ️ **Why no "4-bit AGENT"?** The `AGENT` recipes exist only at 3-bit (113), 6-bit (114/117) and
> 8-bit (115) — the entire 4-bit family (100–106) has no AGENT variant. Rather than hand-roll one
> and publish it under a name implying it is standard, the 6-bit `Q6_0_ROCMFPX_AGENT` fills that
> slot. `AGENT` routing keeps more tensors at higher precision for tool-call coherence.

> ℹ️ **Head protection.** These GGUFs contain **no separate `output.weight`** — the embedding is
> tied — so `--token-embedding-type` is what protects the head here and
> `--output-tensor-type` is a no-op. Audited in the finished files: `token_embd` is **Q6_K** on
> the 4-bit build and **Q8_0** on the 6/8-bit builds.

---

## What was NOT measured
- **No perplexity run**, and no quality A/B against the BF16 source.
- **No long-context testing.** · **No tool-calling evaluation** — note the `AGENT` builds are
  *named* for tool-call routing but we did not benchmark tool use.

Base model licence inherited; all credit for the weights goes to Liquid AI.