Clarify v2 and v3 quantization recipes

README.md

@@ -30,20 +30,29 @@ Both v3 files were quantized with ik_llama.cpp at `d9e2868`, which includes the
 
 ## Choosing a file
 
-| File | On the GPU with the example invocation | Routed experts, in RAM or VRAM | English KLD | Code KLD |
-| --- | ---: | ---: | ---: | ---: |
-| v3 IQ4_KT | 4.5 GiB | 52.6 GiB | 0.058 | 0.056 |
-| v3 IQ3_KT | 4.5 GiB | 47.2 GiB | 0.085 | 0.078 |
-| [IQ4_KT_v2](https://huggingface.co/ji-farthing/Qwen3.8-Flash-Next-ik-llama-GGUF_v2) | 2.4 GiB | 52.6 GiB | 0.133 | 0.125 |
+The version identifies the quantization recipe. All three recipes use the same upstream model.
+
+| Recipe | Total file size | GPU-resident model tensors with the documented offload configuration | Routed experts in RAM or VRAM | English mean KLD | Code mean KLD |
+| --- | ---: | ---: | ---: | ---: | ---: |
+| v3 IQ4_KT | 82.75 GiB | 4.5 GiB | 52.6 GiB | 0.058 | 0.056 |
+| v3 IQ3_KT | 72.52 GiB | 4.5 GiB | 47.2 GiB | 0.085 | 0.078 |
+| [IQ4_KT_v2](https://huggingface.co/ji-farthing/Qwen3.8-Flash-Next-ik-llama-GGUF_v2#files) | 80.72 GiB | 2.4 GiB | 52.6 GiB | 0.133 | 0.125 |
+
+Choose v3 IQ4_KT for the lowest measured KL divergence. Choose v3 IQ3_KT to reduce routed-expert memory. Choose IQ4_KT_v2 to reduce GPU-resident model tensors by about 2 GiB relative to v3.
+
+The v3 files store attention, SSM, hyper-connection, and shared-expert tensors at Q8_0. The DSA indexer projections use BF16. Most of these tensors use IQ4_KT in v2. The v3 IQ3_KT reduces routed-expert precision further than v3 IQ4_KT.
 
-The v3 files store the attention, SSM, hyper-connection, and shared-expert tensors at Q8_0 and the DSA indexer projections at BF16. In the IQ4_KT_v2, most of these tensors use IQ4_KT. The v3 recipe lowers the KL divergence and keeps about 2 GiB more on the GPU. The v3 IQ3_KT needs less RAM for the routed experts than the v3 IQ4_KT, but the same VRAM.
+These GPU figures exclude the KV cache and compute buffers. Total VRAM use depends on context length, micro-batch size, cache types, and offload settings. The KV cache and compute buffer take about 7.5 GiB at 256K context with `-ub 1024` and the example cache types. Total file size is not the RAM requirement when `--defer-ple` leaves the PLE table on disk.
 
-The KV cache and the compute buffer grow with the context length (`-c`), and the compute buffer also grows with the micro-batch size (`-ub`). At 256K context and `-ub 1024`, with the cache types of the example invocation, the two take about 7.5 GiB, more than the difference between the files. Each way to use less VRAM has a cost:
+A lower `-c` reduces context length. A lower `-ub` reduces prompt-processing speed. Moving more tensors to the CPU with `-ot` reduces decoding speed.
 
-- A lower `-c` gives less context.
-- A lower `-ub` makes prompt processing slower.
-- The IQ4_KT_v2, the faster file, has a higher KL divergence.
-- Moving more tensors to the CPU with `-ot` makes decoding slower.
+KLD measures divergence from the BF16 reference on the documented samples. Lower KLD does not establish correctness of generated code. These KLD figures do not establish a throughput ranking.
+
+## Why there are two repositories
+
+IQ4_KT_v2 remains in the `_v2` repository. This main card covers v2 and v3. The `_v2` suffix identifies an earlier recipe. It does not identify the latest release.
+
+The optional MTP draft head is available in this repository. The `_v2` repository contains only the v2 main model.
 
 ## Files
 
@@ -55,17 +64,22 @@ The KV cache and the compute buffer grow with the context length (`-c`), and the
 | [`Qwen3.8-Flash-Next-ik_llama-v3-IQ3_KT-00001-of-00002.gguf`](https://huggingface.co/ji-farthing/Qwen3.8-Flash-Next-ik-llama-GGUF/resolve/main/Qwen3.8-Flash-Next-ik_llama-v3-IQ3_KT-00001-of-00002.gguf?download=true) | v3 IQ3_KT | 44.33 GiB |
 | [`Qwen3.8-Flash-Next-ik_llama-v3-IQ3_KT-00002-of-00002.gguf`](https://huggingface.co/ji-farthing/Qwen3.8-Flash-Next-ik-llama-GGUF/resolve/main/Qwen3.8-Flash-Next-ik_llama-v3-IQ3_KT-00002-of-00002.gguf?download=true) | v3 IQ3_KT | 28.19 GiB |
 | v3 IQ3_KT total | | 72.52 GiB |
+| [`Qwen3.8-Flash-Next-ik_llama-IQ4_KT_v2-00001-of-00002.gguf`](https://huggingface.co/ji-farthing/Qwen3.8-Flash-Next-ik-llama-GGUF_v2/resolve/main/Qwen3.8-Flash-Next-ik_llama-IQ4_KT_v2-00001-of-00002.gguf?download=true) | IQ4_KT_v2 | 44.41 GiB |
+| [`Qwen3.8-Flash-Next-ik_llama-IQ4_KT_v2-00002-of-00002.gguf`](https://huggingface.co/ji-farthing/Qwen3.8-Flash-Next-ik-llama-GGUF_v2/resolve/main/Qwen3.8-Flash-Next-ik_llama-IQ4_KT_v2-00002-of-00002.gguf?download=true) | IQ4_KT_v2 | 36.31 GiB |
+| IQ4_KT_v2 total | | 80.72 GiB |
 | [`Qwen3.8-Flash-Next-MTP-ik_llama-IQ4_KT.gguf`](https://huggingface.co/ji-farthing/Qwen3.8-Flash-Next-ik-llama-GGUF/resolve/main/Qwen3.8-Flash-Next-MTP-ik_llama-IQ4_KT.gguf?download=true) | IQ4_KT | 2.21 GiB |
 | [`qwen4exp-agentic-v3.imatrix`](https://huggingface.co/ji-farthing/Qwen3.8-Flash-Next-ik-llama-GGUF/resolve/main/qwen4exp-agentic-v3.imatrix?download=true) | n/a | 312 MiB |
 | [`qwen4exp-mtp-v1-draft.imatrix`](https://huggingface.co/ji-farthing/Qwen3.8-Flash-Next-ik-llama-GGUF/resolve/main/qwen4exp-mtp-v1-draft.imatrix?download=true) | n/a | 6.6 MiB |
 
-Download both shards of one quant into the same directory and pass the first shard to `llama-server`.
+The v2 download links point to the `_v2` repository.
+
+Download both shards of one quant into the same directory. Pass the first shard to `llama-server`.
 
-The routed expert `ffn_down_exps` tensors use IQ3_KT. The routed expert gate/up tensors use IQ4_KT in the v3 IQ4_KT. In the v3 IQ3_KT they use IQ4_KT in blocks 0 to 5 and 38 to 47 and IQ3_KT in blocks 6 to 37. The PLE table uses IQ4_KT in the v3 IQ4_KT and IQ3_KT in the v3 IQ3_KT. The DSA indexer projections use BF16 and the four-column PLE convolution F16. Every other two-dimensional weight tensor uses Q8_0, including the token embeddings, the output head, the attention projections, the hyper-connection tensors, and the shared experts.
+In the v3 files, the routed expert `ffn_down_exps` tensors use IQ3_KT. The routed expert gate/up tensors use IQ4_KT in the v3 IQ4_KT. In the v3 IQ3_KT they use IQ4_KT in blocks 0 to 5 and 38 to 47 and IQ3_KT in blocks 6 to 37. The PLE table uses IQ4_KT in the v3 IQ4_KT and IQ3_KT in the v3 IQ3_KT. The DSA indexer projections use BF16 and the four-column PLE convolution F16. Every other two-dimensional weight tensor uses Q8_0, including the token embeddings, the output head, the attention projections, the hyper-connection tensors, and the shared experts.
 
 ## MTP draft head
 
-`Qwen3.8-Flash-Next-MTP-ik_llama-IQ4_KT.gguf` is the model's own next-token prediction head, converted from the same BF16 revision as the main files. It is not a standalone model, and it is optional. Load it beside either main file with `-md` and enable it with `--spec-type mtp:n_max=1,p_min=0.0`. If the head does not fit beside the main model, add `-draft "-ub 256 -b 256"`. It gives the head a separate batch size of 256, which makes its compute buffer smaller.
+`Qwen3.8-Flash-Next-MTP-ik_llama-IQ4_KT.gguf` is the model's own next-token prediction head, converted from the same BF16 revision as the main files. It is not a standalone model, and it is optional. Load it beside a v2 or v3 main model with `-md`. Enable it with `--spec-type mtp:n_max=1,p_min=0.0`. If the head does not fit beside the main model, add `-draft "-ub 256 -b 256"`. It gives the head a separate batch size of 256, which makes its compute buffer smaller.
 
 The head carries its own hyper-connection mixer, so the loader binds that rather than falling back to the trunk's. Its importance matrix was collected paired against the IQ3_KT published here on 2026-09-01, on the same corpus as the main files. With the v3 IQ4_KT, `llama-spec-bench` accepted 90.41%, 79.43%, and 68.21% of its drafts on the built-in code, extract, and story tasks (temperature 0, 256 tokens).