ukint-vs commited on
Commit
a55c228
·
verified ·
1 Parent(s): 5864ec7

Clarify INT8 quantization in card and metadata

Browse files
README.md CHANGED
@@ -1,7 +1,7 @@
1
  ---
2
  license: apache-2.0
3
- base_model:
4
- - Nanbeige/Nanbeige4.2-3B
5
  pipeline_tag: text-generation
6
  library_name: coreai
7
  tags:
@@ -9,17 +9,35 @@ tags:
9
  - apple-silicon
10
  - recurrent-transformer
11
  - text-generation
 
 
12
  ---
13
 
14
- # Nanbeige4.2-3B — Core AI
15
 
16
- Core AI conversion of [`Nanbeige/Nanbeige4.2-3B`](https://huggingface.co/Nanbeige/Nanbeige4.2-3B), pinned to source revision [`5ff54fb7ed86ce8e216d78bff5417ab9981de3d4`](https://huggingface.co/Nanbeige/Nanbeige4.2-3B/tree/5ff54fb7ed86ce8e216d78bff5417ab9981de3d4).
17
 
18
- Ported by [Vadim Smirnov (@ukint-vs)](https://github.com/ukint-vs). Source, reproducible conversion recipe, parity gates, and the full optimization report are tracked in [john-rocky/coreai-model-zoo#5](https://github.com/john-rocky/coreai-model-zoo/issues/5).
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
19
 
20
  ## Architecture
21
 
22
- Nanbeige4.2 is a looped Llama: **22 physical transformer blocks execute twice**, with RMS normalization after each pass and independent KV history. The bundle therefore has **44 executed/cache layers while storing one copy of the 22 physical blocks**.
23
 
24
  ## Bundle
25
 
@@ -33,7 +51,7 @@ gpu-pipelined/
33
  tokenizer/
34
  ```
35
 
36
- Configuration: static-S=1, int8 body per-block-32, symmetric absmax int8 untied head, 4,096-token verified context. The embedded tokenizer includes the vendor `chat_template.jinja` with thinking enabled and disabled modes. The `.aimodel` producer is `coreai-core 1.0.0b2`.
37
 
38
  ## Verification
39
 
@@ -41,7 +59,6 @@ Configuration: static-S=1, int8 body per-block-32, symmetric absmax int8 untied
41
  - Quantized Core AI engine: token-exact vs fp32 on 24-token Paris, 16-token freezing-point, and 64-token decimal-reasoning gates.
42
  - M4 Max Release benchmark, prompt 128 / generation 256 / 3 runs: **47.37 prefill / 46.35 decode tok/s**.
43
  - 4,096-token boundary: **29.83 prefill / 32.80 decode tok/s**, 9.17 GiB peak RSS, zero swaps.
44
- - int4 and mixed int4/int8 candidates failed the reasoning quality gate and are intentionally not published.
45
  - iPhone `h18p` device acceptance is pending; no iPhone performance claim is made.
46
 
47
  ## License
 
1
  ---
2
  license: apache-2.0
3
+ base_model: Nanbeige/Nanbeige4.2-3B
4
+ base_model_relation: quantized
5
  pipeline_tag: text-generation
6
  library_name: coreai
7
  tags:
 
9
  - apple-silicon
10
  - recurrent-transformer
11
  - text-generation
12
+ - quantized
13
+ - int8
14
  ---
15
 
16
+ # Nanbeige4.2-3B — Core AI INT8
17
 
18
+ **INT8 weight-quantized Core AI LanguageBundle** converted from [`Nanbeige/Nanbeige4.2-3B`](https://huggingface.co/Nanbeige/Nanbeige4.2-3B), pinned to source revision [`5ff54fb7ed86ce8e216d78bff5417ab9981de3d4`](https://huggingface.co/Nanbeige/Nanbeige4.2-3B/tree/5ff54fb7ed86ce8e216d78bff5417ab9981de3d4).
19
 
20
+ This is a Core AI `.aimodel` bundle for Apple silicon. It is not a Transformers, GGUF, or MLX checkpoint.
21
+
22
+ Ported by [Vadim Smirnov (@ukint-vs)](https://github.com/ukint-vs). Source, reproducible conversion recipe, parity gates, and the optimization report are tracked in [john-rocky/coreai-model-zoo#5](https://github.com/john-rocky/coreai-model-zoo/issues/5).
23
+
24
+ ## Format
25
+
26
+ | Property | Value |
27
+ |---|---|
28
+ | Published variant | **INT8 only** |
29
+ | Body weights | int8, per-block 32 (`int8hu`) |
30
+ | Untied LM head | symmetric absmax int8, per-block 32 (`--head-sym`) |
31
+ | Activations / KV cache | fp16 |
32
+ | Decode input | static sequence length 1 (`--static-ids`) |
33
+ | Bundle size | approximately 4.59 GiB |
34
+ | Verified context | 4,096 tokens |
35
+
36
+ The int4 and mixed int4/int8 candidates failed the reasoning quality gate and are intentionally not published.
37
 
38
  ## Architecture
39
 
40
+ Nanbeige4.2 is a looped Llama: **22 physical transformer blocks execute twice**, with RMS normalization after each pass and independent KV history. The bundle therefore has **44 executed/cache layers while storing and quantizing one copy of the 22 physical blocks**.
41
 
42
  ## Bundle
43
 
 
51
  tokenizer/
52
  ```
53
 
54
+ The embedded tokenizer includes the vendor `chat_template.jinja` with thinking enabled and disabled modes. The `.aimodel` producer is `coreai-core 1.0.0b2`.
55
 
56
  ## Verification
57
 
 
59
  - Quantized Core AI engine: token-exact vs fp32 on 24-token Paris, 16-token freezing-point, and 64-token decimal-reasoning gates.
60
  - M4 Max Release benchmark, prompt 128 / generation 256 / 3 runs: **47.37 prefill / 46.35 decode tok/s**.
61
  - 4,096-token boundary: **29.83 prefill / 32.80 decode tok/s**, 9.17 GiB peak RSS, zero swaps.
 
62
  - iPhone `h18p` device acceptance is pending; no iPhone performance claim is made.
63
 
64
  ## License
gpu-pipelined/nanbeige4_2_3b_decode_int8hu_block32_sym_s1/metadata.json CHANGED
@@ -21,9 +21,11 @@
21
  "hf_model_id": "Nanbeige/Nanbeige4.2-3B",
22
  "hf_revision": "5ff54fb7ed86ce8e216d78bff5417ab9981de3d4"
23
  },
24
- "compression": null,
 
 
25
  "compilation": {
26
  "date": "2026-07-22T15:06:28.708628+00:00",
27
  "targets": []
28
  }
29
- }
 
21
  "hf_model_id": "Nanbeige/Nanbeige4.2-3B",
22
  "hf_revision": "5ff54fb7ed86ce8e216d78bff5417ab9981de3d4"
23
  },
24
+ "compression": {
25
+ "scheme": "int8hu"
26
+ },
27
  "compilation": {
28
  "date": "2026-07-22T15:06:28.708628+00:00",
29
  "targets": []
30
  }
31
+ }