Refresh EAGLE-3 references for the thinking-aware v2 head (2026-07-24 retrain); concurrency section re-measured (501/878/1017 tok/s, ITL 6.2-7.6ms)

README.md

@@ -152,16 +152,16 @@ inter-token latency stays essentially flat under load.
 
 | concurrent streams | aggregate output tok/s | TTFT p50 (ms) | ITL p50 (ms) |
 |---|---|---|---|
-| 4 | 435 | 56 | 7.7 |
-| 8 | 750 | 187 | 8.2 |
-| 16 | 993 | 303 | 8.5 |
+| 4 | 501 | 166 | 6.2 |
+| 8 | 878 | 70 | 6.7 |
+| 16 | 1017 | 242 | 7.6 |
 
 Measured with [GuideLLM](https://github.com/vllm-project/guidellm) on an RTX 5090,
-vLLM v0.25.1, real multi-kB article prompts. Single-stream decode is the 154-244
-tok/s in the Speed table; this shows the same server saturating to ~1000 tok/s
-aggregate at concurrency 16 with per-token latency near-constant, i.e. it batches
-well for multi-user / parallel-agent workloads. Spec acceptance held over the run
-(165k accepted / 502k drafted).
+vLLM v0.25.1, real multi-kB article prompts, EAGLE-3 v2 (thinking-aware) head. The
+server saturates to ~1000 tok/s aggregate at concurrency 16 with per-token latency
+near-flat (6.2 to 7.6 ms), so it batches well for multi-user / parallel-agent
+workloads. TTFT p50 is queueing-dominated at these request counts, not a decode
+signal. Spec acceptance held over the run (154k accepted / 354k drafted, 0.44).
 
 ## Serving (vLLM)
 
@@ -180,10 +180,11 @@ vllm serve <this-repo> --trust-remote-code \
 ```
 
 Since 2026-07-23 the recommended speculation is my
-[EAGLE-3 draft head](https://huggingface.co/NullSense/Nanbeige4.2-3B-EAGLE3):
-221 freeform / 244 short-passage summarize / 262 code tok/s in one config
-(TRITON_ATTN required with it; FlashInfer + spec drops CUDA graphs to piecewise on
-SM120). One measured exception: LONG-document summarization (multi-k-char inputs)
+[EAGLE-3 draft head](https://huggingface.co/NullSense/Nanbeige4.2-3B-EAGLE3),
+thinking-aware since the 2026-07-24 retrain: thinking-mode acceptance 0.41 (0.33
+before), and +12-41% single-stream decode over the prior head depending on context,
+in one config (TRITON_ATTN required with it; FlashInfer + spec drops CUDA graphs to
+piecewise on SM120). One measured exception: LONG-document summarization (multi-k-char inputs)
 still favors the ngram config (`{"method":"ngram","num_speculative_tokens":8,
 "prompt_lookup_max":4,"prompt_lookup_min":2}`): 231 vs 160 output tok/s on 16 real
 articles; verbatim copy-spans reward lookup drafting. Pick per dominant workload.
@@ -230,7 +231,7 @@ transformers-5 loading (llmcompressor, finetuning); vLLM serving does not use th
 | [FP8-Dynamic](https://huggingface.co/NullSense/Nanbeige4.2-3B-FP8-Dynamic) (this repo) | 4.9 GB | 154 / 206 | default: no measured quality loss on any gate. Recommended. |
 | [NVFP4-FP8-LoopShield](https://huggingface.co/NullSense/Nanbeige4.2-3B-NVFP4-FP8-LoopShield) | 4.5 GB | 159 / 202 | smallest artifact that keeps reasoning at FP8 parity; best JSON reliability. Recommended for tight VRAM. |
 | [NVFP4A16](https://huggingface.co/NullSense/Nanbeige4.2-3B-NVFP4A16) | 3.6 GB | 175 / 220 | fastest; summarization/extraction only (reasoning drops 8 GSM8K points). Not for math/agentic. |
-| [EAGLE3 draft](https://huggingface.co/NullSense/Nanbeige4.2-3B-EAGLE3) | +1.5 GB | 221 / 244 (262 code) | add-on speculator for any of the above: +33-58% decode, lossless. Serve with TRITON_ATTN. |
+| [EAGLE3 draft](https://huggingface.co/NullSense/Nanbeige4.2-3B-EAGLE3) | +1.5 GB | +12-41% decode | add-on speculator for any of the above; thinking-aware retrain (2026-07-24), thinking-mode acceptance 0.41, lossless. Serve with TRITON_ATTN. |
 
 All three serve identically (same plugin, same flags); only the checkpoint differs.
 Rule of thumb: this model + 64k fp8-KV context fits in ~7 GB VRAM at FP8, ~6 GB at