IsValorum/Flint-APEX-I-NanoPlus-V2.1-GGUF

Quick Navigation Index

Independently computed using dedicated cloud compute clusters, funded 100% out-of-pocket.

If you consider these releases to be of high quality and they help you in your day-to-day workflow, consider supporting on Ko-fi. Every contribution directly sustains future cloud compute hours and helps support my university studies.

1. Model Overview & Technical Specifications
2. Importance Matrix Optimization
3. Surgical Quantization Topology (779 Tensors)
4. Recommended Server & Inference Configuration
5. Community Compute Fund & Student Support

---

1. Model Overview & Technical Specifications

This repository provides Flint APEX-I-NanoPlus V2.1, an ultra-efficient surgical quantization of RecursiveSI/flint.

Flint is Recursive's mixture-of-experts language model for coding agents, featuring 39.8 billion total parameters and approximately 2.5 billion active per token. It is specifically designed to investigate codebases, fix bugs, implement complex features, and write comprehensive unit tests.

The architecture features 49 layers: two initial dense SwiGLU MLP layers (blk.0 and blk.1) followed by 47 MoE layers (blk.2 to blk.48) with 256 routed micro-experts (top-k=6) and 1 shared expert. In addition, Flint incorporates learned attention sinks across all 49 layers for streaming stability over long contexts up to 262,144 tokens. The model topology encompasses exactly 779 tensors in GGUF.

NanoPlus V2.1 is engineered to hit the 13 GB footprint target (12.07 GiB in VRAM / 12.96 GB on Hub) for maximum memory bandwidth efficiency and high throughput on 16 GB GPUs with large context windows or partial CPU offloading on 12-16 GB setups. It strictly shields routing matrices, attention sinks, and normalization layers in full 32-bit floating point (F32), while strategically compressing interior routed experts with empirical Importance Matrix guidance.

Key Characteristics

• Base Model: RecursiveSI/flint
• Architecture: FlintForCausalLM (Dense-Front + MoE + StreamingLLM Sinks)
• Total Parameters: 39.79 Billion
• Active Parameters: approx. 2.5 Billion per token
• Layers: 49 (Layers 0-1 Dense SwiGLU, Layers 2-48 MoE)
• Routed Experts: 256 per layer (Top-k = 6)
• Shared Experts: 1 per MoE layer (intermediate size 1024)
• Context Window: 262,144 tokens (Sliding Window 513 with Attention Sinks)
• GGUF Tensor Count: Exactly 779 tensors verified
• Quantization Profile: APEX-I NanoPlus V2.1 (approx. 12.07 GiB in VRAM / approx. 12.96 GB on Hub)

---

2. Importance Matrix Optimization

Quantizing a 256-expert MoE architecture without an Importance Matrix results in routing instability, expert starvation, and severe degradation in long-horizon reasoning.

This release utilizes a comprehensive Importance Matrix calibrated by Bartowski (RecursiveSI_flint-imatrix.gguf), providing full 4-moment statistical distributions across all 531 layer components:

1. First Moment / Mean Activation (.in_sum): Allows llama-quantize vector codebook optimizers (IQ2_S, IQ2_XXS, IQ3_XXS, Q4_K) to compute centered variance ($\sigma^2 = \frac{1}{N}\sum x^2 - \mu^2$) rather than uncentered raw energy, eliminating distortion on biased activations.
2. Extended Distribution Statistics (.stats [12]): Prevents outlier clipping across routed expert weights.
3. Chat-Template Aligned Calibration: Tokenized with Flint's native <|system|>...<|user|>...<|assistant|> chat template.

---

3. Surgical Quantization Topology (779 Tensors)

Every single tensor in Flint has been mapped according to the updated APEX-I V2.1 specification:

Structural Component	Tensor Count	Precision Tier	Technical Rationale
output.weight	1	Q6_K	Preserves vocabulary logit fidelity and generation nuances
output_norm.weight	1	F32	Final sequence normalization numerical shield
token_embd.weight	1	Q4_K	Robust input token embedding representation
blk.*.attn_sinks.weight	49	F32	CRITICAL: Sliding-window initial sink token stabilization
Normalizations (attn_norm, attn_q_norm, attn_k_norm, ffn_norm)	196	F32	Numerical stability and layer dynamic range protection
blk.*.ffn_gate_inp.weight (MoE Routers)	47	F32	Strictly shields gating probabilities to prevent router collapse
Dense FFN (blk.0 & blk.1: ffn_gate, ffn_up, ffn_down)	6	Q4_K	Solid trunk representations before MoE branching
Shared Experts (blk.*.ffn_*_shexp.weight)	141	Q4_K	Preserves foundational representations across all tokens
Attention Projections (attn_q, attn_k, attn_v, attn_output)	196	Q4_K	Fast and accurate key-query attention routing
Boundary Routed Experts (4 early: blk.2..blk.5, 4 late: blk.45..blk.48)	24	IQ3_XXS	Balanced boundary expert protection
Core Routed Experts (blk.6..blk.44, 39 layers)	117	IQ2_S (down/gate) / IQ2_XXS (up)	Optimal compression density hitting the 13 GB target
Total Verified Tensors	779	--	100% Accounting Verified

---

4. Recommended Server & Inference Configuration

Runtime Requirement

Requires llama.cpp build b11803 or higher (e.g. b11821+) which incorporates Pull Request #30928 adding native support for FlintForCausalLM and sliding-window attention sinks.

CLI Launch Command

llama-server \
  -m Flint.APEX-I-NanoPlus-V2.1.gguf \
  --host 0.0.0.0 \
  --port 8080 \
  -c 32768 \
  -ngl 99 \
  --temp 1.0 \
  --top-p 0.95 \
  --top-k 20

---

5. Community Compute Fund & Student Support

All APEX-I and VAL-APEX quantizations are produced independently using rented cloud GPU/CPU clusters and validated through tensor-level accounting.

If you find this model useful in your research or production environment, please consider supporting future releases: