GLM-5.2 — colibrì int4 container (~370 GB)
This is the *EXACT SAME THING* as https://huggingface.co/jlnsrk/GLM-5.2-colibri-int4, but with int8 MTP heads
⚠️ This is NOT a GGUF / AWQ / GPTQ / MLX model. It only works with the colibrì engine.
Usage
# Get the engine
git clone https://github.com/JustVugg/colibri && cd colibri/c && ./setup.sh
# download this repo to a FAST local disk
hf download mateogrgic/GLM-5.2-colibri-int4-with-int8-mtp --local-dir /nvme/glm52
# run on CPU
COLI_MODEL=/nvme/glm52 ./coli chat
Requirements: Linux (or WSL2), gcc + OpenMP, AVX2, ≥16 GB RAM, ~400 GB free NVMe.
Provenance & license
Converted from zai-org/GLM-5.2-FP8 (MIT). This derivative is likewise MIT. Conversion performed with colibrì's official converter, unmodified. Cloned & modded from jlnsrk/GLM-5.2-colibri-int4
- Downloads last month
- 15
Inference Providers NEW
This model isn't deployed by any Inference Provider. 🙋 Ask for provider support
Model tree for gubernac/GLM-5.2-colibri-int4-with-int8-mtp
Base model
zai-org/GLM-5.2-FP8