aicivs-joint-p1-v3b2-w4a16

The AICivs student: MiniCPM5-2B fine-tuned (LoRA r64, merged) to power the NPCs of AICivs, a Minecraft mod where villages are living civilizations — villagers with personalities, memories and relationships, quests grounded in the world around the giver, wars, famines, successions, gossip and authored histories. One 2.5B-parameter model answers every operation of the mod's service contract, so the whole runtime fits in about 8 GB of GPU memory next to the game.

This build: GPTQ W4A16 (group 128) via llm-compressor, calibrated on 252 of the service's own prompts; loads in vLLM with the Marlin kernels (--dtype bfloat16, no extra flags). ~2 GB. Quality matched the bf16 merge on the serving-path eval (dialogue validator 100 %, intent vs teacher 73 % vs 75 %); p95 962 ms at 24 concurrent dialogue calls on an RTX 3090 Ti.

What it is for

The model answers nine operations, each a fixed JSON schema (AICivs contract 1.0, prompt version 1):

operation	what the game asks	latency budget
dialogue.respond	a villager's in-character reply to a player line: speech, the player's intent, relationship deltas, mood, actions (offer work, trade, reveal a secret, refer, end the talk), a memory to keep	1.5 s
gossip.render	a villager repeating a rumour, hedged by how many hops it travelled	1 s
quest.generate	a quest from a manifest of real referents: nodes (acquire, deliver, travel, kill, escort, interact, build, defend, learn, gift, wait), motives, six dialogue lines, a reward within budget, chronicle bindings	8 s
quest.repair	the same quest fixed against a list of validation errors, with the changes listed	5 s
npc.personality	traits, speech register, motivations, quirks and a voice line for a batch of new villagers	5 s
npc.memory.consolidate	episodic memories about a player folded into beliefs, with sources and retired memory ids	10 s
civ.event.narrate	a chronicle account of a sim event from its computed facts, plus faction versions, memory injections and mood shifts	15 s
civ.chronicle.summarize	a decade / era / legend summary of a civ's chronicle with the events worth keeping	60 s
civ.history.author	a civ's founding history: eras, events, figures, dynasties, the buildings history leaves behind	300 s

It is not a general chat model. It expects the service's rendered prompts (a short system prompt naming the speaker, a user turn carrying the context and the allowed actions) and answers in the contract's JSON. Outside that framing it will still answer, but nothing about the game's grounding rules applies.

How to use it

The intended path is the AICivs stack: ops/aicivs-up.sh in the mod's repository downloads this repo, starts the servers and the service, and the mod calls the service. To serve the model yourself:

docker run --gpus all -p 8003:8000 --ipc=host -v $PWD/aicivs-joint-p1-v3b2-w4a16:/models/student:ro vllm/vllm-openai:v0.22.0 \
  --model /models/student --served-model-name aicivs-joint-p1-v3b2-w4a16 --dtype bfloat16 \
  --max-model-len 8192 --gpu-memory-utilization 0.20 --max-num-seqs 64 --enable-prefix-caching

vLLM picks the Marlin int4 kernels by itself. With 20 % of a 24 GB card: p50 643 / p95 962 ms at 24 concurrent dialogue calls. Use this build for dialogue.respond and gossip.render; for the structured operations prefer the GGUF build on llama.cpp, whose grammar also enforces the contract's regex patterns (vLLM's silently drops a schema that carries one).

Send OpenAI-style chat completions with response_format: {"type": "json_schema", ...} carrying the operation's schema (the pydantic models in service/aicivs_service/contract/) and chat_template_kwargs: {"enable_thinking": false} — the model was trained with thinking off. Temperature 0.8 for dialogue and gossip, 0.6 for the structured operations.

A real dialogue.respond exchange, exactly as the service renders it:

system: You are Kagard Mikae, child of Nolhou. Voice: childlike — e.g. "I'm Kagard Mikae! Are you from far away?" /
        "I'm not supposed to talk to strangers.".
        Traits: timid. Mood: neutral. Toward the speaker you are devoted to; quirk: counts on fingers.
        Answer in character, at most 30 words, as JSON. Never mention a person, place, item or event that is not
        listed below; use listed ids exactly. Only use actions you are told you may use.
user:   You may: end the talk (end_conversation) / ask for: tag:minecraft:iron_ingot, tag:minecraft:emerald /
        reveal: topic:nolhou_h1 / show emotion (express_emotion).
        Conversation:
        Player: What a miserable little life you have.

{"speech":"I'm not supposed to talk to strangers. Please don't be mean to me.","player_intent":"insult",
 "sincerity":"honest","actions":[],"relationship_delta":{"trust":-10,"affection":-15,"respect":-5,"fear":0},
 "patience_delta":-0.2,"mood_after":"afraid",
 "memory_write":{"summary":"Player insulted my life and me.","salience":0.8,"valence":-1.0,"tags":["insult","fear","hurt"]}}

Two things the serving side must do, because grammar backends do not: clamp numeric bounds (vLLM's grammar ignores minimum/maximum, so a -1.1 valence must be clamped, not rejected) and validate references — every id in an answer is checked against the request's manifest, and the game repairs or falls back on a miss.

How it was made

Prompt distillation (the mod's docs/08-finetuning.md): a 12-civilization world simulated headless for 120 seasons supplied souls, memories, chronicles and manifests; Qwen3.8-27B answered long, rule-heavy prompts (with a teacher-played player for multi-turn conversations, including nonsense lines because players are nonsensical); the answers were filtered through the service's validator plus the mod's judgment layers (feasibility, authority, faction, economics, narrative) and became targets under the service's short prompts. ~35k rows across the nine operations (22k dialogue turns, 2.7k quests, 1.1k repairs, 2.4k personality batches, 2.0k consolidations, 2.3k narrations, 0.5k histories, 0.6k summaries, 3.5k gossip lines), LoRA r64 on MiniCPM5-2B, three rounds (dialogue, then joint, then a 6k-token-window pass because quest prompts run 4–5.5k tokens).

Guidance-off evaluation on held-out rows (JSON must be right without a grammar): dialogue validator 99.9 %, quest.generate 95–98 % validator / 94–97 % fully clean, quest.repair 94 %, personality 99 %, consolidate 98.5 %, narrate 97 %, history 100 %, summarize 100 %, gossip 100 %. Against the 27B teacher's own dialogue answers a teacher-as-judge preferred the student 30 %, tied 24 %.

Limitations

• Prompt version 1 only; a new contract version needs a re-distilled model (the pipeline is in the repository).
• Knows the vocabulary it was trained on: the primitives, actions, intents, registers and event types of contract 1.0. New items, landmarks, civs and names are fine (it copies ids from the request); a new primitive or register is not.
• English only. Writing quality is below the 27B teacher's; structure and grounding are the point.

License

Apache 2.0, as the base model. Trained data was synthetic (a simulated world and a 27B teacher).