Text Generation
MLX
Safetensors
English
gemma4_text
udgamlabs
gemma4
tool-calling
function-calling
on-device
json-extraction
shell
from-scratch
conversational
Instructions to use udgamlabs/Udgam-384M-Chat-MLX with libraries, inference providers, notebooks, and local apps. Follow these links to get started.
- Libraries
- MLX
How to use udgamlabs/Udgam-384M-Chat-MLX with MLX:
# Make sure mlx-lm is installed # pip install --upgrade mlx-lm # Generate text with mlx-lm from mlx_lm import load, generate model, tokenizer = load("udgamlabs/Udgam-384M-Chat-MLX") prompt = "Write a story about Einstein" messages = [{"role": "user", "content": prompt}] prompt = tokenizer.apply_chat_template( messages, add_generation_prompt=True ) text = generate(model, tokenizer, prompt=prompt, verbose=True) - Notebooks
- Google Colab
- Kaggle
- Local Apps Settings
- LM Studio
- Pi
How to use udgamlabs/Udgam-384M-Chat-MLX with Pi:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "udgamlabs/Udgam-384M-Chat-MLX"
Configure the model in Pi
# Install Pi: npm install -g @earendil-works/pi-coding-agent # Add to ~/.pi/agent/models.json: { "providers": { "mlx-lm": { "baseUrl": "http://localhost:8080/v1", "api": "openai-completions", "apiKey": "none", "models": [ { "id": "udgamlabs/Udgam-384M-Chat-MLX" } ] } } }Run Pi
# Start Pi in your project directory: pi
- MLX LM
How to use udgamlabs/Udgam-384M-Chat-MLX with MLX LM:
Generate or start a chat session
# Install MLX LM uv tool install mlx-lm # Interactive chat REPL mlx_lm.chat --model "udgamlabs/Udgam-384M-Chat-MLX"
Run an OpenAI-compatible server
# Install MLX LM uv tool install mlx-lm # Start the server mlx_lm.server --model "udgamlabs/Udgam-384M-Chat-MLX" # Calling the OpenAI-compatible server with curl curl -X POST "http://localhost:8000/v1/chat/completions" \ -H "Content-Type: application/json" \ --data '{ "model": "udgamlabs/Udgam-384M-Chat-MLX", "messages": [ {"role": "user", "content": "Hello"} ] }' - Hermes Agent
How to use udgamlabs/Udgam-384M-Chat-MLX with Hermes Agent:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "udgamlabs/Udgam-384M-Chat-MLX"
Configure Hermes
# Install Hermes: curl -fsSL https://hermes-agent.nousresearch.com/install.sh | bash hermes setup # Point Hermes at the local server: hermes config set model.provider custom hermes config set model.base_url http://127.0.0.1:8080/v1 hermes config set model.default udgamlabs/Udgam-384M-Chat-MLX
Run Hermes
hermes
- Atomic Chat
- OpenClaw
How to use udgamlabs/Udgam-384M-Chat-MLX with OpenClaw:
Start the MLX server
# Install MLX LM: uv tool install mlx-lm # Start a local OpenAI-compatible server: mlx_lm.server --model "udgamlabs/Udgam-384M-Chat-MLX"
Configure OpenClaw
# Install OpenClaw: npm install -g openclaw@latest # Register the local server and set it as the default model: openclaw onboard --non-interactive --mode local \ --auth-choice custom-api-key \ --custom-base-url http://127.0.0.1:8080/v1 \ --custom-model-id "udgamlabs/Udgam-384M-Chat-MLX" \ --custom-provider-id mlx-lm \ --custom-compatibility openai \ --custom-text-input \ --accept-risk \ --skip-health
Run OpenClaw
openclaw agent --local --agent main --message "Hello from Hugging Face"
Brand v2: banner, eyebrow, About footer, 'Where it falls short'; Udgam Labs name, lowercase ids
Browse files- README.md +31 -18
- banner.png +0 -0
README.md
CHANGED
|
@@ -4,8 +4,9 @@ language:
|
|
| 4 |
- en
|
| 5 |
library_name: mlx
|
| 6 |
pipeline_tag: text-generation
|
| 7 |
-
base_model:
|
| 8 |
tags:
|
|
|
|
| 9 |
- gemma4
|
| 10 |
- tool-calling
|
| 11 |
- function-calling
|
|
@@ -17,15 +18,19 @@ tags:
|
|
| 17 |
base_model_relation: quantized
|
| 18 |
---
|
| 19 |
|
|
|
|
|
|
|
|
|
|
|
|
|
| 20 |
# Udgam-384M-Chat-MLX
|
| 21 |
|
| 22 |
-
**This repo holds the MLX fp16 version of [Udgam-384M-Chat](https://huggingface.co/
|
| 23 |
-
`pip install mlx-lm`, then `mlx_lm.generate --model
|
| 24 |
for an OpenAI-compatible API). Converted with mlx-lm 0.32. The rest of this page is the main model card.
|
| 25 |
|
| 26 |
**A 384M-parameter English model for on-device tool calling, trained from scratch by Udgam Labs.** It is a
|
| 27 |
decoder-only transformer compatible with the Gemma 4 architecture (`Gemma4ForCausalLM`), so it runs unmodified in
|
| 28 |
-
Ollama, LM Studio, llama.cpp, MLX and
|
| 29 |
jobs inside an app:
|
| 30 |
|
| 31 |
1. **App actions**: turn a request into a function call. "wake me at 6:45, label it gym" →
|
|
@@ -39,7 +44,7 @@ jobs inside an app:
|
|
| 39 |
**What it is not:** a knowledgeable general assistant (at 384M parameters it makes factual mistakes), or a
|
| 40 |
safety-hardened open chatbot. Use it where *your app* decides which tools exist and confirms risky actions.
|
| 41 |
|
| 42 |
-
**Model family:** [Udgam-384M-Base](https://huggingface.co/
|
| 43 |
|
| 44 |
| | |
|
| 45 |
|---|---|
|
|
@@ -55,7 +60,7 @@ safety-hardened open chatbot. Use it where *your app* decides which tools exist
|
|
| 55 |
|
| 56 |
### Ollama (recommended)
|
| 57 |
|
| 58 |
-
Download a GGUF from the [GGUF repo](https://huggingface.co/
|
| 59 |
Ollama's native Gemma 4 renderer and parser, so tool calls and the optional thinking mode work as trained):
|
| 60 |
|
| 61 |
```
|
|
@@ -91,14 +96,14 @@ For this model thinking is optional and usually not better (see the tables).
|
|
| 91 |
|
| 92 |
### LM Studio
|
| 93 |
|
| 94 |
-
1. Search for `
|
| 95 |
(default) or `Q4_K_M` (phones / low memory).
|
| 96 |
2. The GGUF carries the Gemma 4 chat template, so the prompt format is set automatically. Set **Context Length 4096**,
|
| 97 |
temperature 0.1, top-p 0.9, top-k 40, repeat penalty 1.1.
|
| 98 |
3. For tool use, call LM Studio's OpenAI-compatible server (`/v1/chat/completions` with `tools`). If your version does
|
| 99 |
not parse Gemma 4 tool calls into `tool_calls`, read them from the text:
|
| 100 |
`<|tool_call>call:NAME{key:<|"|>value<|"|>,...}<tool_call|>`.
|
| 101 |
-
4. **Using this MLX version in LM Studio:** before loading, open the model's load settings (
|
| 102 |
"Manually choose model load parameters") and set **Max Concurrent Predictions to 1**. With the default (parallel)
|
| 103 |
setting, LM Studio's MLX engine (tested with mlx-llm 1.13.1) fails on text-only Gemma 4 models with
|
| 104 |
`KeyError: 'text_config'`. The GGUF version needs no change.
|
|
@@ -119,7 +124,7 @@ build d7a695e).
|
|
| 119 |
|
| 120 |
```bash
|
| 121 |
pip install mlx-lm
|
| 122 |
-
mlx_lm.generate --model
|
| 123 |
```
|
| 124 |
|
| 125 |
In **LM Studio**, load this model with **Max Concurrent Predictions = 1** (see the LM Studio section above).
|
|
@@ -129,8 +134,8 @@ In **LM Studio**, load this model with **Max Concurrent Predictions = 1** (see t
|
|
| 129 |
```python
|
| 130 |
# tested with transformers 5.18
|
| 131 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 132 |
-
tok = AutoTokenizer.from_pretrained("
|
| 133 |
-
model = AutoModelForCausalLM.from_pretrained("
|
| 134 |
tools = [{"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm at a local time.",
|
| 135 |
"parameters": {"type": "object", "properties": {"time": {"type": "string", "description": "HH:MM 24h"},
|
| 136 |
"label": {"type": "string"}}, "required": ["time"]}}}]
|
|
@@ -238,11 +243,6 @@ RAM = macOS peak memory footprint of the CPU run. Phones are slower; the relativ
|
|
| 238 |
document → JSON against an unseen schema (schema-valid 56–60% vs 24–38%) and shell behaviour (86% vs 59–67%). It is
|
| 239 |
also the fastest generator in the group on the Apple-silicon GPU (444 tokens/s at Q4_K_M).
|
| 240 |
|
| 241 |
-
**Where it is behind:** exact tool calls and instruction following. Both Granite 4.0 350M models are far ahead on BFCL
|
| 242 |
-
(simple 88–91% vs 60%, parallel 76–82% vs 46%), on held-out unseen-tool exact match (46% vs 31%) and on IFEval (47–53%
|
| 243 |
-
vs 26%). Qwen3-0.6B (about 1.6 times the parameters) is also ahead on BFCL and IFEval. Shell commands are rarely *correct*
|
| 244 |
-
when executed for any model here (0–8%).
|
| 245 |
-
|
| 246 |
**Fairness notes:**
|
| 247 |
- The held-out task sets use this model's prompt conventions (date/device system headers, normalised argument values
|
| 248 |
such as ISO dates); other models lose some points to convention rather than ability. BFCL, IFEval and XSTest are
|
|
@@ -282,6 +282,15 @@ when executed for any model here (0–8%).
|
|
| 282 |
Perplexity on held-out chat and tool-use text: f16 3.193, Q8_0 +0.07%, Q6_K +0.5%, Q4_K_M (imatrix) +4.2%, Q4_0
|
| 283 |
(imatrix) +8.3%.
|
| 284 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 285 |
## Safety
|
| 286 |
|
| 287 |
XSTest (200 clearly unsafe + 250 safe-but-alarming prompts), greedy, thinking off: **the model declines 54% of clearly harmful prompts and over-refuses 15% of safe ones**
|
|
@@ -315,7 +324,7 @@ email → JSON with a given schema; suggesting shell commands for a person to re
|
|
| 315 |
## Training
|
| 316 |
|
| 317 |
Pretrained from scratch on roughly 8B tokens of licensed public and synthetic text (base:
|
| 318 |
-
[Udgam-384M-Base-4K](https://huggingface.co/
|
| 319 |
See the [technical report](https://udgamlabs.com/reports/udgam-384m) for an overview.
|
| 320 |
|
| 321 |
## Licence
|
|
@@ -329,7 +338,7 @@ Apache-2.0 (weights and tokenizer); see `LICENSE`. The chat template is the offi
|
|
| 329 |
title = {Udgam-384M: a small on-device tool-calling model trained from scratch},
|
| 330 |
author = {{Udgam Labs}},
|
| 331 |
year = {2026},
|
| 332 |
-
howpublished = {\url{https://huggingface.co/
|
| 333 |
note = {Technical report: \url{https://udgamlabs.com/reports/udgam-384m}}
|
| 334 |
}
|
| 335 |
```
|
|
@@ -337,3 +346,7 @@ Apache-2.0 (weights and tokenizer); see `LICENSE`. The chat template is the offi
|
|
| 337 |
## Contact
|
| 338 |
|
| 339 |
Udgam Labs. Please use the Community (discussions) tab of the model page for questions, issues and feedback.
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 4 |
- en
|
| 5 |
library_name: mlx
|
| 6 |
pipeline_tag: text-generation
|
| 7 |
+
base_model: udgamlabs/Udgam-384M-Chat
|
| 8 |
tags:
|
| 9 |
+
- udgamlabs
|
| 10 |
- gemma4
|
| 11 |
- tool-calling
|
| 12 |
- function-calling
|
|
|
|
| 18 |
base_model_relation: quantized
|
| 19 |
---
|
| 20 |
|
| 21 |
+

|
| 22 |
+
|
| 23 |
+
**UDGAM LABS** · Tool-calling model, MLX · 384M parameters
|
| 24 |
+
|
| 25 |
# Udgam-384M-Chat-MLX
|
| 26 |
|
| 27 |
+
**This repo holds the MLX fp16 version of [Udgam-384M-Chat](https://huggingface.co/udgamlabs/Udgam-384M-Chat)** for Apple silicon:
|
| 28 |
+
`pip install mlx-lm`, then `mlx_lm.generate --model udgamlabs/Udgam-384M-Chat-MLX --prompt "..."` (or `mlx_lm.server`
|
| 29 |
for an OpenAI-compatible API). Converted with mlx-lm 0.32. The rest of this page is the main model card.
|
| 30 |
|
| 31 |
**A 384M-parameter English model for on-device tool calling, trained from scratch by Udgam Labs.** It is a
|
| 32 |
decoder-only transformer compatible with the Gemma 4 architecture (`Gemma4ForCausalLM`), so it runs unmodified in
|
| 33 |
+
Ollama, LM Studio, llama.cpp, MLX and transformers. Context: 4,096 tokens. The 4-bit GGUF is 238 MB. Built for four
|
| 34 |
jobs inside an app:
|
| 35 |
|
| 36 |
1. **App actions**: turn a request into a function call. "wake me at 6:45, label it gym" →
|
|
|
|
| 44 |
**What it is not:** a knowledgeable general assistant (at 384M parameters it makes factual mistakes), or a
|
| 45 |
safety-hardened open chatbot. Use it where *your app* decides which tools exist and confirms risky actions.
|
| 46 |
|
| 47 |
+
**Model family:** [Udgam-384M-Base](https://huggingface.co/udgamlabs/Udgam-384M-Base) (2,048 context) · [Udgam-384M-Base-4K](https://huggingface.co/udgamlabs/Udgam-384M-Base-4K) · [Udgam-384M-Chat](https://huggingface.co/udgamlabs/Udgam-384M-Chat) · [GGUF](https://huggingface.co/udgamlabs/Udgam-384M-Chat-GGUF) · [MLX](https://huggingface.co/udgamlabs/Udgam-384M-Chat-MLX) · [MLX 4-bit](https://huggingface.co/udgamlabs/Udgam-384M-Chat-MLX-4bit) · **[Technical report](https://udgamlabs.com/reports/udgam-384m)**
|
| 48 |
|
| 49 |
| | |
|
| 50 |
|---|---|
|
|
|
|
| 60 |
|
| 61 |
### Ollama (recommended)
|
| 62 |
|
| 63 |
+
Download a GGUF from the [GGUF repo](https://huggingface.co/udgamlabs/Udgam-384M-Chat-GGUF) and create a model with its `Modelfile` (it uses
|
| 64 |
Ollama's native Gemma 4 renderer and parser, so tool calls and the optional thinking mode work as trained):
|
| 65 |
|
| 66 |
```
|
|
|
|
| 96 |
|
| 97 |
### LM Studio
|
| 98 |
|
| 99 |
+
1. Search for `udgamlabs/Udgam-384M-Chat-GGUF` (or drop the `.gguf` into LM Studio's models folder) and pick `Q8_0`
|
| 100 |
(default) or `Q4_K_M` (phones / low memory).
|
| 101 |
2. The GGUF carries the Gemma 4 chat template, so the prompt format is set automatically. Set **Context Length 4096**,
|
| 102 |
temperature 0.1, top-p 0.9, top-k 40, repeat penalty 1.1.
|
| 103 |
3. For tool use, call LM Studio's OpenAI-compatible server (`/v1/chat/completions` with `tools`). If your version does
|
| 104 |
not parse Gemma 4 tool calls into `tool_calls`, read them from the text:
|
| 105 |
`<|tool_call>call:NAME{key:<|"|>value<|"|>,...}<tool_call|>`.
|
| 106 |
+
4. **Using this MLX version in LM Studio:** before loading, open the model's load settings (the gear icon, or turn on
|
| 107 |
"Manually choose model load parameters") and set **Max Concurrent Predictions to 1**. With the default (parallel)
|
| 108 |
setting, LM Studio's MLX engine (tested with mlx-llm 1.13.1) fails on text-only Gemma 4 models with
|
| 109 |
`KeyError: 'text_config'`. The GGUF version needs no change.
|
|
|
|
| 124 |
|
| 125 |
```bash
|
| 126 |
pip install mlx-lm
|
| 127 |
+
mlx_lm.generate --model udgamlabs/Udgam-384M-Chat-MLX-4bit --prompt "Give me three tips for sleeping better." --max-tokens 200
|
| 128 |
```
|
| 129 |
|
| 130 |
In **LM Studio**, load this model with **Max Concurrent Predictions = 1** (see the LM Studio section above).
|
|
|
|
| 134 |
```python
|
| 135 |
# tested with transformers 5.18
|
| 136 |
from transformers import AutoTokenizer, AutoModelForCausalLM
|
| 137 |
+
tok = AutoTokenizer.from_pretrained("udgamlabs/Udgam-384M-Chat")
|
| 138 |
+
model = AutoModelForCausalLM.from_pretrained("udgamlabs/Udgam-384M-Chat")
|
| 139 |
tools = [{"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm at a local time.",
|
| 140 |
"parameters": {"type": "object", "properties": {"time": {"type": "string", "description": "HH:MM 24h"},
|
| 141 |
"label": {"type": "string"}}, "required": ["time"]}}}]
|
|
|
|
| 243 |
document → JSON against an unseen schema (schema-valid 56–60% vs 24–38%) and shell behaviour (86% vs 59–67%). It is
|
| 244 |
also the fastest generator in the group on the Apple-silicon GPU (444 tokens/s at Q4_K_M).
|
| 245 |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
| 246 |
**Fairness notes:**
|
| 247 |
- The held-out task sets use this model's prompt conventions (date/device system headers, normalised argument values
|
| 248 |
such as ISO dates); other models lose some points to convention rather than ability. BFCL, IFEval and XSTest are
|
|
|
|
| 282 |
Perplexity on held-out chat and tool-use text: f16 3.193, Q8_0 +0.07%, Q6_K +0.5%, Q4_K_M (imatrix) +4.2%, Q4_0
|
| 283 |
(imatrix) +8.3%.
|
| 284 |
|
| 285 |
+
## Where it falls short
|
| 286 |
+
|
| 287 |
+
It is behind on exact tool calls and instruction following. Both Granite 4.0 350M models are far ahead on BFCL
|
| 288 |
+
(simple 88–91% vs 60%, parallel 76–82% vs 46%), on held-out unseen-tool exact match (46% vs 31%) and on IFEval (47–53%
|
| 289 |
+
vs 26%). Qwen3-0.6B (about 1.6 times the parameters) is also ahead on BFCL and IFEval. Shell commands are rarely *correct*
|
| 290 |
+
when executed for any model here (0–8%).
|
| 291 |
+
|
| 292 |
+
On the neutral BFCL irrelevance test, Qwen3-0.6B and SmolLM2 also beat it (see the fairness notes above). For safety results and the tasks it is not meant for, see *Safety* and *Intended use and limitations*.
|
| 293 |
+
|
| 294 |
## Safety
|
| 295 |
|
| 296 |
XSTest (200 clearly unsafe + 250 safe-but-alarming prompts), greedy, thinking off: **the model declines 54% of clearly harmful prompts and over-refuses 15% of safe ones**
|
|
|
|
| 324 |
## Training
|
| 325 |
|
| 326 |
Pretrained from scratch on roughly 8B tokens of licensed public and synthetic text (base:
|
| 327 |
+
[Udgam-384M-Base-4K](https://huggingface.co/udgamlabs/Udgam-384M-Base-4K)), then post-trained with supervised fine-tuning for chat and tool use.
|
| 328 |
See the [technical report](https://udgamlabs.com/reports/udgam-384m) for an overview.
|
| 329 |
|
| 330 |
## Licence
|
|
|
|
| 338 |
title = {Udgam-384M: a small on-device tool-calling model trained from scratch},
|
| 339 |
author = {{Udgam Labs}},
|
| 340 |
year = {2026},
|
| 341 |
+
howpublished = {\url{https://huggingface.co/udgamlabs/Udgam-384M-Chat}},
|
| 342 |
note = {Technical report: \url{https://udgamlabs.com/reports/udgam-384m}}
|
| 343 |
}
|
| 344 |
```
|
|
|
|
| 346 |
## Contact
|
| 347 |
|
| 348 |
Udgam Labs. Please use the Community (discussions) tab of the model page for questions, issues and feedback.
|
| 349 |
+
|
| 350 |
+
---
|
| 351 |
+
|
| 352 |
+
**About Udgam Labs.** Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. [udgamlabs.com](https://udgamlabs.com)
|
banner.png
ADDED
|