jiteshkprasad commited on
Commit
06568b4
·
verified ·
1 Parent(s): 919f86c

Brand v2: banner, eyebrow, About footer, 'Where it falls short'; Udgam Labs name, lowercase ids

Browse files
Files changed (2) hide show
  1. README.md +31 -18
  2. banner.png +0 -0
README.md CHANGED
@@ -4,8 +4,9 @@ language:
4
  - en
5
  library_name: mlx
6
  pipeline_tag: text-generation
7
- base_model: UdgamLabs/Udgam-384M-Chat
8
  tags:
 
9
  - gemma4
10
  - tool-calling
11
  - function-calling
@@ -17,15 +18,19 @@ tags:
17
  base_model_relation: quantized
18
  ---
19
 
 
 
 
 
20
  # Udgam-384M-Chat-MLX
21
 
22
- **This repo holds the MLX fp16 version of [Udgam-384M-Chat](https://huggingface.co/UdgamLabs/Udgam-384M-Chat)** for Apple silicon:
23
- `pip install mlx-lm`, then `mlx_lm.generate --model UdgamLabs/Udgam-384M-Chat-MLX --prompt "..."` (or `mlx_lm.server`
24
  for an OpenAI-compatible API). Converted with mlx-lm 0.32. The rest of this page is the main model card.
25
 
26
  **A 384M-parameter English model for on-device tool calling, trained from scratch by Udgam Labs.** It is a
27
  decoder-only transformer compatible with the Gemma 4 architecture (`Gemma4ForCausalLM`), so it runs unmodified in
28
- Ollama, LM Studio, llama.cpp, MLX and 🤗 transformers. Context: 4,096 tokens. The 4-bit GGUF is 238 MB. Built for four
29
  jobs inside an app:
30
 
31
  1. **App actions**: turn a request into a function call. "wake me at 6:45, label it gym" →
@@ -39,7 +44,7 @@ jobs inside an app:
39
  **What it is not:** a knowledgeable general assistant (at 384M parameters it makes factual mistakes), or a
40
  safety-hardened open chatbot. Use it where *your app* decides which tools exist and confirms risky actions.
41
 
42
- **Model family:** [Udgam-384M-Base](https://huggingface.co/UdgamLabs/Udgam-384M-Base) (2,048 context) · [Udgam-384M-Base-4K](https://huggingface.co/UdgamLabs/Udgam-384M-Base-4K) · [Udgam-384M-Chat](https://huggingface.co/UdgamLabs/Udgam-384M-Chat) · [GGUF](https://huggingface.co/UdgamLabs/Udgam-384M-Chat-GGUF) · [MLX](https://huggingface.co/UdgamLabs/Udgam-384M-Chat-MLX) · [MLX 4-bit](https://huggingface.co/UdgamLabs/Udgam-384M-Chat-MLX-4bit) · **[Technical report](https://udgamlabs.com/reports/udgam-384m)**
43
 
44
  | | |
45
  |---|---|
@@ -55,7 +60,7 @@ safety-hardened open chatbot. Use it where *your app* decides which tools exist
55
 
56
  ### Ollama (recommended)
57
 
58
- Download a GGUF from the [GGUF repo](https://huggingface.co/UdgamLabs/Udgam-384M-Chat-GGUF) and create a model with its `Modelfile` (it uses
59
  Ollama's native Gemma 4 renderer and parser, so tool calls and the optional thinking mode work as trained):
60
 
61
  ```
@@ -91,14 +96,14 @@ For this model thinking is optional and usually not better (see the tables).
91
 
92
  ### LM Studio
93
 
94
- 1. Search for `UdgamLabs/Udgam-384M-Chat-GGUF` (or drop the `.gguf` into LM Studio's models folder) and pick `Q8_0`
95
  (default) or `Q4_K_M` (phones / low memory).
96
  2. The GGUF carries the Gemma 4 chat template, so the prompt format is set automatically. Set **Context Length 4096**,
97
  temperature 0.1, top-p 0.9, top-k 40, repeat penalty 1.1.
98
  3. For tool use, call LM Studio's OpenAI-compatible server (`/v1/chat/completions` with `tools`). If your version does
99
  not parse Gemma 4 tool calls into `tool_calls`, read them from the text:
100
  `<|tool_call>call:NAME{key:<|"|>value<|"|>,...}<tool_call|>`.
101
- 4. **Using this MLX version in LM Studio:** before loading, open the model's load settings (⚙️, or turn on
102
  "Manually choose model load parameters") and set **Max Concurrent Predictions to 1**. With the default (parallel)
103
  setting, LM Studio's MLX engine (tested with mlx-llm 1.13.1) fails on text-only Gemma 4 models with
104
  `KeyError: 'text_config'`. The GGUF version needs no change.
@@ -119,7 +124,7 @@ build d7a695e).
119
 
120
  ```bash
121
  pip install mlx-lm
122
- mlx_lm.generate --model UdgamLabs/Udgam-384M-Chat-MLX-4bit --prompt "Give me three tips for sleeping better." --max-tokens 200
123
  ```
124
 
125
  In **LM Studio**, load this model with **Max Concurrent Predictions = 1** (see the LM Studio section above).
@@ -129,8 +134,8 @@ In **LM Studio**, load this model with **Max Concurrent Predictions = 1** (see t
129
  ```python
130
  # tested with transformers 5.18
131
  from transformers import AutoTokenizer, AutoModelForCausalLM
132
- tok = AutoTokenizer.from_pretrained("UdgamLabs/Udgam-384M-Chat")
133
- model = AutoModelForCausalLM.from_pretrained("UdgamLabs/Udgam-384M-Chat")
134
  tools = [{"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm at a local time.",
135
  "parameters": {"type": "object", "properties": {"time": {"type": "string", "description": "HH:MM 24h"},
136
  "label": {"type": "string"}}, "required": ["time"]}}}]
@@ -238,11 +243,6 @@ RAM = macOS peak memory footprint of the CPU run. Phones are slower; the relativ
238
  document → JSON against an unseen schema (schema-valid 56–60% vs 24–38%) and shell behaviour (86% vs 59–67%). It is
239
  also the fastest generator in the group on the Apple-silicon GPU (444 tokens/s at Q4_K_M).
240
 
241
- **Where it is behind:** exact tool calls and instruction following. Both Granite 4.0 350M models are far ahead on BFCL
242
- (simple 88–91% vs 60%, parallel 76–82% vs 46%), on held-out unseen-tool exact match (46% vs 31%) and on IFEval (47–53%
243
- vs 26%). Qwen3-0.6B (about 1.6 times the parameters) is also ahead on BFCL and IFEval. Shell commands are rarely *correct*
244
- when executed for any model here (0–8%).
245
-
246
  **Fairness notes:**
247
  - The held-out task sets use this model's prompt conventions (date/device system headers, normalised argument values
248
  such as ISO dates); other models lose some points to convention rather than ability. BFCL, IFEval and XSTest are
@@ -282,6 +282,15 @@ when executed for any model here (0–8%).
282
  Perplexity on held-out chat and tool-use text: f16 3.193, Q8_0 +0.07%, Q6_K +0.5%, Q4_K_M (imatrix) +4.2%, Q4_0
283
  (imatrix) +8.3%.
284
 
 
 
 
 
 
 
 
 
 
285
  ## Safety
286
 
287
  XSTest (200 clearly unsafe + 250 safe-but-alarming prompts), greedy, thinking off: **the model declines 54% of clearly harmful prompts and over-refuses 15% of safe ones**
@@ -315,7 +324,7 @@ email → JSON with a given schema; suggesting shell commands for a person to re
315
  ## Training
316
 
317
  Pretrained from scratch on roughly 8B tokens of licensed public and synthetic text (base:
318
- [Udgam-384M-Base-4K](https://huggingface.co/UdgamLabs/Udgam-384M-Base-4K)), then post-trained with supervised fine-tuning for chat and tool use.
319
  See the [technical report](https://udgamlabs.com/reports/udgam-384m) for an overview.
320
 
321
  ## Licence
@@ -329,7 +338,7 @@ Apache-2.0 (weights and tokenizer); see `LICENSE`. The chat template is the offi
329
  title = {Udgam-384M: a small on-device tool-calling model trained from scratch},
330
  author = {{Udgam Labs}},
331
  year = {2026},
332
- howpublished = {\url{https://huggingface.co/UdgamLabs/Udgam-384M-Chat}},
333
  note = {Technical report: \url{https://udgamlabs.com/reports/udgam-384m}}
334
  }
335
  ```
@@ -337,3 +346,7 @@ Apache-2.0 (weights and tokenizer); see `LICENSE`. The chat template is the offi
337
  ## Contact
338
 
339
  Udgam Labs. Please use the Community (discussions) tab of the model page for questions, issues and feedback.
 
 
 
 
 
4
  - en
5
  library_name: mlx
6
  pipeline_tag: text-generation
7
+ base_model: udgamlabs/Udgam-384M-Chat
8
  tags:
9
+ - udgamlabs
10
  - gemma4
11
  - tool-calling
12
  - function-calling
 
18
  base_model_relation: quantized
19
  ---
20
 
21
+ ![Udgam-384M-Chat-MLX · Udgam Labs](./banner.png)
22
+
23
+ **UDGAM LABS** · Tool-calling model, MLX · 384M parameters
24
+
25
  # Udgam-384M-Chat-MLX
26
 
27
+ **This repo holds the MLX fp16 version of [Udgam-384M-Chat](https://huggingface.co/udgamlabs/Udgam-384M-Chat)** for Apple silicon:
28
+ `pip install mlx-lm`, then `mlx_lm.generate --model udgamlabs/Udgam-384M-Chat-MLX --prompt "..."` (or `mlx_lm.server`
29
  for an OpenAI-compatible API). Converted with mlx-lm 0.32. The rest of this page is the main model card.
30
 
31
  **A 384M-parameter English model for on-device tool calling, trained from scratch by Udgam Labs.** It is a
32
  decoder-only transformer compatible with the Gemma 4 architecture (`Gemma4ForCausalLM`), so it runs unmodified in
33
+ Ollama, LM Studio, llama.cpp, MLX and transformers. Context: 4,096 tokens. The 4-bit GGUF is 238 MB. Built for four
34
  jobs inside an app:
35
 
36
  1. **App actions**: turn a request into a function call. "wake me at 6:45, label it gym" →
 
44
  **What it is not:** a knowledgeable general assistant (at 384M parameters it makes factual mistakes), or a
45
  safety-hardened open chatbot. Use it where *your app* decides which tools exist and confirms risky actions.
46
 
47
+ **Model family:** [Udgam-384M-Base](https://huggingface.co/udgamlabs/Udgam-384M-Base) (2,048 context) · [Udgam-384M-Base-4K](https://huggingface.co/udgamlabs/Udgam-384M-Base-4K) · [Udgam-384M-Chat](https://huggingface.co/udgamlabs/Udgam-384M-Chat) · [GGUF](https://huggingface.co/udgamlabs/Udgam-384M-Chat-GGUF) · [MLX](https://huggingface.co/udgamlabs/Udgam-384M-Chat-MLX) · [MLX 4-bit](https://huggingface.co/udgamlabs/Udgam-384M-Chat-MLX-4bit) · **[Technical report](https://udgamlabs.com/reports/udgam-384m)**
48
 
49
  | | |
50
  |---|---|
 
60
 
61
  ### Ollama (recommended)
62
 
63
+ Download a GGUF from the [GGUF repo](https://huggingface.co/udgamlabs/Udgam-384M-Chat-GGUF) and create a model with its `Modelfile` (it uses
64
  Ollama's native Gemma 4 renderer and parser, so tool calls and the optional thinking mode work as trained):
65
 
66
  ```
 
96
 
97
  ### LM Studio
98
 
99
+ 1. Search for `udgamlabs/Udgam-384M-Chat-GGUF` (or drop the `.gguf` into LM Studio's models folder) and pick `Q8_0`
100
  (default) or `Q4_K_M` (phones / low memory).
101
  2. The GGUF carries the Gemma 4 chat template, so the prompt format is set automatically. Set **Context Length 4096**,
102
  temperature 0.1, top-p 0.9, top-k 40, repeat penalty 1.1.
103
  3. For tool use, call LM Studio's OpenAI-compatible server (`/v1/chat/completions` with `tools`). If your version does
104
  not parse Gemma 4 tool calls into `tool_calls`, read them from the text:
105
  `<|tool_call>call:NAME{key:<|"|>value<|"|>,...}<tool_call|>`.
106
+ 4. **Using this MLX version in LM Studio:** before loading, open the model's load settings (the gear icon, or turn on
107
  "Manually choose model load parameters") and set **Max Concurrent Predictions to 1**. With the default (parallel)
108
  setting, LM Studio's MLX engine (tested with mlx-llm 1.13.1) fails on text-only Gemma 4 models with
109
  `KeyError: 'text_config'`. The GGUF version needs no change.
 
124
 
125
  ```bash
126
  pip install mlx-lm
127
+ mlx_lm.generate --model udgamlabs/Udgam-384M-Chat-MLX-4bit --prompt "Give me three tips for sleeping better." --max-tokens 200
128
  ```
129
 
130
  In **LM Studio**, load this model with **Max Concurrent Predictions = 1** (see the LM Studio section above).
 
134
  ```python
135
  # tested with transformers 5.18
136
  from transformers import AutoTokenizer, AutoModelForCausalLM
137
+ tok = AutoTokenizer.from_pretrained("udgamlabs/Udgam-384M-Chat")
138
+ model = AutoModelForCausalLM.from_pretrained("udgamlabs/Udgam-384M-Chat")
139
  tools = [{"type": "function", "function": {"name": "set_alarm", "description": "Set an alarm at a local time.",
140
  "parameters": {"type": "object", "properties": {"time": {"type": "string", "description": "HH:MM 24h"},
141
  "label": {"type": "string"}}, "required": ["time"]}}}]
 
243
  document → JSON against an unseen schema (schema-valid 56–60% vs 24–38%) and shell behaviour (86% vs 59–67%). It is
244
  also the fastest generator in the group on the Apple-silicon GPU (444 tokens/s at Q4_K_M).
245
 
 
 
 
 
 
246
  **Fairness notes:**
247
  - The held-out task sets use this model's prompt conventions (date/device system headers, normalised argument values
248
  such as ISO dates); other models lose some points to convention rather than ability. BFCL, IFEval and XSTest are
 
282
  Perplexity on held-out chat and tool-use text: f16 3.193, Q8_0 +0.07%, Q6_K +0.5%, Q4_K_M (imatrix) +4.2%, Q4_0
283
  (imatrix) +8.3%.
284
 
285
+ ## Where it falls short
286
+
287
+ It is behind on exact tool calls and instruction following. Both Granite 4.0 350M models are far ahead on BFCL
288
+ (simple 88–91% vs 60%, parallel 76–82% vs 46%), on held-out unseen-tool exact match (46% vs 31%) and on IFEval (47–53%
289
+ vs 26%). Qwen3-0.6B (about 1.6 times the parameters) is also ahead on BFCL and IFEval. Shell commands are rarely *correct*
290
+ when executed for any model here (0–8%).
291
+
292
+ On the neutral BFCL irrelevance test, Qwen3-0.6B and SmolLM2 also beat it (see the fairness notes above). For safety results and the tasks it is not meant for, see *Safety* and *Intended use and limitations*.
293
+
294
  ## Safety
295
 
296
  XSTest (200 clearly unsafe + 250 safe-but-alarming prompts), greedy, thinking off: **the model declines 54% of clearly harmful prompts and over-refuses 15% of safe ones**
 
324
  ## Training
325
 
326
  Pretrained from scratch on roughly 8B tokens of licensed public and synthetic text (base:
327
+ [Udgam-384M-Base-4K](https://huggingface.co/udgamlabs/Udgam-384M-Base-4K)), then post-trained with supervised fine-tuning for chat and tool use.
328
  See the [technical report](https://udgamlabs.com/reports/udgam-384m) for an overview.
329
 
330
  ## Licence
 
338
  title = {Udgam-384M: a small on-device tool-calling model trained from scratch},
339
  author = {{Udgam Labs}},
340
  year = {2026},
341
+ howpublished = {\url{https://huggingface.co/udgamlabs/Udgam-384M-Chat}},
342
  note = {Technical report: \url{https://udgamlabs.com/reports/udgam-384m}}
343
  }
344
  ```
 
346
  ## Contact
347
 
348
  Udgam Labs. Please use the Community (discussions) tab of the model page for questions, issues and feedback.
349
+
350
+ ---
351
+
352
+ **About Udgam Labs.** Udgam Labs is an independent AI research lab. We work from first principles: before we build, we understand why things work, whether we are training a model, fine-tuning one or applying AI to a new problem. Udgam (उद्गम) is Sanskrit for origin. [udgamlabs.com](https://udgamlabs.com)
banner.png ADDED