Update README.md
Browse files
README.md
CHANGED
|
@@ -29,7 +29,7 @@ size_categories:
|
|
| 29 |
|
| 30 |
The Muse2 collection is a family of compact language models built **entirely from scratch** — architecture, tokenizer, and inference stack — for edge and on-device use. Muse2 pairs an LFM2-style hybrid layout (short causal convolutions + grouped-query attention) with a parallel merged SwiGLU MLP per block, chosen for inference efficiency on constrained hardware. The instruction-tuned version is optimized for assistant-like chat, formatting, and structured extraction from developer logs.
|
| 31 |
|
| 32 |
-
**Model Developer:** Muse Research
|
| 33 |
|
| 34 |
**Model Architecture:** Muse2 is an auto-regressive language model that uses a hybrid convolution/attention architecture implemented from scratch in PyTorch (`Muse2ForCausalLM`, no `transformers` code). The tuned version is aligned with supervised fine-tuning (SFT) with completion-only loss masking. No RLHF or DPO has been applied.
|
| 35 |
|
|
|
|
| 29 |
|
| 30 |
The Muse2 collection is a family of compact language models built **entirely from scratch** — architecture, tokenizer, and inference stack — for edge and on-device use. Muse2 pairs an LFM2-style hybrid layout (short causal convolutions + grouped-query attention) with a parallel merged SwiGLU MLP per block, chosen for inference efficiency on constrained hardware. The instruction-tuned version is optimized for assistant-like chat, formatting, and structured extraction from developer logs.
|
| 31 |
|
| 32 |
+
**Model Developer:** Muse Research
|
| 33 |
|
| 34 |
**Model Architecture:** Muse2 is an auto-regressive language model that uses a hybrid convolution/attention architecture implemented from scratch in PyTorch (`Muse2ForCausalLM`, no `transformers` code). The tuned version is aligned with supervised fine-tuning (SFT) with completion-only loss masking. No RLHF or DPO has been applied.
|
| 35 |
|