---
language:
- en
library_name: transformers
pipeline_tag: audio-text-to-text
license: other
license_name: sori-research-license
license_link: LICENSE
base_model:
- LiquidAI/LFM2.5-350M-Base
- LiquidAI/LFM2.5-Encoder-350M
- nvidia/audio-flamingo-next-hf
base_model_relation: merge
tags:
- audio
- audio-language-model
- audio-understanding
- python-interpreter
- multiple-choice
- captioning
- asr
- lfm2.5
- distillation
extra_gated_prompt: >-
Sori-1B is released for non-commercial research use, to be shared and built on under the same terms with
attribution. It is built from Liquid AI's LFM2.5-350M-Base and LFM2.5-Encoder-350M (LFM Open License v1.0, kept), and
its audio encoder was initialised by distillation from the outputs of NVIDIA's Audio Flamingo Next encoder (NVIDIA
OneWay Noncommercial License), which is why commercial use is not licensed. The complete terms are in LICENSE.
Access is granted on these conditions: you use the model for non-commercial research only; anything you derive from
it or its outputs is shared under the same terms with a notice of what was changed; and you cite this work in any
publication, system or model whose results depend on it.
extra_gated_fields:
Name: text
Affiliation: text
Intended use: text
I will use this model for non-commercial research only: checkbox
I will share derived models and data under the same terms, with a notice of what was changed: checkbox
I will cite this work in publications, systems and models that depend on it: checkbox
I accept the LFM Open License and the NVIDIA OneWay Noncommercial License terms carried by this model: checkbox
extra_gated_button_content: Request access
---
Sori Agent: Audio Understanding via Code Execution over Structured Perception
An audio understanding agent for speech, music and environmental sound. Sori-1B is its core model.
Seonuk Kim · Human Interface Lab · Seoul National University · 🇰🇷
Released 28 August 2026
Sori Agent — *sori* (소리) is Korean for *sound* — is an audio understanding agent, and Sori-1B is its core model. You give
it a clip and ask a question in plain language; it listens, writes down what it heard, and answers. Speech, music and environmental sound are one
subject to it, and one call is one answer.
## Setup
```bash
pip install "transformers>=5.13" "torch>=2.6" soundfile scipy numpy
huggingface-cli login # the repository is gated: accept the terms on this page first
```
Everything the model needs travels with the checkpoint (`trust_remote_code=True` loads the processor and the model
code). Audio is read with `soundfile` (wav, flac, ogg, and mp3 with libsndfile 1.1+) and resampled to 16 kHz; any
sample rate and any length are accepted, and clips are never cut. The model is fp32 end to
end, so the answer does not depend on the device or the batch size, and CPU works.
## Usage
```python
from transformers import AutoModel, AutoProcessor
proc = AutoProcessor.from_pretrained("snkii/Sori-1B", trust_remote_code=True)
model = AutoModel.from_pretrained("snkii/Sori-1B", trust_remote_code=True).cuda().eval()
sori = model.agent(proc)
sori.ask("clip.wav", "What is happening in this recording?")
sori.ask("clip.wav", "How many people are speaking?")
sori.ask("clip.wav", "What does the second speaker say?")
sori.ask("clip.wav", "When does the door slam?")
sori.ask("clip.wav", "Is a siren audible?", ["yes", "no"])
sori.ask("clip.wav", "What is heard right after the door?", ["drilling", "chainsaw", "jackhammer", "vacuum cleaner"])
sori.ask(["a.wav", "b.wav"], "Which clip is louder?", ["the first", "the second"])
sori.ask_many([{"audio": "a.wav", "question": "Describe the scene."},
{"audio": "b.wav", "question": "How many speakers are there?"}])
```
With `choices`, the answer is one of them, verbatim. Without them, it is a sentence, a number or a list — whatever the
question asks for. `explain=True` returns the answer together with the steps the agent took to reach it.
**Two ways to answer, one preference.** A question that can be computed is answered by code that a real interpreter
runs, every step shown and checkable; what cannot be computed — a description, what a speaker meant, a judgement — is
answered in language. The computed path needs the `sorilm` package (`pip install git+https://github.com/snkii/sori`);
with the checkpoint alone, the agent hears and answers in language.
**For people.** A terminal conversation about a file, where plain requests become interpreter lines and are shown:
```bash
python -m sorilm.core.chat clip.wav --ckpt snkii/Sori-1B
```
**For agents.** The tool surface is one tool. `sori.tools()` emits its JSON schema — `ask(audio, question, choices?,
explain?)` — and a Model Context Protocol server serves it, so a planner can ask about a clip the way it calls any other
tool, without prompt engineering and without parsing prose:
```bash
python -m sorilm.core.mcp_server --ckpt snkii/Sori-1B # stdio transport
```
## The model
Sori-1B is one architecture from the waveform to the words. Its audio encoder is built from Liquid AI's
[LFM2.5-Encoder-350M](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M), initialised by distillation from the outputs
of NVIDIA's [Audio Flamingo Next](https://huggingface.co/nvidia/audio-flamingo-next-hf) encoder and then trained; its
language model is [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base), fully trained here. Together
they are 933 million parameters. This is a research preview under active development, for academic use; how it is built and
evaluated is described in the technical report that accompanies the project.
## License
Sori-1B is released for **non-commercial research use**: use it, share it and build on it under the same terms, with
attribution. Models, data and systems derived from it or from its outputs carry this license, a notice of what was
changed, and a citation of this work; so do publications whose results depend on it. Commercial use is not licensed:
the audio encoder was initialised by distillation from the outputs of NVIDIA's Audio Flamingo Next encoder (NVIDIA
OneWay Noncommercial License), and part of the training data is licensed for non-commercial use. The Liquid AI models
it is trained from remain under the LFM Open License v1.0, reproduced in full in [LICENSE](LICENSE), where the complete
terms are.
## Citation
```bibtex
@software{kim_sori_agent_2026,
author = {Kim, Seonuk},
title = {{Sori Agent: Audio Understanding via Code Execution over Structured Perception}},
year = {2026},
month = aug,
institution = {Human Interface Laboratory, Seoul National University},
url = {https://huggingface.co/snkii/Sori-1B}
}
```