--- language: - en library_name: transformers pipeline_tag: audio-text-to-text license: other license_name: sori-research-license license_link: LICENSE base_model: - LiquidAI/LFM2.5-350M-Base - LiquidAI/LFM2.5-Encoder-350M - nvidia/audio-flamingo-next-hf base_model_relation: merge tags: - audio - audio-language-model - audio-understanding - python-interpreter - multiple-choice - captioning - asr - lfm2.5 - distillation extra_gated_prompt: >- Sori-1B is released for non-commercial research use, to be shared and built on under the same terms with attribution. It is built from Liquid AI's LFM2.5-350M-Base and LFM2.5-Encoder-350M (LFM Open License v1.0, kept), and its audio encoder was initialised by distillation from the outputs of NVIDIA's Audio Flamingo Next encoder (NVIDIA OneWay Noncommercial License), which is why commercial use is not licensed. The complete terms are in LICENSE. Access is granted on these conditions: you use the model for non-commercial research only; anything you derive from it or its outputs is shared under the same terms with a notice of what was changed; and you cite this work in any publication, system or model whose results depend on it. extra_gated_fields: Name: text Affiliation: text Intended use: text I will use this model for non-commercial research only: checkbox I will share derived models and data under the same terms, with a notice of what was changed: checkbox I will cite this work in publications, systems and models that depend on it: checkbox I accept the LFM Open License and the NVIDIA OneWay Noncommercial License terms carried by this model: checkbox extra_gated_button_content: Request access ---

Sori Agent: Audio Understanding via Code Execution over Structured Perception

An audio understanding agent for speech, music and environmental sound. Sori-1B is its core model.

Sori

SNU · Human Interface Lab

Lab Email Homepage GitHub
Seonuk Kim  ·  Human Interface Lab  ·  Seoul National University  ·  🇰🇷
Released 28 August 2026
Sori Agent — *sori* (소리) is Korean for *sound* — is an audio understanding agent, and Sori-1B is its core model. You give it a clip and ask a question in plain language; it listens, writes down what it heard, and answers. Speech, music and environmental sound are one subject to it, and one call is one answer. ## Setup ```bash pip install "transformers>=5.13" "torch>=2.6" soundfile scipy numpy huggingface-cli login # the repository is gated: accept the terms on this page first ``` Everything the model needs travels with the checkpoint (`trust_remote_code=True` loads the processor and the model code). Audio is read with `soundfile` (wav, flac, ogg, and mp3 with libsndfile 1.1+) and resampled to 16 kHz; any sample rate and any length are accepted, and clips are never cut. The model is fp32 end to end, so the answer does not depend on the device or the batch size, and CPU works. ## Usage ```python from transformers import AutoModel, AutoProcessor proc = AutoProcessor.from_pretrained("snkii/Sori-1B", trust_remote_code=True) model = AutoModel.from_pretrained("snkii/Sori-1B", trust_remote_code=True).cuda().eval() sori = model.agent(proc) sori.ask("clip.wav", "What is happening in this recording?") sori.ask("clip.wav", "How many people are speaking?") sori.ask("clip.wav", "What does the second speaker say?") sori.ask("clip.wav", "When does the door slam?") sori.ask("clip.wav", "Is a siren audible?", ["yes", "no"]) sori.ask("clip.wav", "What is heard right after the door?", ["drilling", "chainsaw", "jackhammer", "vacuum cleaner"]) sori.ask(["a.wav", "b.wav"], "Which clip is louder?", ["the first", "the second"]) sori.ask_many([{"audio": "a.wav", "question": "Describe the scene."}, {"audio": "b.wav", "question": "How many speakers are there?"}]) ``` With `choices`, the answer is one of them, verbatim. Without them, it is a sentence, a number or a list — whatever the question asks for. `explain=True` returns the answer together with the steps the agent took to reach it. **Two ways to answer, one preference.** A question that can be computed is answered by code that a real interpreter runs, every step shown and checkable; what cannot be computed — a description, what a speaker meant, a judgement — is answered in language. The computed path needs the `sorilm` package (`pip install git+https://github.com/snkii/sori`); with the checkpoint alone, the agent hears and answers in language. **For people.** A terminal conversation about a file, where plain requests become interpreter lines and are shown: ```bash python -m sorilm.core.chat clip.wav --ckpt snkii/Sori-1B ``` **For agents.** The tool surface is one tool. `sori.tools()` emits its JSON schema — `ask(audio, question, choices?, explain?)` — and a Model Context Protocol server serves it, so a planner can ask about a clip the way it calls any other tool, without prompt engineering and without parsing prose: ```bash python -m sorilm.core.mcp_server --ckpt snkii/Sori-1B # stdio transport ``` ## The model Sori-1B is one architecture from the waveform to the words. Its audio encoder is built from Liquid AI's [LFM2.5-Encoder-350M](https://huggingface.co/LiquidAI/LFM2.5-Encoder-350M), initialised by distillation from the outputs of NVIDIA's [Audio Flamingo Next](https://huggingface.co/nvidia/audio-flamingo-next-hf) encoder and then trained; its language model is [LFM2.5-350M-Base](https://huggingface.co/LiquidAI/LFM2.5-350M-Base), fully trained here. Together they are 933 million parameters. This is a research preview under active development, for academic use; how it is built and evaluated is described in the technical report that accompanies the project. ## License Sori-1B is released for **non-commercial research use**: use it, share it and build on it under the same terms, with attribution. Models, data and systems derived from it or from its outputs carry this license, a notice of what was changed, and a citation of this work; so do publications whose results depend on it. Commercial use is not licensed: the audio encoder was initialised by distillation from the outputs of NVIDIA's Audio Flamingo Next encoder (NVIDIA OneWay Noncommercial License), and part of the training data is licensed for non-commercial use. The Liquid AI models it is trained from remain under the LFM Open License v1.0, reproduced in full in [LICENSE](LICENSE), where the complete terms are. ## Citation ```bibtex @software{kim_sori_agent_2026, author = {Kim, Seonuk}, title = {{Sori Agent: Audio Understanding via Code Execution over Structured Perception}}, year = {2026}, month = aug, institution = {Human Interface Laboratory, Seoul National University}, url = {https://huggingface.co/snkii/Sori-1B} } ```