georgeven/songmae-large-32x1-bird-detector

SongMAE-Large 32x1 bird detector

A fine-tuned SongMAE-Large 32x1 encoder (96.5M parameters) with a linear head predicts, for every 5-ms frame and each of 128 mel bins (20–16,000 Hz), the probability that the bin belongs to a bird vocalization.

• SongMAE model collection
• Code: github.com/georgevenven/birdsong-detect-distill

Example

Black Wheatear (Oenanthe leucura), XC839867, not used in training. Top: mel spectrogram. Bottom: regions detected by this model. Recording by Esperanza Poveda, licensed CC BY-NC-SA 4.0; this image is an adaptation under the same license.

Training

The detector was trained without human labels. A vision-language model teacher, Qwen3.8-27B (Q8_0, llama.cpp), drew bounding boxes on 5-second viridis spectrograms of Xeno-Canto recordings (BirdSet XCL) using coordinate axes, chain-of-thought reasoning, and one numbered self-review pass.

Results

Model	WABAD AP	WABAD IoU	Hawaii AP	Hawaii IoU	XC-AJ AP	XC-AJ IoU	NIPS4Bplus AP	NIPS4Bplus IoU
BirdCODE	—	—	—	—	0.836	0.526	0.657	0.545
YOLO11n (human labels)	0.552	0.347	0.541	0.369	0.700	0.546	0.630	0.455
YOLO11l (human labels)	0.549	0.354	0.546	0.381	0.676	0.556	0.590	0.464
YOLO11n (teacher labels)	0.566	0.348	0.619	0.412	0.769	0.512	0.749	0.468
YOLO11l (teacher labels)	0.568	0.359	0.604	0.400	0.774	0.518	0.748	0.532
SongMAE-Large (teacher labels)	0.633	0.385	0.716	0.479	0.836	0.529	0.777	0.488

Localization on held-out test sets (Table 2 of the paper). WABAD and Hawaii: time-frequency localization (pixel-level AP and IoU); XC-AJ and NIPS4Bplus: onset-offset localization (frame-level). Models trained on teacher labels: means across three seeds. Bold: best; underline: second best.

Usage

Install

pip install torch "transformers>=4.36,<5" librosa scipy safetensors huggingface_hub
hf download georgeven/songmae-large-32x1-bird-detector detect.py --local-dir .

Command line

python detect.py recording.wav --repo georgeven/songmae-large-32x1-bird-detector

Option	Default	Description
--seed	0	Checkpoint to use (0, 1, or 2)
--threshold	calibrated per seed	Probability threshold for the mask
--out	predictions/	Output folder
--device	cuda if available	cuda or cpu

Outputs

File	Contents
predictions/recording.npz	probability and mask, 128 mel bins × 5-ms frames
predictions/recording.csv	One row per detected region: start_s, end_s, low_hz, high_hz, peak_probability

Python

import librosa
from detect import load, probabilities, regions

backbone, head, config = load("georgeven/songmae-large-32x1-bird-detector", device="cuda")
waveform, _ = librosa.load("recording.wav", sr=32000, mono=True)
probability = probabilities(waveform, backbone, head, config, device="cuda")  # (128, frames)
events = regions(probability, config["threshold"], config)  # [(start_s, end_s, low_hz, high_hz, peak), ...]