SupraTTS 1.0 Preview ONNX conversion and CPU frontend improvements
Hi @arthurblg1802 ,
My name is Jim Grimsby Jr. I’m blind and maintain Sonata Sherpa Voices, an open-source speech driver for the NVDA screen reader. The project makes speech models available to blind and visually impaired users, with particular attention to practical Windows CPU performance, responsiveness and reliable cancellation.
Thank you for creating SupraTTS. It produces impressive speech for such a small model, and I’ve been investigating its suitability for screen-reader use.
Working with ChatGPT, which I call Jack, I’ve completed an experimental ONNX conversion of SupraTTS 1.0 Preview:
- Four FP32 ONNX graphs cover duration prediction, the two acoustic stages and the spectrum decoder. No quantization or reduced precision was used.
- Duration expansion and waveform reconstruction run in NumPy, so playback no longer requires PyTorch.
- Thirteen ordinary speech tests closely matched the original output. Additional tests checked variable input lengths and duration predictions.
- On my Intel i5-8400, standalone warm generation of “Hello” took approximately 10 ms. This measures complete waveform generation, not NVDA’s audio-output latency.
I also investigated the pronunciation fallback. Multithreaded BLAS overhead made some unusual words and long identifiers extremely slow on this system. A model-local adaptation avoids that overhead without changing global threading settings. Its generated phonemes matched the original fallback across 53 tested words.
For longer input, the experimental adapter reuses Sonata’s existing text chunker, then counts the actual generated phoneme tokens and divides at word or clause boundaries within the 207-token limit. Cancellation and subsequent speech recovery have also been tested.
This remains development work, not a released Sonata integration. I can provide the converted models, exporter, adapter source, provenance, benchmark results and generated recordings for your review, retaining the original licenses and attribution.
Would you be interested in reviewing and hosting the ONNX conversion alongside your original model? I would much rather have it hosted upstream with its creator than maintain a separate model distribution.
Thanks again for your work. Small, responsive speech models can make a meaningful difference to people who depend on speech throughout their day.
Best regards,
Jim Grimsby Jr.
Absolutely!
Thank you for your submission.
I would love to check it out and eventually publish it as a mainstream direct convertion.
Hi Arthur,
Thank you for the encouraging response! I’ve prepared a review package for you:
Download the SupraTTS ONNX conversion and supporting files
The archive includes:
- The four FP32 ONNX graphs and pronunciation assets.
- The exporter and instructions for reproducing the conversion.
- A standalone playback example requiring ONNX Runtime and NumPy, without PyTorch.
- The bounded-input and pronunciation-performance improvements.
- Benchmark, numerical-parity and transcription results, plus ten generated recordings.
The conversion retains full precision. On my i5-8400 using two CPU threads, the standalone sentence and paragraph tests generated speech approximately 44–46 times faster than real time. Those are generation measurements, not NVDA audio-startup timings.
For screen-reader use, I added splitting based on actual phoneme counts, cancellation support and a model-local pronunciation optimization that avoids changing global threading settings.
The package documents the testing conditions and limitations, including the distinction between standalone tests and isolated tests through Sonata’s actual driver and helper. Licensing and attribution are included.
I would be very pleased for you to review the work and, if suitable, host the conversion alongside your model. Please let me know if you need anything else. Once hosted, the final download URLs would let me connect it to Sonata’s model catalog.
Thank you again for creating another lightweight speech option for blind users.
Best regards,
Jim Grimsby Jr.
Alright !
I'm currently not home, so I do not have access to my computer to review everything and upload.
So I will have to ask you to wait until between Saturday and Monday for it to be uploaded.
Thank you for your time.