yousansanoTTS-jp (Japanese TTS on a microcontroller)
A 559K-parameter Japanese TTS that synthesizes faster than real time entirely on an ESP32-S3, including on-device reading and pitch-accent estimation for kanji text.
Overview
I applied the distillation recipe from the sanoTTS paper (arXiv:2608.21378) to Japanese, using my own TTS, piper-plus, as the teacher, and built a 559K-parameter Japanese TTS. After flashing the firmware to an M5Stack CoreS3 (Stack-chan), you type “今日は良い天気ですね。” over serial and it speaks. The demo runs the same C99 code as the microcontroller, compiled to WebAssembly, in the browser.
What I did
I handled everything from teacher-label generation and distillation into three small models (duration, acoustic and waveform) to the dependency-free C99 inference core, on-device G2P, the ESP32-S3 firmware and Arduino / PlatformIO library, on-device measurement and release. I wrote it independently from the paper and the piper-plus code, without referring to the official implementation (GPL-3.0).
Technical notes
- The weights are about 654 KB in int8. Inference depends only on libm and never calls
malloc. - On the device it runs at a real-time factor (xRT) of 0.473, starts playing within 407–433 ms, and had zero underruns across all test sentences.
- Porting the English recipe as-is breaks on reading kanji, so the device takes “hiragana plus accent marks” and converts them to phonemes with an 877-byte table. A TTS-specific dictionary format (13.7 MB) later made it possible to read kanji on the device alone on 16 MB boards.
- This is a proof of concept, not a product. The code is MIT; the model weights are under a separate, non-MIT license.
