yousanRalomi (Japanese speech recognition on a microcontroller)
A 2.05M-parameter speech recognition model that turns Japanese speech into hiragana, running entirely on an M5Stack CoreS3 (ESP32-S3) with no PC or network.
Overview
A lightweight Japanese speech recognition model built for microcontrollers and embedded devices. It converts 16 kHz audio into normalized hiragana. After flashing a single file to an M5Stack CoreS3 (Stack-chan), you tap the screen and speak, and the recognized hiragana appears on the display. The demo converts speech to hiragana inside the browser.
What I did
I handled the model design, training from scratch on about 10,000 hours of ReazonSpeech, quantization for the ESP32-S3, the firmware, the Python inference code, the browser demo, evaluation and release.
Technical notes
- It uses a DFSMN + CTC architecture. The FP32 ONNX model is about 8.3 MB, and the 16-bit quantized (W16A16) model for the ESP32-S3 is about 4.3 MB.
- The quantized model stays within 0.5 points of FP32 on 300 utterances each from Common Voice 8.0 and ReazonSpeech. 8-bit quantization lost too much accuracy and was not adopted.
- It recognizes while recording in 1-second chunks, so results appear about 0.5 seconds after you stop speaking.
- Output is hiragana only, with no kanji conversion or punctuation. The character error rate on the validation set (1,319 utterances) is 21.03%. The weights and code are Apache-2.0.
