← Back to Works

JapaneseTinyAgentLM (Japanese tool calling on a microcontroller)

A 3.15M-parameter Japanese language model that runs entirely on an ESP32-S3 and turns Japanese requests into robot action calls (JSON).

Date

October 2026

Role

Design, data, training, on-device implementation, release

Tech

Tiny LM, Tool calling, ESP32-S3, INT4, C, WebAssembly

Overview

A tiny Japanese language model that runs entirely on a microcontroller (ESP32-S3). The first model, Action LM (3.15M parameters), turns short Japanese requests such as “look to the left” into action calls (JSON) such as look(direction=left). It moves the head and changes the facial expression of M5Stack Stack-chan (CoreS3) using only the device itself, with no NPU and no network. The demo runs the same C runtime as the device (compiled to WebAssembly) with the INT4 weights in the browser.

What I did

I generated and checked the training data, trained the model from scratch, quantized it to INT4, wrote the C inference runtime and the ESP32-S3 firmware, built the browser demo, and handled evaluation and release.

Technical notes

• The file written to the device (INT4 weights and tokenizer) is 2.0 MB, and the median response time is 1.3 seconds per sentence.
• Grammar-constrained decoding picks only schema-valid tokens at each step, so the output is always well-formed JSON. When confidence falls below a threshold, the model does nothing.
• It scores 85.2 ± 6.4% (mean of 5 seeds) on human-written requests, triggers no actions on 1,097 human-written non-requests (0.0%), and the device output matches PC (PyTorch) on 300 / 300 inputs.
• The code is Apache-2.0 and the model weights are CC BY-SA 4.0.

Links

Demo (Hugging Face Space)JapaneseTinyAgentLM Action 3M - a Hugging Face Space by ayousanzhuggingface.coGitHubayutaz/JapaneseTinyAgentLMTiny Japanese language models that run entirely on an ESP32-S3. Action LM (3.15M params, INT4) turns Japanese requests into robot actions on M5Stack Stack-chan. - ayutaz/JapaneseTinyAgentLMgithub.comHugging Faceayousanz/JapaneseTinyAgentLM-Action-3Mhuggingface.co