July 25, 2026
Small chip, huge main character energy
Running a 28.9M parameter LLM on an $8 microcontroller
Tiny $8 chip writes stories by itself, and commenters are losing it
TLDR: A developer got a tiny text generator running fully offline on an $8 microchip, which is a big deal because devices this cheap usually can’t do anything close to that. Commenters were split between pure amazement at the speed and bigger dreams about turning the trick into talking gadgets or even scaling it to larger systems.
An $8 chip that usually ends up inside cheap gadgets is suddenly the star of a full-on "wait, it can do WHAT?" moment. The maker says they got a tiny language model — basically a baby version of the text generators powering chatbots — to run entirely on the device, with no internet at all, spitting out little story words onto a tiny screen at about 9 words-ish per second. For the community, the biggest shock wasn’t even that it works. It was that it works that fast. One commenter flat-out marveled that 9.7 tokens per second "actually seems like a lot," which became the thread’s unofficial mood: impressed, delighted, slightly disbelieving.
The applause was loud — lots of "super cool" and "thanks for sharing" energy — but the spicy bit came from the people instantly trying to stretch this into something bigger. One commenter jumped straight to the sci-fi future: if similar tiny models can handle speech, could a no-network ESP32 become a pocket device that reads to you in real time? Another pushed the obvious next question: if this trick works on a bargain-bin chip, why not scale it up and run much larger models off regular computers using storage instead of expensive memory? That’s where the nerd drama kicks in: is this a clever party trick, or the beginning of a new low-cost AI playbook? Either way, the comments read like a live crowd watching someone cram a piano into a backpack and then asking if it can also sing.
Key Points
- •The article describes a 28.9M-parameter language model running entirely on an $8 ESP32-S3 microcontroller with no server connectivity.
- •The system achieves about 9.5 tokens per second end to end, with a 14.9MB 4-bit model footprint.
- •About 25 million parameters are stored in flash as a lookup table, while smaller active components remain in SRAM and PSRAM.
- •The design adapts Google's Per-Layer Embeddings from Gemma 3n and Gemma 4 to microcontroller memory constraints.
- •The model was trained on TinyStories, so it generates short simple stories but does not support instruction following, coding, question answering, or factual recall.