News · 2026-07-26

An $8 Microcontroller Now Runs a 28.9-Million-Parameter Language Model

A developer has run a 28.9-million-parameter language model on an ESP32-S3, a Wi-Fi microcontroller that costs about $8 and has 512KB of fast internal memory. The previous best on comparable hardware was around 260 thousand parameters, roughly a hundred times smaller. It works because most of the model never enters fast memory at all: about 25 million of its parameters sit in flash storage and are sampled a few hundred bytes at a time.

Key facts

• 28.9 million stored parameters on a board with 512KB of SRAM, 8MB of external PSRAM, and 16MB of flash.
• About 9.5 tokens per second end to end; the packed 4-bit model file is 14.9MB.
• Roughly 450 bytes of the 25-million-parameter flash table are read per generated token.
• Primary source: the slvDev/esp32-ai repository, MIT licensed, which had passed 1,400 stars at the time of writing.

The developer's own framing is refreshingly free of hype: "This is a 28.9 million parameter language model that generates text on an ESP32-S3, a microcontroller that costs about $8. It runs on the chip itself, with nothing sent to a server." And then, immediately: "It will not answer questions, follow instructions, write code, or know facts."

Here is why that combination is interesting rather than contradictory. Normally, a model has to be reachable from fast memory, which is what caps microcontroller models at a few hundred thousand parameters. But not all parameters are used the same way. Most of a small language model's weights live in an embedding table, which the model reads from rather than computes with. It is the difference between a chef's knives, which must be within arm's reach every second, and a pantry, which only needs to be walkable-to when a specific ingredient is required. Nobody rents a bigger kitchen so the pantry can sit on the counter.

So the pantry goes in flash. The repository puts it plainly: "you can leave that 25 million row table in slow flash and pull only the few rows each token needs, about 450 bytes, while the small part that does the actual work stays in fast memory. The large model then costs almost nothing to run, because you never load most of it." The technique is Google's Per-Layer Embeddings, used in the Gemma 3n and Gemma 4 model families, where it was designed for phones and GPUs. The novelty here is applying it to the three-tier memory layout of a microcontroller - fast SRAM for the computing core, medium PSRAM for the output layer, huge slow flash for the table. As the developer notes: "As far as I can tell, nobody had tried it on a chip this small."

The measured result contains a correction that most quick takes missed. The 25-million-parameter flash table is not the speed problem. Reading its six rows costs about 0.12 milliseconds per token. The expensive step is the output layer - the part that scores all 32,768 possible next words - which has to be scanned out of the slower external PSRAM at roughly 17 milliseconds per token. The model's capacity is nearly free; picking the next word is what costs. That is a useful corrective for anyone reading "flash makes inference cheap" as a general law, and it is the same memory-bandwidth constraint that governs inference on hardware costing a hundred thousand times more.

The repository is unusually honest about its own history. Training code, quantization, ablations, the C runtime, the flash partition map and the firmware build steps are all published, and the host runtime is checked against all 32,768 PyTorch logits before flashing. The developer also left the mess in on purpose: "That includes a bug I found in my own parameter accounting, which had inflated an early number, and the corrected result that followed once I fixed it." The controlled experiment behind the design is small but real - at the same fast-memory budget, the flash-table version scored meaningfully better on held-out story text than an equivalent dense model.

The Hacker News discussion, which reached 270 points, produced the strongest counter-argument: for roughly the same money you can buy a Linux-capable single-board computer with vastly more RAM and a neural accelerator, and build a far more useful local-AI product. That critique is correct about product selection and beside the point about the result. The claim being made is specifically about a Wi-Fi microcontroller with half a megabyte of fast memory and no operating system, and whether it can do autoregressive text generation at all if the model is redesigned around its memory hierarchy.

The honest caveat is that this remains one person's self-reported experiment, published with reproduction instructions but not yet independently rebuilt on the specified board, and the capability ceiling is set by TinyStories, the synthetic children's-story dataset it trained on. Nothing here is a tiny assistant. What it does establish is a design principle with legs: a microcontroller can hold far more learned state than it can actively compute over, provided that state is structured as sparse conditional memory. Pair it with the recent run of results pushing the local inference floor downward, and the question stops being how small a model can get and becomes how cheap the silicon underneath it can get.

Primary source, verified: read the paper →

Ground Truth. (2026, July 26). An $8 Microcontroller Now Runs a 28.9-Million-Parameter Language Model. Ground Truth. https://groundtruth.day/news/an-8-dollar-microcontroller-now-runs-a-28-million-parameter-model.html

@misc{groundtruth:an-8-dollar-microcontroller-now-runs-a-28-million-parameter-model,
  title  = {An $8 Microcontroller Now Runs a 28.9-Million-Parameter Language Model},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/news/an-8-dollar-microcontroller-now-runs-a-28-million-parameter-model.html}
}

Topics: on-device · edge-ai · open-source · small-models · memory