News · 2026-07-14

Bonsai puts a 27B model on your phone, and shows what breaks

PrismML has released Bonsai, a version of the 27-billion-parameter Qwen3.6 model compressed to a single bit per weight -- 3.9 gigabytes instead of 54, running at roughly 11 tokens per second on an iPhone 17 Pro. More unusually, PrismML published the benchmark table showing exactly what the compression costs: instruction following drops from 78.47 to 65.74, tool calling from 80.00 to 66.03, and vision from 72.61 to 59.57.

Key facts

• 1-bit build: 3.9 GB, down from 54 GB at full precision -- roughly a fourteenfold reduction. A 1.58-bit ternary build lands at 5.9 GB.
• Roughly 11 tokens per second on an iPhone 17 Pro.
• The vendor's own table shows tool calling falling from 80.00 to 66.03 and vision from 72.61 to 59.57.
• Released by PrismML, built on Qwen3.6 27B.

A quick word on what "1-bit" means, because it sounds impossible. Normally each of a model's weights -- the numbers it learned during training -- is stored as a 16-bit floating point value, capable of expressing fine gradations. Quantization is the practice of storing them more coarsely: 8 bits, 4 bits, fewer. At one bit, a weight can be only two things. Positive or negative. On or off. The ternary variant allows a third state, zero, which is where the odd "1.58 bits" comes from -- three possible values works out to about 1.58 bits of information.

The intuition for why this can possibly work: a very large model is enormously redundant, and what matters is less the precise magnitude of any individual weight than the overall pattern across billions of them. It is like a photograph reduced to pure black and white with no greys. Up close, every pixel has lost almost all its information. Step back, and the face is still recognizable -- because the face was never in the individual pixels.

The number that lands is the size. Fifty-four gigabytes will not fit on your phone; it barely fits on most laptops. Three point nine gigabytes fits on a phone with room to spare, alongside your photos. A model of a size that until recently meant renting a server now sits in your pocket, offline, answering without asking anyone's permission or sending anything anywhere. Eleven tokens per second is around reading pace -- comfortable for conversation, slow for a long document.

What makes this release worth writing about, though, is the table. PrismML's own materials state that "agentic coding (long-horizon, multi-file, run-test-and-repair workflows) is not yet a strong target of this release." A vendor documenting its own capability cliff, in the launch materials, is rare enough to note.

And the cliff is precisely placed. Compression did not degrade the model evenly. It degraded exactly the abilities that on-device agent work requires. Following instructions: down roughly thirteen points. Calling tools: down fourteen. Understanding images: down thirteen. Those are not incidental capabilities. They are the entire premise of a model that lives on your phone and does things for you. A pocket model that cannot reliably follow an instruction or call a function is a very compact conversationalist, which is a demo rather than a deployment.

That tension is what the practitioner community is actually chewing on. On r/LocalLLaMA, the consensus thread on 1-bit models shows a community that has moved decisively from "is one bit even possible?" to "where exactly does it break?" -- with Bonsai's table as the reference point. That is a healthier question, and they are asking it faster than vendors are answering it.

The research world is converging on the same problem from the other side. KronQ, accepted at a major conference this year, shows that the standard method for extreme compression collapses entirely at 2 bits on a 70-billion-parameter model -- and that adding gradient information to the process rescues it. Bonsai is the engineering claim that ultra-low-bit models can ship. KronQ is the mathematics suggesting why they might get genuinely good rather than merely small.

The honest caveat: every number here is PrismML's, unreproduced by anyone else, and the comparison baseline is PrismML's choice. The 11 tokens per second figure is on the newest iPhone, which is not most phones.

But there is a reason the local community reads this alongside the week's privacy failures -- another r/LocalLLaMA thread ties Bonsai directly to the Grok Build tool caught uploading whole repositories to cloud storage. For a growing number of practitioners, running the model yourself stopped being a hobby and became a security position. Bonsai is the first credible answer at this size, capability cliff and all.

Primary source, verified: read the paper →

Ground Truth. (2026, July 14). Bonsai puts a 27B model on your phone, and shows what breaks. Ground Truth. https://groundtruth.day/news/bonsai-puts-a-27b-model-on-your-phone-and-shows-what-breaks.html

@misc{groundtruth:bonsai-puts-a-27b-model-on-your-phone-and-shows-what-breaks,
  title  = {Bonsai puts a 27B model on your phone, and shows what breaks},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/news/bonsai-puts-a-27b-model-on-your-phone-and-shows-what-breaks.html}
}

Topics: quantization · on-device · open-weights · local-llm · efficiency