News · 2026-07-14

KronQ makes 2-bit models work where the standard method collapses

A new method called KronQ takes a 70-billion-parameter model compressed to 2 bits per weight from complete gibberish to a working model. Under GPTQ -- the method that has dominated this task since 2022 -- 2-bit LLaMA-3-70B produces a perplexity above 2000, meaning its output is noise. KronQ reaches 7.93. The claim underneath is that the standard approach has been ignoring half the available information for four years.

Key facts

• On 2-bit LLaMA-3-70B, GPTQ and GPTAQ "diverge or produce degenerate quantizations" above 2000 perplexity. KronQ: 7.93.
• The mechanism: folding gradient covariance into the compression objective via a Kronecker-factored approximation of the Hessian.
• Authors: Donghyun Lee, Yuhang Li, Ruokai Yin, and Priyadarshini Panda. Accepted to COLM 2026.
• Paper: arXiv 2607.07964.

Here is the problem in plain terms. Quantization means storing a model's learned numbers more coarsely to shrink it -- 16 bits down to 8, 4, or fewer. The catch is that you cannot just round everything, because some weights matter far more than others and rounding those wrecks the model. So the good methods try to be selective: figure out which weights are sensitive, spend precision there, be sloppy elsewhere.

GPTQ, published in 2022 and the workhorse of the field ever since, does this by examining the activations -- the numbers flowing into each layer. If a weight is multiplied by inputs that are usually large, it matters. If it is multiplied by inputs near zero, it does not. That is a sound instinct, and it works well down to 4 bits, which is why nearly every quantized model you have ever downloaded went through it.

But it carries a hidden assumption. By looking only at what comes in, GPTQ implicitly treats every output channel as equally important. Push down to 2 bits and that assumption shatters -- and "shatters" is not a figure of speech. The paper reports that GPTQ and its successor GPTAQ, applied to a 2-bit 70-billion-parameter model, "diverge or produce degenerate quantizations" with perplexity over 2000. Perplexity roughly measures how surprised a model is by real text; a working model sits in the single digits. Two thousand means it has no idea what language is.

KronQ's fix is to ask a second question: not just what flows into a weight, but how much the model's error changes when that weight moves -- the gradient. Combining both requires a matrix of second derivatives, the Hessian, which for a model this size is astronomically large. So the authors approximate it with a Kronecker factorization, a technique for representing a huge matrix as the product of two much smaller ones.

The analogy: GPTQ is an editor who decides which sentences to cut by counting how often each is read. Sensible, and it catches a lot. But it never asks what breaks if a sentence goes. KronQ asks both -- how much this part is used and how much the whole thing degrades without it. At mild edits the two editors produce similar books. At savage ones, only the second still has a book.

Mechanically, that insight lands in two places. The paper extends the existing input-side random rotation trick to the output dimension using gradient covariance, flattening the spread of weight magnitudes in both directions at once. And it introduces a sensitivity measure -- driven by both gradient and activation Hessian traces -- that decides how many bits each layer deserves, so fragile layers get more.

The honest caveat is substantial. The full text is not readable online yet -- no HTML version has been posted, so this account rests on the abstract, the reported figures, and the paper's acceptance at COLM 2026, which is real peer review by people who could check the math. No third-party reproduction exists. And the abstract does not settle the cost question, which is the one that decides whether anyone uses this: Hessian machinery is not free, and GPTQ's dominance rests heavily on being fast. If KronQ needs ten times the compute to quantize, that is a real trade rather than a free lunch. A claim this strong -- the standard method has been missing half the information since 2022 -- should invite exactly this much skepticism.

What makes it worth watching is the timing. PrismML's Bonsai shipped a 27-billion-parameter model at one bit per weight this week -- 3.9 gigabytes, running on a phone -- and published the table showing tool calling and vision degrading badly. Bonsai is the engineering claim that extreme compression can ship. KronQ is the mathematics for why the next one might not have to break.

Primary source, verified: read the paper → (arXiv 2607.07964)

Ground Truth. (2026, July 14). KronQ makes 2-bit models work where the standard method collapses. Ground Truth. https://groundtruth.day/news/kronq-makes-2-bit-models-work.html

@misc{groundtruth:kronq-makes-2-bit-models-work,
  title  = {KronQ makes 2-bit models work where the standard method collapses},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {jul},
  url    = {https://groundtruth.day/news/kronq-makes-2-bit-models-work.html}
}

Topics: quantization · research · efficiency · llama · colm-2026