News · 2026-11-12

Huawei's open MACaQ quantizer cuts divergence by about 18-23% versus public GGUFs at the same file size

Huawei's Computing Systems Lab has open-sourced MACaQ, a calibrated quantizer that its own blog table says produces 4-bit files with about 18 to 23 percent lower divergence from the original model than two popular public files of the same size. On a Qwen3-4B test measured November 3, MACaQ's IQ4_XS file scores a KL divergence of 0.0280 at 2.26 GB, versus 0.0344 for Bartowski's IQ4_XS (2.27 GB) and 0.0366 for Unsloth's.

Key facts

• What: MACaQ ("macaque"), Apache-2.0, from Huawei's lab in Switzerland, the same group as KVarN; repo at github.com/huawei-csl/MACaQ.
• Paper: arXiv 2611.10649, "MACaQ: Fast, Model Agnostic Calibrated Quantization for Low-Precision Weights and Activations", submitted November 7, 2026 by Lorenz K. Muller and colleagues. We read it only via a summary.
• Best like-for-like result: MACaQ matches Bartowski's Q4_K_M quality (KL 0.0277 at 2.50 GB) in a roughly 10% smaller 2.26 GB file.
• Larger download: the MACaQ-R Qwen3.6-35B-A3B GGUF on Hugging Face is 22.36 GB; no VRAM requirement is stated in the sources we read, and the weights alone occupy about that much (a floor, computed from the file size).

What it does

The name is a pun on "macaque": the README says it is "so easy to use, even a macaque could get the hang of it". Technically, it calibrates one linear layer at a time, so memory stays small. It combines per-channel scaling, small Hadamard transforms (their inverses folded into the weights) and a "Flip-Refinement" step, a Hessian-aware coordinate descent that nudges individual 4-bit codes away from plain round-to-nearest. Its target for weights and activations is MXFP4, called W4A4, and the same recipe is applied to text, image, video, audio and 3D models. The README lists backends for llama.cpp (2-, 3- and 4-bit GGUF), vLLM, Diffusers and PyTorch speech models; Ascend kernels exist only as open pull requests. For background, see our lessons on quantization and model file formats.

The honest part: like-for-like GGUF numbers

Divergence here is KL divergence from the FP16 model's output, where lower is better; see forward and reverse KL. The Qwen3-4B table is a small model and one table, but it measures against named public files at equal size, which is more than most quantization announcements offer.

MACaQ-R, the variant that drew the most attention on Hacker News, refines codes inside an existing GGUF and keeps the stock format. Huawei's card says its Qwen3.6-35B-A3B file reaches KL 0.0134 against 0.0157 for Unsloth's UD-Q4_K_XL at the same 22.36 GB size, about 15% lower. That is real but small, and it compares one baseline on one model.

The headline claims, as Huawei states them

• A 122-billion-parameter model quantized in 64 minutes on 32.5 GB of memory (README).
• Lower perplexity than MR-GPTQ and DuQuant++, or 11 times less quantization time at matched Llama-3-8B perplexity (README and abstract).
• About 2.3% end-to-end overhead over pure MXFP4 for Qwen3-32B in vLLM (arXiv summary; the blog says batch 1, context 512).
• For a 27B Qwen model in vLLM, W4A4 versus BF16 prefill about 2.3x faster and decode about 2.6x faster, with a small perplexity rise from 2.85 to 2.90 (README).

All are self-reported on Huawei's hardware and not independently reproduced.

Reception

Hacker News item 50458166, "Huawei releases MACaQ, its internal quantizer for local inference", had about 119 points when checked (not 142 as an earlier note had it). Comments were mostly macaque puns and one note that people were already reproducing the quants and posting GGUFs on LocalLLaMA; we saw no benchmark disputes. Both the repo and model card ask users to star the repo, and the repo says so even for AI agents.

What to take from it

KL divergence is a proxy: it measures drift from FP16 output, not task accuracy, and none of the sources we read reports downstream accuracy. And MACaQ is one of three quantization items this week that compare to a weaker or narrower baseline. A fourth, a LocalLLaMA post titled "KQuant ... 3.7x better quants" for MLX, has no source we could open; the closest match, asher's mlx-kquant, reports divergence cuts of 1.8x to 2.8x versus MLX affine quants, and does not mention 3.7x, so treat that figure as unsourced. MACaQ's gains are real but modest once the baseline is named, and that is the useful way to read all of them.

Primary source, verified: read the paper → (arXiv 2611.10649)

Ground Truth. (2026, November 12). Huawei's open MACaQ quantizer cuts divergence by about 18-23% versus public GGUFs at the same file size. Ground Truth. https://groundtruth.day/news/huawei-macaq-4-bit-quantizer-cuts-gguf-divergence-about-a-fifth-at-equal-size.html

@misc{groundtruth:huawei-macaq-4-bit-quantizer-cuts-gguf-divergence-about-a-fifth-at-equal-size,
  title  = {Huawei's open MACaQ quantizer cuts divergence by about 18-23% versus public GGUFs at the same file size},
  author = {{Ground Truth}},
  year   = {2026},
  month  = {nov},
  url    = {https://groundtruth.day/news/huawei-macaq-4-bit-quantizer-cuts-gguf-divergence-about-a-fifth-at-equal-size.html}
}

Topics: huawei · quantization · llama-cpp · local-llm · open-source · gguf